HackerTrans
TopNewTrendsCommentsPastAskShowJobs

xyc

no profile record

comments

xyc
·hace 11 meses·discuss
I'm running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok/s for 20B model and 12 tok/s for 120B model. I'd say it's pretty usable.
xyc
·el año pasado·discuss
For TypeScript you can refer to https://github.com/modelcontextprotocol/typescript-sdk/blob/...

There isn't much documentation available right now but you can ask coding agent eg. Claude Code to generate an example.
xyc
·el año pasado·discuss
Great to see more local AI tools supporting MCP! Recently I've also added MCP support to recurse.chat. When running locally (LLaMA.cpp and Ollama) it still needs to catch up in terms of tool calling capabilities (for example tool call accuracy / parallel tool calls) compared to the well known providers but it's starting to get pretty usable.
xyc
·el año pasado·discuss
It's a protocol that doesn't dictate how you are calling the tool. You can use in-memory transport without needing to spin up a server. Your tool can just be a function, but with the flexibility of serving to other clients.
xyc
·el año pasado·discuss
recurse.chat + M2 max Mac
xyc
·el año pasado·discuss
I recently discovered toolhive which is pretty handy too https://github.com/stacklok/toolhive
xyc
·hace 2 años·discuss
Check out https://recurse.chat/