HackerTrans
トップ新着トレンドコメント過去質問紹介求人

xyc

no profile record

コメント

xyc
·11 か月前·議論
I'm running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok/s for 20B model and 12 tok/s for 120B model. I'd say it's pretty usable.
xyc
·昨年·議論
For TypeScript you can refer to https://github.com/modelcontextprotocol/typescript-sdk/blob/...

There isn't much documentation available right now but you can ask coding agent eg. Claude Code to generate an example.
xyc
·昨年·議論
Great to see more local AI tools supporting MCP! Recently I've also added MCP support to recurse.chat. When running locally (LLaMA.cpp and Ollama) it still needs to catch up in terms of tool calling capabilities (for example tool call accuracy / parallel tool calls) compared to the well known providers but it's starting to get pretty usable.
xyc
·昨年·議論
It's a protocol that doesn't dictate how you are calling the tool. You can use in-memory transport without needing to spin up a server. Your tool can just be a function, but with the flexibility of serving to other clients.
xyc
·昨年·議論
recurse.chat + M2 max Mac
xyc
·昨年·議論
I recently discovered toolhive which is pretty handy too https://github.com/stacklok/toolhive
xyc
·昨年·議論
If you are on a Mac, give https://recurse.chat/ a try. As simple as download the model and start chatting. Just added the new multimodal support in LLaMA.cpp.
xyc
·昨年·議論
Actually this is a good way to find product ideas. I placed a query in Grok to find posts about what people want, similar to this. Then it performs multiple searches on X including embedding search, and suggested people want stuff like tamagotchi, ICQ etc. back.
xyc
·昨年·議論
It seems that this is possibly not necessary, since LLaMA.cpp already integrates Jinja with CPP implementation (through minja)
xyc
·2 年前·議論
Check out https://recurse.chat/