ochafik

no profile record

Submissions

Llama.cpp now supports tool calling (OpenAI-compatible)

3 pointsby ochafikปีที่แล้ว1 comments

comments

ปีที่แล้วdiscuss

Thanks for the kind words!

I've indeed done all that on my spare time (still under Google copyright), very happy to see this used and appreciated :-)

About to start a new job / unsure if I'll be able to contribute more, but it's been a lovely ride! (largely thanks to the other contributors and ggerganov@ himself!)

ochafik

ปีที่แล้วdiscuss

Supports 8+ models’ native formats (Llama, Functionary, Hermes, Mistral, Firefunction, DeepSeek) + generic fallback for all other models.

Runs anywhere; on a Mac:

  brew install llama.cpp
  llama-server --jinja -fa -hf bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M

Still fresh / lots of bugs to discover, feedback welcome!

ochafik

ปีที่แล้วdiscuss

When there's a will, there's a way.

It's a bit silly but I rolled out my own(*) "no-deps" C++ Jinja template library (https://github.com/google/minja) just to add tool call support to llama.cpp (https://github.com/ggerganov/llama.cpp/pull/9639).

(*): I mean, technically my employer's :-D*