ochafik

no profile record

Submissions

Llama.cpp now supports tool calling (OpenAI-compatible)

3 pointsby ochafikw zeszłym roku1 comments

comments

w zeszłym rokudiscuss

Thanks for the kind words!

I've indeed done all that on my spare time (still under Google copyright), very happy to see this used and appreciated :-)

About to start a new job / unsure if I'll be able to contribute more, but it's been a lovely ride! (largely thanks to the other contributors and ggerganov@ himself!)

ochafik

w zeszłym rokudiscuss

Supports 8+ models’ native formats (Llama, Functionary, Hermes, Mistral, Firefunction, DeepSeek) + generic fallback for all other models.

Runs anywhere; on a Mac:

  brew install llama.cpp
  llama-server --jinja -fa -hf bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M

Still fresh / lots of bugs to discover, feedback welcome!

ochafik

w zeszłym rokudiscuss

When there's a will, there's a way.

It's a bit silly but I rolled out my own(*) "no-deps" C++ Jinja template library (https://github.com/google/minja) just to add tool call support to llama.cpp (https://github.com/ggerganov/llama.cpp/pull/9639).

(*): I mean, technically my employer's :-D*