Llamafile v0.8.14: a new UI, performance gains, and more(hacks.mozilla.org)
hacks.mozilla.org
Llamafile v0.8.14: a new UI, performance gains, and more
https://hacks.mozilla.org/2024/10/llamafile-v0-8-14-a-new-ui-performance-gains-and-more/
3 comments
I believe that on-device models will become a significant trend. I previously released a local file manager app called Riffo, and we have been using the OpenAI API since its launch. However, our users have informed us that they need on-device models. This indicates to me that it's an important issue.
We have been developing an on-device model recently. For macOS, we are using Ollama to host the on-device model on Mac. We have not yet found a perfect solution for the Windows version. Thank you for sharing Llamafile; we might test it for the Windows on-device model.
[deleted]