I started working on Alki because wrangling and deploying AI models on edge devices felt clunky. Going from a Hugging Face model to something production-ready required too many manual steps. Alki aims to simplify that by offering more conversion options, packaging models into reproducible bundles, and making them easy to publish or run on edge hardware.
Llama.cpp was a great starting point, but it led me down the rabbit hole of building a toolchain that could handle different quantization methods, export formats, and deployment bundles for orchestration.
It’s still early and I’d love feedback from infra engineers and startups running LLMs outside the cloud, especially around packaging formats, performance, and what would make the biggest quality-of-life improvements.
Congrats Recall team, I've been a customer for the past 1.5 years and the entire infra layer behind meetings has let our company really focus on "what makes our beer taste better" instead of having to worry about building universal support for different (and tedious) platforms like Teams and Zoom. Eager to give the desktop recording sdk a try soon.
I started working on Alki because wrangling and deploying AI models on edge devices felt clunky. Going from a Hugging Face model to something production-ready required too many manual steps. Alki aims to simplify that by offering more conversion options, packaging models into reproducible bundles, and making them easy to publish or run on edge hardware.
Llama.cpp was a great starting point, but it led me down the rabbit hole of building a toolchain that could handle different quantization methods, export formats, and deployment bundles for orchestration.
It’s still early and I’d love feedback from infra engineers and startups running LLMs outside the cloud, especially around packaging formats, performance, and what would make the biggest quality-of-life improvements.