I've been working on Red Candle, a Ruby gem that runs LLMs (Llama, Mistral, Gemma, Phi) directly in your Ruby process through Rust bindings (based on the candle crate from Hugging Face). No Python, no servers - just FFI with Metal/CUDA acceleration.
It's been useful for adding AI features to Rails apps without the complexity of managing separate services. Would love feedback from anyone working with LLMs in Ruby.
110M to 50 firms over 5 years is about 440k/year/firm. That’s roughly 2, maybe 3 FTEs on average. Don’t get me wrong, I’m not a fan and averages can hide a lot, but that’s not exactly swimming in foreign money either.
It's been useful for adding AI features to Rails apps without the complexity of managing separate services. Would love feedback from anyone working with LLMs in Ruby.
GitHub: https://github.com/assaydepot/red-candle