Sharing because I was wondering if OpenAI would built a bot in the future and it seems to me having spacial audio on the model could provide a hint at that.
A small experiment to see if we are there yet with highly virtualized CPU compute and Small Language Models (SLM). The answer is a resounding maybe, but most likely not. Huge thanks to Justine for her work on Llamafile supported by Mozilla. Hope folks find this R&D useful.
Yup, this seems right. You pay for tokens no matter what. Even in other APIs. Did you know you can set an expire for files, vector stores, etc? No need to pay for long term storage on those. Also, threads are free.
I have some assumptions/guesses on how billing works. Gonna do a post on this on my unremarkable.ai blog, please do signup for posts there, no spam. I could be right or wrong but need to do some experiments and publish later.
Great points. Dont even get me started about how function calling in other LLMs costs me tokens. Something OpenAI provides OOTB. I'm also not a big fan of OpenAI's lock in. Right now I'm on a huge Claude 3 Haiku kick. That said, OpenAI does seem to get the APIs right and my hunch is the new Assistants API is going to potentially disrupt things again. Time will tell.
Thanks @beoberha, I am too. I like one take I heard on Twitter. The sentiment was something like these types of systems are useful under the AI-Powered Productivity industry which has incremental gains, no big bangs. Said another way, if your job was to help a TON of your employees be more productive individually, it is worth it because companies measure those efforts broadly and the payoff is there. But again, not big. My advice for folks to stay lower level and hook AI automation up with simple, closed loop, LLM patterns that feel more like basic API calls in a choreographed manner. OMG, hope all that made sense
Glad Google is catching up with Ruby. But if you really want to learn how I am doing this with AWS and how easy it is to handle Rails on Lambda, check out the Lamby work here. https://lamby.custominktech.com
Many of my users were friends I met at various Ruby/Rails development circles. HomeMarks is a passion project that I have strived to re-write every few years. I threw away ~7K lines of custom MV* JS (before there were any) written in Prototype.js to create this new version in Rails/SpineJS. I hope others find it useful. History here.
I never liked Rails templates because I felt the quality of the final product was never on par with what you would setup from scratch. To that end, I created my own system that I call a Rails application prototype. If you are interested you can learn more about it here.
I can see both sides of the argument here. My application prototype is really slim and focuses on bootstrapping an application with a major emphasis on testing tools. Even tho it has a JavaScript MVC framework (Spine.JS) it will remove all traces depending on your setup questions. This approach to a lean prototype means it can easily be updated like a normal Rails application and changed/forked as needed. Best of both worlds hopefully.