Gemma 2 on AWS Lambda with Llamafile(unremarkable.ai)
unremarkable.ai
Gemma 2 on AWS Lambda with Llamafile
https://www.unremarkable.ai/serverless-ai-inference-with-gemma-2-using-mozillas-llamafile-on-aws-lambda/
3 comments
This is great work. Has anyone used it enough to compare the lambda costs with the cost of running a comparable model on, say, OpenAI?
A small experiment to see if we are there yet with highly virtualized CPU compute and Small Language Models (SLM). The answer is a resounding maybe, but most likely not. Huge thanks to Justine for her work on Llamafile supported by Mozilla. Hope folks find this R&D useful.
Can you expound a bit about why not?
Does it produce bad results? Is it slow to respond? Slow to load?
I've been wanting to play around with llamafile-based edge functions but storing even small models in GitHub (for automated deploys) is a terrible and often impossible experience.
Does it produce bad results? Is it slow to respond? Slow to load?
I've been wanting to play around with llamafile-based edge functions but storing even small models in GitHub (for automated deploys) is a terrible and often impossible experience.