Show HN: Deploy Hugging Face Models to AWS Lambda(github.com)

1 pointsby cnussvor 2 Jahren0 comments

Show HN: Deploy Hugging Face Models to AWS Lambda

https://github.com/scaffoldly/scaffoldly-examples/tree/python-huggingface

I've been working on Scaffoldly since 2020 to simplify AWS Lambda deployments. Recently discovered you can run Hugging Face models efficiently using EFS for caching. Here's what's interesting:

   - Uses EFS for model file persistence
   - Pre-downloads models after deployment for faster cold starts
   - Cold start: ~20s (model loading), warm requests: 5-20s (CPU inference)
   - Fully automated container builds and deployment
   - Works with private/gated models via HF_TOKEN

Example deployment:

  npx scaffoldly create app --template python-huggingface
  cd python-huggingface && npx scaffoldly deploy

Scaffoldly is Open Source and I'm excited for all feedback and contributions from the community!

https://github.com/scaffoldly/scaffoldly

https://github.com/scaffoldly/scaffoldly-examples/tree/pytho...

0 comments

—