HackerTrans
TopNewTrendsCommentsPastAskShowJobs

dataversity

no profile record

Submissions

Show HN: Run Hugging Face models with a single command

llmpm.co
2 points·by dataversity·há 4 meses·3 comments

comments

dataversity
·há 4 meses·discuss
The goal is to make working with local models easier for developers, especially when experimenting with different models or sharing setups with others.

Would love to hear back from the community
dataversity
·há 4 meses·discuss
Some of the features llmpm supports today:

• Install and run open-source models (almost all Hugging Face models)

• Package models with projects so environments are reproducible

• Serve models through an OpenAI compatible API

• Built-in chat UI for interacting with running models

• CLI workflow designed for developers

• Model benchmarking and rankings to help choose models Example:

llmpm serve llama3

This exposes an API endpoint similar to:

http://localhost:8080/v1/chat/completions
dataversity
·há 4 meses·discuss
Hi folks, I’ve been working on llmpm, a CLI tool that makes open-source LLMs installable like packages. The idea came from the frustration of trying to run different models locally. Every model tends to have slightly different instructions, dependencies, or runtimes. I wanted something that worked more like npm or pip, install a model and run it immediately. With llmpm you can do things like:

llmpm install llama3

llmpm run llama3
dataversity
·há 4 meses·discuss
[dead]
dataversity
·há 4 meses·discuss
Tried this out this morning. It’s a pretty powerful CLI tool for developers working with open-source LLMs.
dataversity
·há 4 meses·discuss
Also, is there is way I can invoke the models or is there an API which your tool exposes?
dataversity
·há 4 meses·discuss
Looks crazy! Thanks for building it, there is indeed a need for a npm or pip like package manager for AI Models.