Show HN: LLM-Use – An LLM router that chooses the right model for each prompt(github.com)
github.com
Show HN: LLM-Use – An LLM router that chooses the right model for each prompt
https://github.com/JustVugg/llm-use
2 comments
Does it work for non-API access to Claude and ChatGPT? For example, I access ChatGPT through an account with Plus level access.
Hi, thanks for the question! The project works exclusively through official APIs for interacting with models like ChatGPT and Claude. It requires API access with valid keys and does not support non-API usage like logging into ChatGPT Plus via the web interface or similar methods.
If you want to use it, you’ll need to have API credentials from the providers.
Feel free to ask if you want more details!
If you want to use it, you’ll need to have API credentials from the providers.
Feel free to ask if you want more details!
I built *LLM-Use*, an open-source intelligent router that helps reduce LLM API costs by automatically selecting the most appropriate model for each prompt.
I created it after realizing I was using GPT-4 for everything — including simple prompts like “translate hello to Spanish” — which cost $0.03 per call. Models like Mixtral can do the same for $0.0003.
### How it works: - Uses NLP (spaCy + transformers) to analyze prompt complexity - Routes to the optimal model (GPT-4, Claude, LLaMA, Mixtral, etc.) - Uses semantic similarity scoring to preserve output quality - Falls back gracefully if a model fails or gives poor results
### Key features: - Real-time streaming support for all providers - A/B testing with statistical significance - Response caching (LRU + TTL) - Circuit breakers for production stability - FastAPI backend with Prometheus metrics
### Early results: - Personal tests show up to 80% cost reduction - Output quality preserved (verified via internal A/B testing)
### Technical notes: - 2000+ lines of Python - Supports OpenAI, Anthropic, Google, Groq, Ollama - Complexity scoring: lexical diversity, prompt length, semantic analysis - Quality checks: relevance, coherence, grammar
Repo: [https://github.com/JustVugg/llm-use](https://github.com/JustVugg/llm-use)
Thanks! Happy to answer questions.