Show HN: Humiris – Next-Gen AI Mixture Layer to Build Advanced Applications
16 comments
This looks like an amazing tool for businesses leveraging AI. How does it compare in cost-effectiveness to deploying single LLMs on custom infra?
Our technology is typically more cost-effective than deploying single LLMs on custom infrastructure due to several key factors:
Optimized Resource Use: Humiris dynamically routes each task to the most suitable model, ensuring you only use the necessary resources and avoid paying for unused capacity.
Multi-Model Efficiency: By leveraging multiple specialized LLMs, Humiris handles diverse tasks more efficiently than a one-size-fits-all model, improving performance without the need for overprovisioning.
Lower Operational Costs: As a managed platform, Humiris reduces the expenses and complexities associated with maintaining your own infrastructure, including updates, scaling, and monitoring.
Scalability: Humiris seamlessly scales with your business needs, allowing you to handle increased workloads without the significant costs tied to scaling single-model setups.
Long-Term Savings: Intelligent model selection and resource optimization lead to predictable and reduced costs over time compared to the high initial and ongoing expenses of single LLM deployments.
Optimized Resource Use: Humiris dynamically routes each task to the most suitable model, ensuring you only use the necessary resources and avoid paying for unused capacity.
Multi-Model Efficiency: By leveraging multiple specialized LLMs, Humiris handles diverse tasks more efficiently than a one-size-fits-all model, improving performance without the need for overprovisioning.
Lower Operational Costs: As a managed platform, Humiris reduces the expenses and complexities associated with maintaining your own infrastructure, including updates, scaling, and monitoring.
Scalability: Humiris seamlessly scales with your business needs, allowing you to handle increased workloads without the significant costs tied to scaling single-model setups.
Long-Term Savings: Intelligent model selection and resource optimization lead to predictable and reduced costs over time compared to the high initial and ongoing expenses of single LLM deployments.
Met the guys in sf, the concept is great, can we really build o1-type of models with this? Congratulations on the launch.
How does the routing model work under the hood? Is it based on reinforcement learning or traditional decision trees?
Great question!
Our gating model(the heart) in Humiris' system functions as an intelligent gating mechanism that dynamically selects and orchestrates multiple large language models (LLMs) based on predefined parameters such as cost, performance, privacy, and speed. While the exact implementation specifics can vary, here’s how it might work under the hood:
The gating model evaluates each query and the available models' characteristics to determine the optimal routing. This involves dynamically assigning weights to multiple parameters and scoring models based on the query's requirements. The architecture combines elements of both machine learning and decision-making algorithms rather than relying solely on traditional decision trees or reinforcement learning.
Neural Networks with Softmax Activation:
A neural network trained to route queries based on encoded query features and user priorities. The softmax function outputs probabilities for each model, and the model with the highest probability is selected (or multiple models in collaborative tasks).
Reinforcement Learning (RL):
In advanced systems, reinforcement learning may be employed. The routing model learns optimal routing strategies by maximizing rewards (e.g., high response quality, low latency, reduced cost). RL can also adapt to new models or parameters over time, improving efficiency through trial and feedback loops.
The routing model in Humiris likely uses a hybrid approach, combining machine learning (neural networks) for dynamic decision-making with principles of multi-criteria optimization. The advanced mode incorporate reinforcement learning for adaptability in complex and evolving environments
Our gating model(the heart) in Humiris' system functions as an intelligent gating mechanism that dynamically selects and orchestrates multiple large language models (LLMs) based on predefined parameters such as cost, performance, privacy, and speed. While the exact implementation specifics can vary, here’s how it might work under the hood:
The gating model evaluates each query and the available models' characteristics to determine the optimal routing. This involves dynamically assigning weights to multiple parameters and scoring models based on the query's requirements. The architecture combines elements of both machine learning and decision-making algorithms rather than relying solely on traditional decision trees or reinforcement learning.
Neural Networks with Softmax Activation:
A neural network trained to route queries based on encoded query features and user priorities. The softmax function outputs probabilities for each model, and the model with the highest probability is selected (or multiple models in collaborative tasks).
Reinforcement Learning (RL):
In advanced systems, reinforcement learning may be employed. The routing model learns optimal routing strategies by maximizing rewards (e.g., high response quality, low latency, reduced cost). RL can also adapt to new models or parameters over time, improving efficiency through trial and feedback loops.
The routing model in Humiris likely uses a hybrid approach, combining machine learning (neural networks) for dynamic decision-making with principles of multi-criteria optimization. The advanced mode incorporate reinforcement learning for adaptability in complex and evolving environments
Do I understand right that it breaks down every task into subtasks and routes them to the most suitable model?
Yes, that's correct! Humiris breaks down tasks into subtasks when appropriate and routes each to the most suitable model. This process ensures that every part of the task is handled by an LLM optimized for factors specific tasks or domain and others parameters like quality, speed, cost, energy efficiency, or privacy.
It’s all about using the right tool for the right job to maximize efficiency and performance.
The concept is solid, but I’d love to see more transparency on costs when routing multiple LLMs. What do you think about Openrouter
Great AI revolution, amazing tool
Does Humiris offer built-in tools for model drift detection when combining LLMs
Yes, Humiris is exploring built-in tools for model drift detection as part of future updates. Currently, we recommend integrating tools like Alibi Detect for monitoring drift in your workflows. Stay tuned for updates as we enhance our platform's capabilities!
Amazing
Our platform combines advanced routing, custom reasoning models, and mix tuning to help businesses leverage AI at scale without sacrificing quality or control.
Here’s how it works:
• Routing Intelligence: Humiris uses a large-scale routing model to automatically select the best LLM based on your objectives (e.g., quality, cost, speed, energy, or privacy).
• Custom Reasoning Models: Create mix-models tailored to your needs, combining the strengths of multiple LLMs to outperform any single model based on RL and ML techniques.
• Flexibility: Deploy your models via SaaS, private instances, or your infrastructure for full control.
Here’s a quick demo: [https://youtu.be/Om1ytDfTg2M].
Why did we build this?
After working with various AI tools, we noticed a significant gap between what enterprises need and what most AI platforms offer. Businesses struggle to balance cost, performance, and complexity when using LLMs. We wanted to build a solution that simplifies this process while giving businesses the power to optimize for their unique needs. How does it compare? Most AI platforms rely on single-model implementations, which can be limiting. Humiris offers a multi-model approach, allowing you to:
• Achieve higher accuracy by combining models optimized for specific tasks.
• Reduce costs by routing requests to the most efficient model for the job.
• Protect privacy with custom deployments on your own infrastructure.
For example, our routing model ensures your chatbot queries, data analysis tasks, or code generation requests are handled by the most suitable LLM, saving time and money without compromising quality.
What have we learned? Building Humiris has taught us a lot about the challenges of multi-LLM integration. From managing data routing efficiently to ensuring models adapt dynamically to changing objectives, it’s been a fascinating journey. Some key insights include:
• Tradeoffs in Optimization: Balancing quality and cost often requires fine-grained adjustments that general-purpose models don’t handle well.
• Dynamic Adaptation: Many business needs evolve in real time, so static models fall short.
• Scalable Deployments: The ability to scale while maintaining performance is critical, and we’ve built Humiris to handle these demands seamlessly.
Who is it for?
Humiris is designed for businesses and engineers who want to go beyond basic AI capabilities. Whether you’re building a chatbot, automating workflows, or designing advanced reasoning systems, Humiris provides the tools to do so with precision and flexibility.
If you’re curious, check out our quickstart guide (https://docs.humiris.ai/quickstart) or play around with our platform (platform.humiris.ai).
How we do that ? See our research paper here (https://github.com/Humiris/MixtureofAI)
We’d love to hear your thoughts on our approach to AI, feedback on the platform, or any challenges you face with current AI devtools. Ask us anything!
We tried to make a video to explain it; you can check it out here (https://youtu.be/7kETS3UXZb0)