Point is to actually avoid hallucinations, Think I didnt focus on it as much as I wanted.
To avoid hallucinations, make sure you use strict prompt when creating the chatbot/api key.
Combination of rag + strict prompt will make sure that AI does not go out of bounds of it contextualised knowledge and avoids using LLM's internal knowledge.
Honestly this is very very fresh, I was tinkering with hosting some models and wanted to optimize costs, tried few inference engines. Just want to collaborate on organizing data.
Agree, we will add a MUI table very soon. Also some charts.
I genuinely want someone to roast the way I did my benchmark process described there. Want something good enough yet easy to run.
Yeah we want to do exactly this, benchmark and add more data from differnt gpus/cloud providers, will appreciate your help a lot!
There are many inference engines which can be tested and updated to find best inference methods