New tool helps you figure out which Chatbots to trust(technologyreview.com)

New tool helps you figure out which Chatbots to trust

https://www.technologyreview.com/2024/04/25/1091835/chatbot-hallucination-new-tool-trustworthy-language-model/

1 comments

Large language models are famous for their ability to make things up—in fact, it’s what they’re best at. But their inability to tell fact from fiction has left many businesses wondering if using them is worth the risk.

A new tool created by Cleanlab, an AI startup spun out of a quantum computing lab at MIT, is designed to give high-stakes users a clearer sense of how trustworthy these models really are. Called the Trustworthy Language Model, it gives any output generated by a large language model a score between 0 and 1, according to its reliability. This lets people choose which responses to trust and which to throw out. In other words: a BS-o-meter for chatbots.