AI inference costs dropped up to 10x on Nvidia's Blackwell(venturebeat.com)

7 pointsby CrankyBearhá 5 meses3 comments

AI inference costs dropped up to 10x on Nvidia's Blackwell

https://venturebeat.com/infrastructure/ai-inference-costs-dropped-up-to-10x-on-nvidias-blackwell-but-hardware-is

4 comments

> Sully.ai cut healthcare AI inference costs by 90% (a 10x reduction) while improving response times 65% by switching from proprietary models to open-source models running on Baseten's Blackwell-powered platform, according to Nvidia. The company returned over 30 million minutes to physicians by automating medical coding and note-taking tasks that previously required manual data entry.

Are the margins that low that it would make sense to give up on quality of output and use open source models?

mountainriverhá 5 meses

Interviewed at Sully, absolutely insane company, I’m sure they aren’t making a rational decision here

hn12há 5 meses

You have me curious, mountainriver. While I don't understand what you've written, I want to know more. Are you saying that open-source models can't be trusted as well as (some of the?) proprietary ones, and therefore aren't fit for "mission-critical" medical applications?

[deleted]há 5 meses