been using savvy for a bit - shantanu gave me early access!
love how simple it is to use, personally been loving the savvy ask/explain in the last couple of weeks - been super easy to ask questions in the terminal and just get answers
but the history is quite good too, very cool to share notebooks with others on the team
want to thank the op for sharing this; i threw this together in the last couple of days ramping out to the "steamroll" - we think one of the key problems in LLMs in general but esp voice is evals and wanted to have a good place to evaluate voice-to-voice systems. these systems can be end-to-end like openai or (asr+llm)->tts or asr->(llm+tts) or asr->llm->tts
we built an ELO benchmark very much in the style of LMSYS and will be releasing results every two weeks