(1) could you hash everyone's mp3 files so that if person a had the same songs as person b, you only need to store them once? (2) people obviously storing copyright stuff in there -- do you have legal issues?
Can't use third party service -- compliance stuff. This is helpful though. Using another model to tidy it up -- maybe Alpaca -- could be an option too. Then we'll just do speaker separation etc. manually later.
I have to transcribe a tonne of Chinese interviews soonish -- any further thoughts or experiments you can think of? Maybe some preprocessing steps to the audio? For example, cut it into one minute chunks with some overlap, then transcribe those, so that it can't skip those bits...? Or can we finetune it on a library of Chinese mp3s + transcripts?
Yep. This is such an obvious use case. Have you seen the best thing out there that does this? Where can I load 100gb of pdfs and ask questions about what's in them??
Just because it produces sequences of tokens that we typically experience as evidence of reasoning does not mean that it reasoned to produce them. We know -- generally -- the process by which those tokens were produced. It is not deterministic, but its properties are understood. It's a language model! It is not a mind.