I’d also be interested in seeing the rate of hallucinated citations from the pre-ChatGPT era.
I’m not entirely convinced this is purely an LLM-related issue. I’ve definitely come across countless misattributed citations in big four reports long before generative AI became widespread.
Testing this hardware LLM (LLAMA 3.1 8B on a chip) I get ~16k tokens per second.
With frontier models plateauing, I’ve been convinced AI will end up like bitcoin mining, and that NVIDIA’s general-purpose GPUs will be replaced by model-specific chips.
Forget everything you know and consider that it might be a misguided and risky negotiation tactic.
Disclaimer: This is not business advice and should be read using Cartman’s voice.
Step 1: Announce publicly that you are not renewing your contract.
Step 2: If the market has viable alternatives or the service you are negotiating isn’t that hard to replicate, other actors will manifest to fill in the gaps, especially if your business is attractive. (E.g., The top comment is building an alternative; other comments point to alternative services.)
Step 3: Congratulations, you now have leverage for a significant discount with your previous provider because they face the real prospect of losing your business entirely to a competitor. If the competitor is private, you can even double dip by investing in their company before attributing them the contract.
I’m not entirely convinced this is purely an LLM-related issue. I’ve definitely come across countless misattributed citations in big four reports long before generative AI became widespread.