If they formally sunset the feature, there’s less of a case for someone to sue.
One can’t say they didn’t know or consent to their group chat info being used for training data if there’s no reason to assume your chat data is private
There are a number of projects working on evals that can check how 'smart' a model is, but the methodology is tricky.
One would want to run the exact same prompt, every day, at different times of the day, but if the eval prompt(s) are complex, the frontier lab could have a 'meta-cognitive' layer that looks for repetitive prompts, and either:
a) feeds the model a pre-written output to give to the user
b) dumbs down output for that specific prompt
Both cases defeat the purpose in different ways, and make a consistent gauge difficult. And it would make sense for them to do that since you're 'wasting' compute compared to the new prompts others are writing.
A theory that’s floating around is that since frontier models are so good at sounding like humans, companies paying for ads are arguing that Dead Internet Theory -> ad costs should go down.
Therefore, the push to ID everyone using the internet (even down to the hardware) is a way to prove that ads are being served to real humans in their target demographic.
If I had to guess it’s because GDP growth in industrialized countries are driven by financialization facilitated by cheap power and energy, and the countries that are more financialized don’t exactly want the countries with precious minerals to be industrialized enough to be self sufficient.
That’s why a lot of global south countries with valuable materials (oil, data center building blocks, etc) are often chronically destabilized, but not so much so that it’s impossible to find someone to buy unrefined materials from.
One can’t say they didn’t know or consent to their group chat info being used for training data if there’s no reason to assume your chat data is private