It's exactly what you've said. I'm speaking about Anthropic though, I have no idea whether OpenAI does it.
> Anthropic agrees that Customer (a) retains all rights to its Inputs, and (b) owns its Outputs. Anthropic disclaims any rights it receives to the Customer Content under these Terms. Subject to Customer’s compliance with these Terms, Anthropic hereby assigns to Customer its right, title and interest (if any) in and to Outputs. Anthropic may not train models on Customer Content from Services. “Inputs” means submissions to the Services by Customer or its Users and “Outputs” means responses generated by the Services to Inputs (Inputs and Outputs together are “Customer Content”).
This is the only commercial ToS clause about how they handle your data for subscription users. They only promise not to train the model on your exact input and exact output. There's nothing about not washing your data - the "clean-room" approach, which is obviously easily automatable by a company that specializes in automation. That is not training a model on your data, it is using a model to create derivatives of your data, then training it on "their" derivatives.
People really needs to apply pressure and start demanding answers from these companies regarding this - because it is a huge problem. Historically, the amount of labor required to do something like this would make it entirely unfeasible, so this is all new territory. The existing laws and the requirements around clarity surrounding these conditions do not reflect the technology progress.
That's not at all how they do it. They wash the data. The end result is they can steal your IP insights without it being explicitly tied to your business. All of the decisions you made to build your product? Those become the standard suggestions in the next model for people building the same sorts of products. All of that error correcting you did, which in a normal business would be considered hard-earned IP (like in the case of Apple's lawsuit - the "what not to do" is just as valuable as the "what to do"), is now free correction for the next model to produce the perfect result in a one-shot. Now the AI company can commoditize your labor and your industry, or compete against you if they so wish.
Yes. They're constantly lying, and constantly getting caught for it. They have a reputation for it. Why do you think this would be any different?
Their standard opt-out agreement frames it as if they won't train on your data, but they do anyway, due to legal loopholes. They essentially clean-room everyone who opts-out, so while it's "technically" not training on "your" data, to the model it makes no difference. Your alpha and IP is not safe. Paying customers are now more easily able to clone your business as well, not just Anthropic themselves.
The only reason this hasn't leaked yet is fear. Anthropic is a very litigious and dangerous company. Only a matter of time though, someone there will grow a spine and speak up.
They do all of this, plus far worse. Including but not limited to flooding the internet with guerrilla marketing and sentiment-shifting bots across all social media platforms.
This is the sort of company they are, and it's just the tip of the iceberg: https://clawd.rip
This is the key to training better and better models.
You collect all of the direction prompts and anti-direction prompts, and you finetune the model to behave that way for the particular domain, under those particular conditions, as a default.
This is also why I don't believe for a second that the AI labs aren't training on our data, even when we opt out. This sort of data collection is a requirement for improving the models.
It's also why Anthropic purposefully gives you a dumb model or a corner cutting model to goad you into guiding it into the correct answers, so that you do the fine-tuning for them. OpenAI at least, does not do that. Which is why ChatGPT is far less infuriating to work with.
Obviously they are looking at your IP and code. Anthropic trains on your data regardless of you opting out, I know that one for certain. There's no coincidence they "keep your data temporarily despite opting out" - because they wash it in legal loopholes. There is no opt out. These companies WILL steal your business. Only a matter of time before they are sued as well.
There's algorithmic ways to accomplish the same thing without requiring monitoring of the brain, if there's enough data to pull from. Semantic analysis + engagement monitoring is more than enough. Peoples emotions expressed via comments is a reflection of brain region activation.
The end result of this will be that creating viral or maximally converting content will not be restricted to the top content creators. It will also result in down regulation of those brain regions due to constant stimulation, so the bar will increase for what classifies as attention grabbing and the brain will filter most of it out as noise.
All in all, I don't really see this being a big difference to the outcomes we're already seeing.
I want my model to help me build up its own infrastructure that instills it with the sort of constraints I want for my project, rather than have it behave generically and automatically for everything.
It should follow instructions incredibly well while inferring contradictions or gaps in logic and surfacing those to the user as suggestions for improvements and persistence.
I really hate how Claude just assumes you want to do X/Y/Z and goes off and breaks everything and you're constantly screaming at it STOP DOING THAT. Instead, it should just do the minimal things while building its own guidance along the way in a persisted memory, like, 'would you like me to do X, now, and in the future?' etc.
Agreed that they shouldn't be writing vulnerable code to begin with. You'd think the models would be trained to know when they are working on something that has security implications, and to validate the security of what they're building, as they're building it.