The Nonprofit Feeding the Internet to AI Companies(theatlantic.com)
theatlantic.com
The Nonprofit Feeding the Internet to AI Companies
https://www.theatlantic.com/technology/2025/11/common-crawl-ai-training-data/684567/
5 comments
https://archive.ph/kObM3
Ironically on an article about subverting paywalls, I posted a gift link but hackernews has stripped out the token.
Journalists are stuck between a rock and a hard place: they want their articles available for people to read, but they want them hidden behind a paywall because that's how they get paid.
It reminds me a bit of the dilemma faced by authors of open source software. Collectively, developers have surrendered to the corporations, given up the GPL and given their work away for free. They don't worry about it because they can have a profitable sideline working for those same corporations, but journalists, writers and other artists don't have a lucrative fallback plan.
It reminds me a bit of the dilemma faced by authors of open source software. Collectively, developers have surrendered to the corporations, given up the GPL and given their work away for free. They don't worry about it because they can have a profitable sideline working for those same corporations, but journalists, writers and other artists don't have a lucrative fallback plan.
This is a biased article which argues in favor of DRM (as paywalls) because it is published in a paywalled magazine.
Further in, the article admits that such content is already free and sits inside the page source but is obfuscated by running code.
US law has the precedent (and a recent case that's about AI training) that training, reading, and transforming are not illegal if the materials themselves are legally obtained. Wholesale duplication of copyright material is illegal, but AI companies have already shown in court that they don't duplicate material but rather transform it at great effort and expense.
Further in, the article admits that such content is already free and sits inside the page source but is obfuscated by running code.
US law has the precedent (and a recent case that's about AI training) that training, reading, and transforming are not illegal if the materials themselves are legally obtained. Wholesale duplication of copyright material is illegal, but AI companies have already shown in court that they don't duplicate material but rather transform it at great effort and expense.
The article makes it clear that Common Crawl is ignoring copyright take down requests, and only modified it's search engine to fake having taken down content.