Well, Google "screen scrapes" millions of websites. They bet their business on it and they seem to be doing OK.
If you upload something to an http server connected to the public internet on tcp/80, and you don't exclude the path to it in robots.txt, then should anyone be surprised if it is copied? HTTP clients don't read TOS.
If Google had to read and interpret every every website's TOS, I doubt they could easily, if at all, produce an index the size of the one they have. It seems by ignoring a "fear of scraping" they managed to produce something valuable that the courts seem to side with in spite of offended copyright holders.
Moreover there's no requirement for them to make their "cache" publicly accessible. But they do. And again this has held up in court quite well. I doubt anyone would be surprised that people are using it. Or "scraping" it if you want to play word games.
I'm glad you are not offended by my remarks. It may seem like I'm taking a shot at developers or Mozilla. As a user who hates bloated web browsers and Java, maybe I am. But from an objective viewpoint, I'm not. I'm just saying journalists and developers have as yet not been able to find common ground, even though they both know they need each other. Something is amiss.
Meanwhile solid journalism is deteriorating.
As for me, I am taking action, but at a lower level. The stuff I'm working on is much bigger than page views, pay per click, or even news. I'm interested in software that can alter paradigms. I want to see the original version of the internet. No middlemen.
The idea is good i.e. teach journalists to hack.
But there's a conflict of interest. Look who's doing the teaching. Mozilla and Google are in business together.
Mozilla != journalists
Mozilla == web developers == middleman
FAIL. (It's been tried before: Web developers purporting to "help" journalists. The results have not been good.)
Even in print, there is only one NewYorker. There are some other mags that are almost comparable, but it's not like there are scores of them to choose from.
And how exactly does longform help websites advance their dodgey advertising tactics?
It's just my opinion but I honestly think writing on the web was better in the 1990's. These type of websites would likely be criticized today as "walls of text" and for not having enough JavaScript or using enough of the latest CSS or HTML features.
The more interesting question to me is the relationship between talented journalists and website developers. They do not necessarily see the world the same way. They operate from different models.
What if salaried journalists committed to the craft of writing (not pandering to advertisers or search engines) had the technological empowerment to build high speed, highly organised distribution (publishing) systems without the need to consult with web developers and accept developers' dreams of dodgey pay by the click web advertising? Do I believe there is a disconnect between the two groups? Yes.
There are still middlemen in publishing. They are just new middlemen. Alas, they can't afford to pay salaries as their predecessors could.
Perhaps what needs to be taught is the technical skill so that ageing journalists schooled in long form can cut out the middleman, start their own publishing businesses and hire the top talent to train the young people entering the field.
"Sinbad the Sailor and Tinbad the Tailor and Jinbad the Jailer and Whinbad the Whaler and Ninbad the Nailer and Finbad the Failer and Binbad the Bailer and Pinbad the Pailer and Minbad the Mailer and Hinbad the Hailerand Rinbad the Railer and Dinbad the Kailer and Vinbad the Quailer and Linbad the Yailer and Xinbad the Phthailer."
If you upload something to an http server connected to the public internet on tcp/80, and you don't exclude the path to it in robots.txt, then should anyone be surprised if it is copied? HTTP clients don't read TOS.
If Google had to read and interpret every every website's TOS, I doubt they could easily, if at all, produce an index the size of the one they have. It seems by ignoring a "fear of scraping" they managed to produce something valuable that the courts seem to side with in spite of offended copyright holders.
Moreover there's no requirement for them to make their "cache" publicly accessible. But they do. And again this has held up in court quite well. I doubt anyone would be surprised that people are using it. Or "scraping" it if you want to play word games.