Ask HN: Is Web Scraping Legal?
Is web scraping legal? It seems like a grey area. Wondering if there is a definitive answer on this question.
5 comments
I researched this theme with web.
In 2020 state,
Open data (everyone can read without login) scraping is basically legal. But there are the following notes: 1. use(over copyright(like re-publish) is illegal) 2. access volume(site attack is illegal) 3. robots.txt(site denied contents scraping might be illegal)
Closed data(need login) scraping is follow the site's terms. Consider many service bans scrape it terms, there are the high legal risk. If it don't grow legal trouble, site close the scraping account easily.
So scraping open data and store your side is thought legal(like google or many other bots do), other scraping situation include illegal case.
In 2020 state,
Open data (everyone can read without login) scraping is basically legal. But there are the following notes: 1. use(over copyright(like re-publish) is illegal) 2. access volume(site attack is illegal) 3. robots.txt(site denied contents scraping might be illegal)
Closed data(need login) scraping is follow the site's terms. Consider many service bans scrape it terms, there are the high legal risk. If it don't grow legal trouble, site close the scraping account easily.
So scraping open data and store your side is thought legal(like google or many other bots do), other scraping situation include illegal case.
What about selling scraped data ?
It probably depends more on what kind of data you are scraping and what you want to do with it. Scraping and redistributing copywritten data is clearly illegal, but otherwise isn't it a bit like taking photographs of billboards?
Yes. Bots do it all the time. GoogleBot, AhrefsBot, etc.