Ask HN: Legally Scraping Reddit Images
1 comments
You can look at the "legal" aspect of it by Googling "is it legal to scrape". My understanding (IANAL) is that it is OK as long as the user agreement is "opt-out" as opposed to "opt-in" (eg: clicking a consent box before viewing the content). You'll have to read up on this yourself and weigh the risk/reward of doing your project. I (NAFL) would assume the risk is quite small. The reward -- that's your call.
As for the other part, getting the data: it's called scraping. Depending on your experience with scraping, you may need to pay for certain aspects of it (eg, getting a large list of proxies so Reddit does not block you, or using a scraping API). Or maybe your project is small enough (or time constraints large enough) that you can slowly siphon the data via your own means.
As per Reddit allowing it: Refer to the legality of scraping, and apply it to Reddit.
As for the other part, getting the data: it's called scraping. Depending on your experience with scraping, you may need to pay for certain aspects of it (eg, getting a large list of proxies so Reddit does not block you, or using a scraping API). Or maybe your project is small enough (or time constraints large enough) that you can slowly siphon the data via your own means.
As per Reddit allowing it: Refer to the legality of scraping, and apply it to Reddit.
The only problem is that I don't have any experience with collecting that magnitude of data. Does anyone have experience with scraping reddit and can offer pointers on how to approach it? I'm also having a difficult time figuring out if reddit even allows this.