PolandKid·5 年前·議論You should just declare your bot as a user-agent. Most publishers won't even bother to do it, but leaving a publisher an option is the correct etiquette for any bot. Random user agents is cloaked scraping.
PolandKid·5 年前·議論 @AwkwardPandaAnd how does a site opt out of your scraping? Do you have a unique user-agent when you scrape? A set of IPs?
PolandKid·5 年前·議論We're considering the angle of an app whose sole purpose is to capture the valuable IP of an author and remove their ability to monetize it.It's not like they're paying them via some publisher program, just pure scraping and re-organizing.