HackerTrans
TopNewTrendsCommentsPastAskShowJobs

Firefishy

no profile record

comments

Firefishy
·5 माह पहले·discuss
Disclosure: I am part of the mostly volunteer run OpenStreetMap ops team.

Technically we able to block and restrict the scrapers after the initial request from an IP. We've seen 400,000 IPs in the last 24 hours. Each IP only does a few requests. Most are not very good at faking browsers, but they are getting better. (HTTP/1.1 vs HTTP/2, obviously faked headers etc)

The problem has been going on for over a year now. It isn't going away. We need journalists and others to help us push back.
Firefishy
·5 माह पहले·discuss
Disclosure: I am part of the OpenStreetMap mostly-volunteer sysadmin team fighting this.

The scrapers try hard to make themselves look like valid browsers, sending requests via residential IP addresses (400,000+ IPs at last count).

I reached out to journalists because despite strong technical measures, the abuse will not go away on its own.
Firefishy
·10 माह पहले·discuss
OpenStreetMap Slack https://slack.openstreetmap.us/ was forced to downgraded to the free edition earlier this year for similar reasons.
Firefishy
·पिछला वर्ष·discuss
https://protectli.com/ Good quality devices. Real serial consoles to allow recovery when you make a networking configuration mistake ;-)