throwawaybot9·3 years ago·discussMy crawlers generate give or take 100 million requests a year and it costs me $100 per month.