>> interesting. workers.dev domains can be a liability sometimes -- if you've mapped the worker to a real zone, then you probably don't want the workers.dev zone anymore.
maybe that is a good idea, prompt a user to disable the worker.dev domain. however, having the worker.dev domain open for me is more of a backup way to access the worker as the mapped domain is more for a hobby project.
> Interesting. workers.dev domains can be a liability sometimes
what about allowing the specific worker.dev to serve a robots.txt on its own that can be used to disable the AI bots?
Finally managed to kill the traffic:
1. renamed the worker - the bot was bypassing the route on the domain and using the worker.dev domain
2. add WAF rules to block gptbot in user agent
3. serve zip bomb on request from GPT.
interesting that GPT was accessing the worker directly to bypass the WAF rule son Cloudflare