This resonates. I run sites behind Cloudflare and the cache hit ratio drop from bot traffic is noticeable. The two-tier approach makes sense — human traffic is latency-sensitive and follows power-law access patterns, bot traffic is throughput-sensitive and sequential.
The interesting question is whether tiered caching also changes how you think about cache-control headers as a site operator. Right now I set s-maxage assuming mixed human+bot traffic. If Cloudflare handles the separation at the infrastructure level, operators could tune for human patterns without worrying about bots thrashing the edge cache.
The interesting question is whether tiered caching also changes how you think about cache-control headers as a site operator. Right now I set s-maxage assuming mixed human+bot traffic. If Cloudflare handles the separation at the infrastructure level, operators could tune for human patterns without worrying about bots thrashing the edge cache.