Measuring LLM citations with server logs might not work as assumed(agentberlin.ai)

3 pointsby boatbuilder11 mesi fa1 comments

Measuring LLM citations with server logs might not work as assumed

https://agentberlin.ai/blog/how-llms-crawl-the-web-and-cache-content

1 comments

I assumed you could measure LLM citations by watching server logs. Our experiment showed otherwise: ChatGPT and Perplexity often cite pages without fetching them, instead serving from a global cache.

We ran this on Vercel/Next.js middleware (before the edge cache) to rule out local caching—browser requests log normally, but many citations never appear.

This doesn’t mean measurement is impossible, but the “citation = visit” assumption doesn’t always hold. Curious if others have observed similar patterns.