HackerTrans
TopNewTrendsCommentsPastAskShowJobs

boatbuilder

no profile record

Submissions

Measuring LLM citations with server logs might not work as assumed

agentberlin.ai
3 points·by boatbuilder·11 bulan yang lalu·1 comments

comments

boatbuilder
·11 bulan yang lalu·discuss
I assumed you could measure LLM citations by watching server logs. Our experiment showed otherwise: ChatGPT and Perplexity often cite pages without fetching them, instead serving from a global cache.

We ran this on Vercel/Next.js middleware (before the edge cache) to rule out local caching—browser requests log normally, but many citations never appear.

This doesn’t mean measurement is impossible, but the “citation = visit” assumption doesn’t always hold. Curious if others have observed similar patterns.
boatbuilder
·2 tahun yang lalu·discuss
Have been using this for a while. Yep, stupidly simple and super super useful! Thank you
boatbuilder
·2 tahun yang lalu·discuss
The recent project from jina AI was a clone of this idea?

https://jina.ai/reader/
boatbuilder
·2 tahun yang lalu·discuss
So is this like dspy, but has a very specific user flow I.e chat
boatbuilder
·2 tahun yang lalu·discuss
Nice. Not sure how many people would need a service like this to make it financially viable for you to maintain though. Also noticed a bug: not getting verify email