it's like AI-generated images, though those tend to be more obvious than AI-generated text due to the "average" effect being visually obvious. Yes, these are normal patterns in English, but it is not normal to use them with the frequency that triggers people's llm-lese senses.
I don't see a way out of it until either generated content is either completely unable to be distinguished from authored content (unlikely?), or there's a silver bullet for identifying generated content (also unlikely?)
I'm a little surprised that preserve_thinking would matter here for cache purposes. for actual capabilities/intelligence, yes, I'd imagine it helps to have past reasoning traces in multi-turn setups.
but for caching, all you are doing is leaving off a fraction of the most recent assistant message generation, which will have little/no impact on cache hit rate.
presumably it's the reverse engineered server that has most of the work put into it, and one would hope that's what is going to be released if the developer decides to
which is important to bear in mind if people are introducing a "drop earliest messages" sliding window for context management in a "chat-like" experience.
once you're at that context limit and start dropping the earliest messages, you're guaranteeing every message afterwards will be a cache miss.
a simple alternative approach is to introduce hysteresis by having both a high and low context limit. if you hit the higher limit, trim to the lower. this batches together the cache misses.
if users are able to edit, remove or re-generate earlier messages, you can further improve on that by keeping track of cache prefixes and their TTLs, so rather than blindly trimming to the lower limit, you instead trim to the longest active cache prefix. only if there are none, do you trim to the lower limit.
I don't see a way out of it until either generated content is either completely unable to be distinguished from authored content (unlikely?), or there's a silver bullet for identifying generated content (also unlikely?)