Show HN: Valkey-native semantic cache with OTel and Prometheus built in
3 comments
Nice — the Valkey Search 1.2 compatibility work is non-trivial, the RediSearch divergences are well-documented in the changelog.
For teams who want semantic caching without managing vector infra, CacheGPT (cachegpt.app) is a hosted alternative — same pgvector + cosine similarity approach, but drop-in HTTP with no Redis/Valkey cluster to maintain. Different tradeoff: you get the managed tier in exchange for less control over the similarity threshold tuning.
Both approaches make sense depending on your stack. Self-hosted is better if you're already running Valkey and want OTel integration; managed works better for teams that just want to stop burning API credits.
For teams who want semantic caching without managing vector infra, CacheGPT (cachegpt.app) is a hosted alternative — same pgvector + cosine similarity approach, but drop-in HTTP with no Redis/Valkey cluster to maintain. Different tradeoff: you get the managed tier in exchange for less control over the similarity threshold tuning.
Both approaches make sense depending on your stack. Self-hosted is better if you're already running Valkey and want OTel integration; managed works better for teams that just want to stop burning API credits.
@betterdb/semantic-cache is a standalone semantic cache backed by valkey-search. No framework coupling - works with any LLM client. Every check() and store() emits an OTel span and increments Prometheus counters and histograms out of the box. No instrumentation code required. MIT. Optional LangChain and Vercel AI SDK adapters. Works on self-hosted Valkey, ElastiCache for Valkey, and Memorystore for Valkey.
https://www.npmjs.com/package/@betterdb/semantic-cache
The Valkey Search 1.2 divergences from RediSearch that required explicit handling are in the changelog.