Show HN: AI Cost Firewall – OpenAI-compatible gateway with semantic caching(github.com)

1 pointsby vcaluser4 месяца назад1 comments

Show HN: AI Cost Firewall – OpenAI-compatible gateway with semantic caching

https://github.com/vcal-project/ai-firewall

2 comments

IxInfra4 месяца назад

The 0.92 semantic similarity threshold is interesting. curious how you landed on that number? Thinking too low and you risk false cache hits but too high and you lose most of the semantic benefit. Did you run experiments or is it tunable per use case?

vcaluser4 месяца назад