LLM Observability tools have taken off, but they all result in you sending away your generations and chat history. That is, if you actually have users.
If you want to keep that all for yourself, I made a small script that deploys a simple monitoring solution that you control in your own resource group on Azure.
Feedback / critiques / suggestions all welcome, thanks.
If you want to keep that all for yourself, I made a small script that deploys a simple monitoring solution that you control in your own resource group on Azure.
Feedback / critiques / suggestions all welcome, thanks.
https://github.com/aavetis/azure-openai-logger