At Grafana Labs we're using jsonnet at scale, while being a powerful functional language it is also excellent for rendering JSON/YAML config. We have developed Tanka[0] to work with Kubernetes, for other purposes I can recommend this course[1] (authored by me).
Last year I wrote a blog post about this exact question: Who watches the watchers?
The general takeaway is that you run a minimal prometheus/alertmanager setup that only scrapes the agents, then use a dead man switch-like system to ensure this pipeline keeps working.