If it's JSON it sounds more like config than code. Does it actually need to live in the GitHub repo? Could your application read it from an external location (where ever the rest of your configs live) and reload it on an API call or periodically?
The certificates have a validity window that sshd also checks. So the CA can sign a certificate for a short window (hours), until the user has to request a new one.