I have one (at least) fundamental concern about the approach - let's say I'm building an anti-fraud system that uses AI (through API), and maybe I'm asking AI whether my user [email protected] is a potential fraudster. By masking this email address I'm sabotaging my own AI prompt - the AI cannot longer reason based on the facts that 1) the email is a free public email 2) the email says 'fraud' right in your face.
There is one potential issue with scanning the entire codebase IMHO - if the codebase is old enough, there can be old snippets of code that DON'T correspond to the current preferences anymore. So scanning the last 200 PRs and reading comments is a bit safer maybe.
This is what my vibe coded PoC did - it just used GitHub API to go through the last X merged PRs, see the diffs and all discussions and create a document based on that - the developer manifesto kind of, a set of preferences and rules for this dev team. Then, use this manifesto as context whenever reviewing new PRs
Your example is with Codex - OpenAI could implement this easily on their end right? Every prompt of yours was an API call and they have a log, they can easily re-create a quick history of what you did/asked for before?
I think CodeRabbit/others do this already? They learn from your code base/previous PRs, then use that in the future. I built a mini PoC of this a few weeks ago as well, out of curiosity