You’re right: structured assessments are old news.
The thing I’m testing is not “new tests,” but a tighter system: derive the rubric from real team tasks, apply the same rubric to both public evidence and live assessments, and make every score traceable (with lower weight on easy-to-fake AI-era signals).
If that doesn’t improve consistency/speed/quality vs current hiring loops, then it’s just old process with new branding.
This is a very fair critique, and I agree with it.
You’re right that many strong engineers can’t legally share employer code, and “has OSS time” is not a universal signal. So I’m now thinking of this as a dual-path system:
1) Public evidence path (for people with OSS/public technical work), where existing contributions are treated as reusable evidence.
2) Structured assessment path (for people without public artifacts), using scoped tasks/pair debugging/incident reasoning mapped to the same rubric.
So OSS should be an advantage when present, but never a requirement.
Also agree on AI confounders: raw public activity can’t be trusted at face value anymore. We need to weight traceable process signals (review back-and-forth, bug-to-fix chain, consistency over time) higher than easy-to-generate text/code volume.
If you were hiring with this, what would be your minimum bar for “credible evidence”?
You hit the nail on the head regarding Juniors—they are already in a tough spot, and I certainly don't want to build a tool that makes that worse.
However, my hypothesis is that matching on 'Problem Vectors' might actually help break the cycle of 'only getting hired for what you've already done.'
Transferable Complexity: A traditional recruiter sees 'Game Dev' and ignores them for a Fintech role. But a vector model might see that the candidate solved a 'distributed concurrency' problem in a game that is mathematically similar to the 'payment sync' issue in the JD. It matches on capability, not just domain keywords.
Signal for Juniors: Currently, ATS filters reject Juniors based on '0 years experience.' If a Junior has tackled a complex logic problem in a hobby project or Hackathon, this system highlights that specific signal. It gives them a fighting chance based on code reality rather than resume keywords.
That said, I agree this model naturally leans towards Senior/Specialist roles where specific technical gaps need immediate filling. It's not a silver bullet for 'hiring for potential,' but I hope it's a step up from the current keyword-soup approach.
I love the "mental model" approach here. Most guides I've seen either get bogged down in the minute details of TLS/Handshakes immediately or are way too high-level. The interactive packet visualization is a really nice touch to bridge that gap. Thanks for sharing!
Pavlo is right to be skeptical about MCP security. The entire philosophy of MCP seems to be about maximizing context availability for the model, which stands in direct opposition to the principle of Least Privilege.
When you expose a database via a protocol designed for 'context', you aren't just exposing data; you're exposing the schema's complexity to an entity that handles ambiguity poorly. It feels like we're just reinventing SQL injection, but this time the injection comes from the system's own hallucinations rather than a malicious user.
The 'once a day' fetching limitation is a fascinating idea. It really captures the vibe of reading a physical newspaper in the morning rather than constantly checking for updates. I think many of us could use a tool that enforces a bit of 'digital silence' like this.