Ask HN: A proposal for interviewing "AI-Augmented" Engineers
5 comments
I interview like I always interview - behaviorally.
I filter for “smart and gets things done” (Joel Spolsky circa 2001).
“tell me about the project that you are most proud of” and then we talk about the architecture, tradeoffs, technical and business complexities, etc.
“I see you’ve been working for $x years. I’m sure there is a project you look back on knowing what you know now and cringe. Tell me about the project and what would you do differently?”
There are a few other questions. But I am usually also trying to measure soft skills and what level of scope and ambiguity they are comfortable with. The last thing I’ve ever needed when I am looking to hire is another “ticket taker”.
Even before AI, why would ever hire a junior dev? They are practically useless, do negative work and easy enough to poach someone with experience from another company for only slightly more money if you paying standard enterprise dev wages.
I filter for “smart and gets things done” (Joel Spolsky circa 2001).
“tell me about the project that you are most proud of” and then we talk about the architecture, tradeoffs, technical and business complexities, etc.
“I see you’ve been working for $x years. I’m sure there is a project you look back on knowing what you know now and cringe. Tell me about the project and what would you do differently?”
There are a few other questions. But I am usually also trying to measure soft skills and what level of scope and ambiguity they are comfortable with. The last thing I’ve ever needed when I am looking to hire is another “ticket taker”.
Even before AI, why would ever hire a junior dev? They are practically useless, do negative work and easy enough to poach someone with experience from another company for only slightly more money if you paying standard enterprise dev wages.
> Even before AI, why would ever hire a junior dev?
Were you ever a junior developer? Ever work with, and learn from, people who once were?
Were you ever a junior developer? Ever work with, and learn from, people who once were?
Well actually as a professional - no.
By the time I got my first job in 1996 I had been a hobbyist writing in assembly language for Apple //es and played around with it for 68k, PPC and x86 computers.
Three months into my first job as the only person who could code - I was hired as a computer operator based on a prior internship - I was tasked with writing a relatively complicated data entry system in C that ran across 12 network computers. I had to talk to the customer for the new contract and demo it. We ran the data entry center. 30 years later if I had to do the same using modern technology (pre AI), it would still take me maybe 3-4 months to get it done by myself.
Now on the other hand, I did admittedly become an “expert beginner” at my second job between 2002-2008.
But that’s besides the point , I have had two open reqs that I had to fight for and I’m judged based on my ability as a lead to get work done on time, within budget and meets requirements.
I hired a senior who I used my filter for at my last job. Why would I hire a junior that puts more work on me and my team?
I told him on a very high level “we need $x, these are the only constraints we are working under”. He came back with a design plan, he did the implementation, the design review and I gladly stepped back and gave him all of the credit.
That’s always my m.o., I’m far past the point where I need the glory.
By the time I got my first job in 1996 I had been a hobbyist writing in assembly language for Apple //es and played around with it for 68k, PPC and x86 computers.
Three months into my first job as the only person who could code - I was hired as a computer operator based on a prior internship - I was tasked with writing a relatively complicated data entry system in C that ran across 12 network computers. I had to talk to the customer for the new contract and demo it. We ran the data entry center. 30 years later if I had to do the same using modern technology (pre AI), it would still take me maybe 3-4 months to get it done by myself.
Now on the other hand, I did admittedly become an “expert beginner” at my second job between 2002-2008.
But that’s besides the point , I have had two open reqs that I had to fight for and I’m judged based on my ability as a lead to get work done on time, within budget and meets requirements.
I hired a senior who I used my filter for at my last job. Why would I hire a junior that puts more work on me and my team?
I told him on a very high level “we need $x, these are the only constraints we are working under”. He came back with a design plan, he did the implementation, the design review and I gladly stepped back and gave him all of the credit.
That’s always my m.o., I’m far past the point where I need the glory.
Thats a nice story. But it doesn't seem like a plan.
If its no longer worth hiring junior devs, then fairly soon there'll be few mid-level devs. And after that almost no senior developers. Is that the plan?
If its no longer worth hiring junior devs, then fairly soon there'll be few mid-level devs. And after that almost no senior developers. Is that the plan?
That’s a collective action problem - that’s not anyone’s problem who is currently working as a hiring manager.
If I were a hiring manager - I’m not now. But I’m in the interview pool as what Amazon would call the “bar raiser” (former employee) - my goal is to make sure my department goals are met - not what is going to happen in the industry in a decade or two.
Besides, the average tenure of any developer is around 2-4 years, how does my department or company benefit from hiring juniors? I’m incentivized as is everyone in seniors positions to meet quarterly and annual goals.
And going back to my previous (true) story, why should I take on the work of a junior dev along with all of my other work when I could and did just throw an ambiguous project at a senior?
If I were a hiring manager - I’m not now. But I’m in the interview pool as what Amazon would call the “bar raiser” (former employee) - my goal is to make sure my department goals are met - not what is going to happen in the industry in a decade or two.
Besides, the average tenure of any developer is around 2-4 years, how does my department or company benefit from hiring juniors? I’m incentivized as is everyone in seniors positions to meet quarterly and annual goals.
And going back to my previous (true) story, why should I take on the work of a junior dev along with all of my other work when I could and did just throw an ambiguous project at a senior?
I’m currently rethinking our hiring process. Like many of you, I feel that traditional algorithmic tests (LeetCode style) are becoming less relevant now that LLMs can solve them instantly. Furthermore, prohibiting AI during interviews feels counter-productive; I want to hire engineers who know how to use these tools effectively to multiply their output.
I am designing a new evaluation framework based on real-world open-source work, and I would love the community’s feedback on whether this sounds fair, effective, or if I’m missing something critical.
The Core Philosophy: We shouldn't test if a candidate can write syntax better than an AI. We should test if they can guide, debug, and improve upon an AI's output to handle the "last mile" of complex engineering.
The Proposed Process:
1. Task Selection (Real World Context) Instead of synthetic puzzles, we select open issues or discussions from public GitHub repositories that share a tech stack with our product.
2. Establishing the "AI Baseline" Before giving the task to a candidate, we run it through current SOTA models with minimal human intervention.
3. The Candidate Test Candidates are required to use their preferred AI coding tools. We ask them to submit not just the code, but their chat/prompt history.
How We Evaluate (The "AI Delta"):
We aren't just looking at the final code. We analyze the "diff" between the Candidate’s process and our "AI Baseline":
My Questions for HN:
Thanks for your insights!
(Full disclosure: In the spirit of this topic, this post was composed by AI based on my draft notes.)