The engineering interview process is broken, AI cheating is exposing it faster
18 comments
I normally ask candidates to build a sample and easy component similar to those we use at work. There are a couple of points in the exercise that I use to understand if the candidate is aware of certain deep implications of the language and/or the framework they are using. If something is not on their CV, I don't test for that, but if something is, I will test for that extensively. I ask questions all through the process. I allow them to look up resources during the interview,
but I don't allow AI. This works well and it's really hard to game, especially if the candidate shares their screen or is interviewing in office. I mostly tailor the interview process to minimize false positives and I don't much care about false negatives: these are much riskier and hard to catch, and to be honest allowing them is not a big deal.
interviewing and finding the right candidates is very simple process - make sure you know who the right person is (but be open to be blown away by something entirely different) and work backwards from there. what is your ideal candidate? how are they the missing lego block to complete the masterpiece? what do other already-assembled blocks do? what tools are they using? I’ve been on so many interviews (on both sides) and am always blown away by what percentage of that process is wasted on meaningless things (like reversing a linked list). you know what you need and are probably surrounded by many amazing people - just find another one of them to complete the puzzle
Just wait for the next YC batch - we are coming to save the developer community and rewrite the rules of the engineering hiring market!
>We're still using Leetcode-style problems that barely reflect the actual work engineers do.
There are plenty of "leetcode questions" that are pretty practical and can be 'solved' with basics data structures people use all the time such as maps, lists, sets and maybe some loops or recursion and flow contorl.
There are plenty of "leetcode questions" that are pretty practical and can be 'solved' with basics data structures people use all the time such as maps, lists, sets and maybe some loops or recursion and flow contorl.
In person interview. Solving real problem. Ideally, one I met recently.
Agreed. Nothing beats in-person from the point of view of safety and nothing beats real problems to understand if the candidate is able to work with the current team. Many think that interviewing is about detecting hidden signals telling you that the candidate is a misunderstood genius, so you have advantage on competitors, but reality is much less dramatic: you just need someone with good communication skills who is used to work on the day to day issues at hand.
Open source contribution history and interviewing with paid projects.
See https://algora.io
No, I don’t write one line of code that I don’t get paid for - ie random open source contributions.
You can be paid for open source contributions, sometimes.
I did. I contributed to a popular official open source “AWS Solution” while working there. But this is about people who are looking for jobs.
So that open source contribution can be part of your CV when looking for a job, right?
I guess what I'm trying to say is that employees should try to get open source contributions as part of their job. Either by open sourcing a component or by contributing to projects on which the company depends.
I guess what I'm trying to say is that employees should try to get open source contributions as part of their job. Either by open sourcing a component or by contributing to projects on which the company depends.
Yuk so you are judged on volunteer work. While the execs... aren't.
Don't try and fight reality. You'll just end up tired, frustrated, and in the exact same spot years later.
It ain't my reality. I get work without doing freebies.
until you don’t…
28 years, 10 jobs, including one at BigTech. I’ve never written one line of code that I haven’t been paid for since a year before graduating from college - in 1995.
with that much experience you should consider volunteering you services to good causes, no money, but you’ll feel good. I am similar in years and have written probably six-digit lines of code I haven’t been paid for.
I live a very commitment free life. Outside of work and my wife.
Our home in Florida easily converts to a short term rental (a unit in a condotel - https://www.investopedia.com/terms/c/condotel.asp) when we leave for longer stints. We did the digital nomad thing for a year around the US and in the next couple of years, we plan to alternate between Panama City Panama and San Jose Costa Rica during the winter in the US - their dry season.
During the summer, we might spend a month back where are friends and family are in GA.
The benefit of remote work. We have experience living out of two suitcases and picking up and flying random places and staying for a few weeks.
Our home in Florida easily converts to a short term rental (a unit in a condotel - https://www.investopedia.com/terms/c/condotel.asp) when we leave for longer stints. We did the digital nomad thing for a year around the US and in the next couple of years, we plan to alternate between Panama City Panama and San Jose Costa Rica during the winter in the US - their dry season.
During the summer, we might spend a month back where are friends and family are in GA.
The benefit of remote work. We have experience living out of two suitcases and picking up and flying random places and staying for a few weeks.
We're still using Leetcode-style problems that barely reflect the actual work engineers do. Candidates are expected to grind 50+ algorithm questions just to get a shot—even if the job has nothing to do with algorithms. Now, AI tools are being used to pass take-home assignments, do live coding rounds, or even write resumes that bypass ATS filters.
And the thing is, it's not even cheating in the traditional sense—it's just people using the tools available to them. The problem is deeper: we’re assessing the wrong things in the first place. Should we care if someone uses AI to solve a take-home if they’ll use AI on the job anyway? Should we really be judging an engineer’s ability based on whether they remember how to reverse a linked list under pressure?
We’re stuck in a system that’s easy to game and hard to justify. Has anyone seen companies doing this better? How are you adjusting your hiring process in the age of AI?