Tell HN: Stop Accepting Shitty Interviews
126 comments
> Stop accepting shitty interviews
Why should I stop? My personal experience is that many of the best companies to work for have shitty interviews. All I need to do is to study a bit and go through a 5 hour interview and I have a cushy high paying job? Sign me up. Complaining about interviews is missing the forest from the trees, I don't spend the majority of my working life interviewing, I spend it working. So if preparing a lot for interviews and getting my ego bruised a little means I get a good job so be it.
Why should I stop? My personal experience is that many of the best companies to work for have shitty interviews. All I need to do is to study a bit and go through a 5 hour interview and I have a cushy high paying job? Sign me up. Complaining about interviews is missing the forest from the trees, I don't spend the majority of my working life interviewing, I spend it working. So if preparing a lot for interviews and getting my ego bruised a little means I get a good job so be it.
What a strangely adversarial comment. In my experience, I can gather more information about a candidate's ability in just 15 minutes talking 1 on 1 with them rather than hours of Leetcode busywork. Funneling candidates through an annoying, needlessly rigid, and blatantly time consuming process is not a good thing, and probably leads to perfectly qualified people being denied; a system that optimizes for 'signals' that should not exist in the first place.
To be fair the parent commenter's post doesn't claim that this process is even remotely efficient, only that it's reality and they find it easy to game.
Also more meta-level commentary, I find the word "should" interesting, as in general I have found that comments on HN which contain this word reflect a level of frustration that X is X instead of Y, because "X should be Y". Someone else may come along and say "X should be Z", and all the meanwhile, X is X.
Also more meta-level commentary, I find the word "should" interesting, as in general I have found that comments on HN which contain this word reflect a level of frustration that X is X instead of Y, because "X should be Y". Someone else may come along and say "X should be Z", and all the meanwhile, X is X.
The problem comes when a majority of us agree that X is not fit for purpose as it is, but we still get people saying "X is X, why are we talking about Y and Z?"
What does adversarial mean in this context? That I disagree with the original poster?
> In my experience, I can gather more information about a candidate's ability in just 15 minutes talking 1 on 1 with them rather than hours of Leetcode busywork
This is your perspective as someone doing hiring. My perspective is from someone being hired. I really don't think leetcode is that difficult all things considered so a system based on leetcode is just not that bad in my opinion. It's really not that difficult, there is a set of 15 or so algorithms that are all 40 or so lines long and you just need to memorize them to the point of instant recall. Now you can get hired almost anywhere.
> In my experience, I can gather more information about a candidate's ability in just 15 minutes talking 1 on 1 with them rather than hours of Leetcode busywork
This is your perspective as someone doing hiring. My perspective is from someone being hired. I really don't think leetcode is that difficult all things considered so a system based on leetcode is just not that bad in my opinion. It's really not that difficult, there is a set of 15 or so algorithms that are all 40 or so lines long and you just need to memorize them to the point of instant recall. Now you can get hired almost anywhere.
Even if you're correct about your ability to sniff out talent you don't scale, and you can't be implemented by someone with eight hours of training, or replaced by automated testing, or a pencil and paper test. Legible processes that are known to work to within some tolerance are great for large organisations.
If you can tell if someone's good within fifteen minutes of talking to them you should be able to get rich by setting up a software consulting company or startup with other people's false negative rejects. That's equivalent to being able to take mine tailings and spit out gold.
If you can tell if someone's good within fifteen minutes of talking to them you should be able to get rich by setting up a software consulting company or startup with other people's false negative rejects. That's equivalent to being able to take mine tailings and spit out gold.
Aren't these shitty interview processes just gates and proxies for commitment and a dark pattern per se? If someone is willing to do the work and endure the shitty process to get the job then they must be committed to get the job and will likely be committed to keep the job.
That commitment can be exploited.
It's like Nigerian Prince scams, those willing to jump through the hoops are more easily to convince and get jumping through future hoops.
It's also likely they will rationalize that their job is great because they had to do all this shit to get it. The employer could think "if we can get their foot in the door and convince them to fully enter the room even though we make it difficult to fully open the door, they're less likely to leave and more likely to fight to stay in the room." It keeps uncommitted folks out, but doesn't necessarily filter for competent people, just committed people.
Cognitive dissonance is a powerful thing.
That commitment can be exploited.
It's like Nigerian Prince scams, those willing to jump through the hoops are more easily to convince and get jumping through future hoops.
It's also likely they will rationalize that their job is great because they had to do all this shit to get it. The employer could think "if we can get their foot in the door and convince them to fully enter the room even though we make it difficult to fully open the door, they're less likely to leave and more likely to fight to stay in the room." It keeps uncommitted folks out, but doesn't necessarily filter for competent people, just committed people.
Cognitive dissonance is a powerful thing.
So in those 15 minutes, how do you know this person is able to write good code? How do you know they are not just riding on the success of their former colleagues, and bullshitting you with high level responses they heard their colleages talk about?
How many people did you interview and hire?
Just asking since I interview and hire a lot of people, and find it very hard to asses someones capabilities.
How many people did you interview and hire?
Just asking since I interview and hire a lot of people, and find it very hard to asses someones capabilities.
That question works both ways:
In those few LC questions, how do I know the person can write good, maintainable, reliable software with a team of engineers and didn't just learn for the test?
Yes, interviews are hard. Assessing a persons capabilities is a difficult job.
In those few LC questions, how do I know the person can write good, maintainable, reliable software with a team of engineers and didn't just learn for the test?
Yes, interviews are hard. Assessing a persons capabilities is a difficult job.
> In my experience, I can gather more information about a candidate's ability in just 15 minutes talking 1 on 1 with them rather than hours of Leetcode busywork.
You think you can gather more information. I doubt that your decision is more accurate than leetcode interviews on average.
You think you can gather more information. I doubt that your decision is more accurate than leetcode interviews on average.
I've done over 800 interviews, and I call bullshit on this.
I've spent 30min talking to candidates about their projects, experience, tech they used, choices made, etc. and been super excited and positive about them.
Then I ask them to find the intersection of two lists in better than O(n^2) and they completely fail to write functional code in their preferred language in the next 30 minutes. This isn't even "leetcode" level of challenge.
If I'm hiring you to be a professional software engineer for 6 figures income to spend 8-10 hours a day designing, writing, and debugging code, you need to be able to write code.
The disparity between the conversation and the coding performance can be significant, and both need to be evaluated.
If there's a better way to evaluate if someone can write code, I'm open to try it. But take home tests, portfolios, pair-programming, "a day in the life", etc. all have their problems and disadvantage different groups.
I've spent 30min talking to candidates about their projects, experience, tech they used, choices made, etc. and been super excited and positive about them.
Then I ask them to find the intersection of two lists in better than O(n^2) and they completely fail to write functional code in their preferred language in the next 30 minutes. This isn't even "leetcode" level of challenge.
If I'm hiring you to be a professional software engineer for 6 figures income to spend 8-10 hours a day designing, writing, and debugging code, you need to be able to write code.
The disparity between the conversation and the coding performance can be significant, and both need to be evaluated.
If there's a better way to evaluate if someone can write code, I'm open to try it. But take home tests, portfolios, pair-programming, "a day in the life", etc. all have their problems and disadvantage different groups.
I don't see anything adversarial, it is just someone describing how they play the interview game.
There's a whole leetcode culture on sites like teamblind.com. A lot of people are happy to play the game.
Not my cup of tea either, but to each their own.
There's a whole leetcode culture on sites like teamblind.com. A lot of people are happy to play the game.
Not my cup of tea either, but to each their own.
I get your point, but if every good/senior developer stopped accepting "shitty interviews" the companies would be forced to change their methods and you would be surrounded by better engineers instead of people who studied a bit to go through a 5 hour interview.
Exactly. Nothing is more valuable than a coworker who can implement a half-ass hashtable with a linked list even though there are state-of-the-art implementations available in nearly every programming language.
I’ve got a sickness, and the only prescription is more bloom filters, baby
I’ve got a sickness, and the only prescription is more bloom filters, baby
Depends on where you sit, I'd take a half assed hash table if it consumes 1/10th of memory and code space of a reference library implementation.
Embedded space sadly suffers from a dearth of good implementations for different parameters than CPU cycles.
Embedded space sadly suffers from a dearth of good implementations for different parameters than CPU cycles.
You venn diagram of engineering talent has good engineers in the side that don't understand the value proposition I posted above? It feels silly saying this but the idea that people working at these tech companies are shitty programmers that only know competitive programming is wrong.
Engineering skill is orthogonal to willingness to put oneself through meaningless bullshit to get a job.
> people who studied a bit
Most of the population couldn't do 200 leetcode problems if they were given two months full time one on one tuition and told they'd get $100,000 in cash at the end. That would probably hold true even if you restricted it to college graduates and definitely would for high school graduates. Most people are really, really bad at mathematical thinking. It's more than just a bit of study.
Most of the population couldn't do 200 leetcode problems if they were given two months full time one on one tuition and told they'd get $100,000 in cash at the end. That would probably hold true even if you restricted it to college graduates and definitely would for high school graduates. Most people are really, really bad at mathematical thinking. It's more than just a bit of study.
>Most of the population couldn't do 200 leetcode problems
Most of the population are not programmers.
For someone who already knows a PL or 2, the picture is very different: There are maybe 32-50 questions that get repeated in almost every "code puzzle"-style interview. Which ones these are is actively monitored and curated in several communities.
A person intelligent enough to know a PL, can simply memorize these questions. "Learning for the test" is a standard, and successful, strategy to game predictable test environments. We see it in educational systems using standardized multiple choice tests, why would it be different in bad code interviews?
Most of the population are not programmers.
For someone who already knows a PL or 2, the picture is very different: There are maybe 32-50 questions that get repeated in almost every "code puzzle"-style interview. Which ones these are is actively monitored and curated in several communities.
A person intelligent enough to know a PL, can simply memorize these questions. "Learning for the test" is a standard, and successful, strategy to game predictable test environments. We see it in educational systems using standardized multiple choice tests, why would it be different in bad code interviews?
I was asked the stupid implement a calculator question at least five times.
Lol, given this market, by the time they solved 40 they'd be ready to get a 50K hiring bonus and 300K a year job at the FAANG. You need to offer 500K or more to even motivate someone to solve 200 leetcodes.
In my experience, a broken HR indicates a broken organization. Yes, you can hunker down and focus on your field and move on in a while but I can’t, usually.
As to shitty interviews. I was approached by Google for a non-programmer role a few years back. Failed the coding part as I’ve not been a professional programmer for a 20 years, despite being able to write all the code that I need for my research. Was told they’d be in touch for a development program as I’d apparently aced the rest of the interview. Never heard anything. Was approached again after a while and I agreed under the condition that there was no coding interview - I knew I had failed that and was going to fail again. Was assured there’d be no coding interview. Only to have one scheduled immediately.
So an organization that manages what’s probably the largest information system on the planet cannot for the love of their life implement a simple sales lead process? Why? Because they fundamentally do not care about you.
As to shitty interviews. I was approached by Google for a non-programmer role a few years back. Failed the coding part as I’ve not been a professional programmer for a 20 years, despite being able to write all the code that I need for my research. Was told they’d be in touch for a development program as I’d apparently aced the rest of the interview. Never heard anything. Was approached again after a while and I agreed under the condition that there was no coding interview - I knew I had failed that and was going to fail again. Was assured there’d be no coding interview. Only to have one scheduled immediately.
So an organization that manages what’s probably the largest information system on the planet cannot for the love of their life implement a simple sales lead process? Why? Because they fundamentally do not care about you.
Yes, if you know the job is good go for it, it's a minor hurdle
Otherwise you might not just bother, because it's usually a reflection of what the company values and how it thinks
Otherwise you might not just bother, because it's usually a reflection of what the company values and how it thinks
Funny enough, I was looking for jobs relatively recently and decided to apply to cruise but didn't bother because they wanted a cover letter.
Yes, it’s easy to forget we are in of the highest paying industry. But yeah interviews should skip some redtape and HR bs.
Still, it's not like any other field of engineering has something equivalent to leetcode. You don't make someone with a civil engineering degree and a resume of bridge building go up to a whiteboard and work out an irrelevant physics problem like it's undergrad again. You don't have an M.D. talk you through a hypothetical surgery. Only in computer science is there this metagame that you have to dedicate unpaid time toward, yet is hardly ever relevant to the tasks outlined in the job description. I feel like it's been pushed onto recruiters by companies that specialize in creating this crap like most other Enterprise software sold to HR teams, and now there is this expectation that this is how you are supposed to do these interviews because it's been going on for years now. Snake oil salesmen and lemmings in HR brought this upon us, not technical leadership demanding competency.
Civil Engineers and Doctors have professional boards that certify they meet a minimum level of competency.
I've done an EE "leetcode" phone interview with a non-technical HR person asking scripted problems. Didn't get to a real interview, then they foolishly called me back a year later where I just told them not again. This is for a company in the hinterlands of Oregon where you're making a sacrifice to work for the only viable employer in the area.
Is this Intel? The people of Hillsboro would resent you referring to their Portland suburb as "the hinterlands of Oregon"
And other fields of engineering don’t pay nearly as well.
I’d choose a much higher pay over a reasonable interview.
I’d choose a much higher pay over a reasonable interview.
In interview design there are a lot of trade offs.
Coding something quick within an hour? "It's too time constrained."
Take home interview question? "It takes way too long for the busy professional who already has a job, and plus, even if we time constrain it, they'll put way more time in. Or they'll cheat with friends."
Ask to see previous code? "Many good candidates do not do open source and couldn't tell us what they work on in detail due to NDAs."
Don't use automated interviewing for initial rounds? "We are wasting a lot of expensive engineering time on poor results"
Just ask about work experience and make a subjective call? "We don't know if they actually know how to code, and it's way too random, prone to bias and opens us to lawsuits."
Use different interview methods for different candidates? See above
Watch over a person live code? "Too anxiety provoking". Don't? "How about cheating?"
Trial to work for 2 weeks? "Immigrants cannot do that, and people wont leave their stable job for an unsure offer. So only the unemployed could do it, and why are they unemployed?"
etc, etc
Also there is the entire mountain of going over glacial HR departments and getting them to change their interview processes.
I agree leetcode is shit and I made it a personal mission to change the processes at work to remove the leetcode and add more work sample interviewing, but it's difficult!
Coding something quick within an hour? "It's too time constrained."
Take home interview question? "It takes way too long for the busy professional who already has a job, and plus, even if we time constrain it, they'll put way more time in. Or they'll cheat with friends."
Ask to see previous code? "Many good candidates do not do open source and couldn't tell us what they work on in detail due to NDAs."
Don't use automated interviewing for initial rounds? "We are wasting a lot of expensive engineering time on poor results"
Just ask about work experience and make a subjective call? "We don't know if they actually know how to code, and it's way too random, prone to bias and opens us to lawsuits."
Use different interview methods for different candidates? See above
Watch over a person live code? "Too anxiety provoking". Don't? "How about cheating?"
Trial to work for 2 weeks? "Immigrants cannot do that, and people wont leave their stable job for an unsure offer. So only the unemployed could do it, and why are they unemployed?"
etc, etc
Also there is the entire mountain of going over glacial HR departments and getting them to change their interview processes.
I agree leetcode is shit and I made it a personal mission to change the processes at work to remove the leetcode and add more work sample interviewing, but it's difficult!
Following a comment I saw on HN, we now have several code snippets that we ask them to read through (biggest is like 12 lines) and tell us, at a high level, what it's trying to do. Great results, we've found people who learnt all the lingo completely failed when it came to reading and interpreting code.
Yeah I like the 'debug the real code block' style of interview too!
Just pretty much anything that is not leetcode bingo is a real improvement.
Just pretty much anything that is not leetcode bingo is a real improvement.
I can't give you enough upvotes. I also do a lot of interviews and hire people, and it's just very hard.
Plus, you only know the false positives, but never the false negatives.
And I also think that a lot of people who never interview candidates don't realize how many of them can't even code the most basic stuff.
Plus, you only know the false positives, but never the false negatives.
And I also think that a lot of people who never interview candidates don't realize how many of them can't even code the most basic stuff.
So, the answer is to ask easier leetcodes. I unironically only ask fizzbuzz and then domain specific questions (in my case it's ML/NLP). You can even design domain specific "leetcode like" questions for your domain. In my case, it could be to come up with a very simple implementation of K-NN. I accept psudocode and leave the editor, compiler (err, run button...) And Google open as resources that they can use during this time.
But companies won't do this! Bloody Google of all places had me do a code interview in a Google doc!. Imagine my horror as I am fighting indentation issues while trying to come up with the lc hard solution he asked in 45 mins.
But companies won't do this! Bloody Google of all places had me do a code interview in a Google doc!. Imagine my horror as I am fighting indentation issues while trying to come up with the lc hard solution he asked in 45 mins.
Google is pretty bad in general with their 5 leetcode policy :p
But that is what hacker rank and automated interviewing is supposed to be usually, an automated leetcode easy.
But that is what hacker rank and automated interviewing is supposed to be usually, an automated leetcode easy.
Reading this it seems like the company's objective isn't to hire someone but to find every possible reason not to hire them.
This are real responses and complaints about changing interviews from leetcode type to not leetcode type, or to fix other common issues and anxieties that people have in interviews. Every interview type has issues and you have to choose as a company what negatives you want to accept for which items you want to favor.
And yes, a bias towards avoiding the false positive is a explicit goal of many engineering interviews, and I think google has said so explicitly.
And yes, a bias towards avoiding the false positive is a explicit goal of many engineering interviews, and I think google has said so explicitly.
So what do you propose as a replacement?
In your case, they should've certainly dug into your open source, at least if it was directly in their line of business.
I certainly love it when I ask a candidate "tell me about a project you found interesting" and they can go into details, talk about tradeoffs, etc. Sadly, most candidates can't. They can't come up with anything better than "well I rewrote the thing, and it was challenging since I hadn't used Kotlin before, learning Kotlin was interesting." So at that point I'm unimpressed, but I'll give them a chance to code still because (a) not everybody has had opportunities to work on interesting stuff and (b) I wouldn't have enough people to hire otherwise.
But if "has open source software we use" is the bar, you're gonna fail a lot more good engineers than you would using algo questions.
In your case, they should've certainly dug into your open source, at least if it was directly in their line of business.
I certainly love it when I ask a candidate "tell me about a project you found interesting" and they can go into details, talk about tradeoffs, etc. Sadly, most candidates can't. They can't come up with anything better than "well I rewrote the thing, and it was challenging since I hadn't used Kotlin before, learning Kotlin was interesting." So at that point I'm unimpressed, but I'll give them a chance to code still because (a) not everybody has had opportunities to work on interesting stuff and (b) I wouldn't have enough people to hire otherwise.
But if "has open source software we use" is the bar, you're gonna fail a lot more good engineers than you would using algo questions.
I ask the same thing you do: "Tell me about a project you've worked on." There is tons of ground we can cover with that:
* implementation choices with pros and cons
* details about how the underlying system works
* domain knowledge, where applicable
* sample code where the candidate writes to mimic something from the system
My interview style has always been: "Tell me what you know, and we'll see if that's true or not." Surprisingly, most candidates fail even that since they don't seem to understand their own projects. But I find this approach the fairest of all since it allows the candidate to lead the interview and it doesn't assume anything about the candidate's knowledge or prior experience.
* implementation choices with pros and cons
* details about how the underlying system works
* domain knowledge, where applicable
* sample code where the candidate writes to mimic something from the system
My interview style has always been: "Tell me what you know, and we'll see if that's true or not." Surprisingly, most candidates fail even that since they don't seem to understand their own projects. But I find this approach the fairest of all since it allows the candidate to lead the interview and it doesn't assume anything about the candidate's knowledge or prior experience.
I once would've completely agreed with your point idea. But since then, I've had multiple experiences of talking with someone who made a great impression, super knowledgeable, but couldn't actually write any code in a real-world setting.
This approach simply doesn't always work. Some people are amazing-sounding but terrible in practice. The reverse is also true - some people are terrible at talking about their work, but produce amazing code. (Their lack of communication skills might or might not hamper their actual job performance, depending on multiple factors.)
This approach simply doesn't always work. Some people are amazing-sounding but terrible in practice. The reverse is also true - some people are terrible at talking about their work, but produce amazing code. (Their lack of communication skills might or might not hamper their actual job performance, depending on multiple factors.)
I agree that the interview can't be just war stories. There has to be code in there somewhere, like whiteboard sketches of an algorithm the candidate has implemented.
On the other hand, leet code questions definitely don't show that the candidate can actually write maintainable, well-documented code with test suites and easy-to-use APIs.
On the other hand, leet code questions definitely don't show that the candidate can actually write maintainable, well-documented code with test suites and easy-to-use APIs.
Yes, luckily those aren't the only two options :)
Though I will say that having some kind of fizz-buzz style phone screen is still, in my mind, a good idea. It saves a lot of time if it turns out that your candidate can't even write a simple for loop (and so many can't, to my great dismay).
Though I will say that having some kind of fizz-buzz style phone screen is still, in my mind, a good idea. It saves a lot of time if it turns out that your candidate can't even write a simple for loop (and so many can't, to my great dismay).
I work in the same industry as you and I agree. Your problems are never how to reverse a linked list, or finding the missing number. It's always tradeoffs: why does the business benefit from this technical decision over that one? What's the point of this language? What do you need in terms of hardware? How do you organise the code to suit the needs?
The interview that's always worked for me in hiring is where we talk about the issue and we see if the person understands the problem. There's an infinite number of things we can get into, and if the person is experienced they'll never run out of things to say. I've never hired someone who turned out not to know the basic things like some particular algorithm.
I suppose if we're talking the leetcode style interview loops, they normally have a system design part. That might be the closest to what I do.
The interview that's always worked for me in hiring is where we talk about the issue and we see if the person understands the problem. There's an infinite number of things we can get into, and if the person is experienced they'll never run out of things to say. I've never hired someone who turned out not to know the basic things like some particular algorithm.
I suppose if we're talking the leetcode style interview loops, they normally have a system design part. That might be the closest to what I do.
"We need to serve 1TB of data. How would you do it?"
Start there. Let the candidate walk through the problem. Ask questions, and introduce hypothetical limitations. "We need to display the data in a table on a webpage." (frontend) ... "We need an API that can query the data." (backend) ... "We need to sort the data." (discuss algos) It's not that hard to find people's strengths, and make sure you're hiring qualified folks.
Start there. Let the candidate walk through the problem. Ask questions, and introduce hypothetical limitations. "We need to display the data in a table on a webpage." (frontend) ... "We need an API that can query the data." (backend) ... "We need to sort the data." (discuss algos) It's not that hard to find people's strengths, and make sure you're hiring qualified folks.
And, maybe, c) not many people have both strong verbal skills and strong technical skills?
While I agree in principle, the average developer is not going to have released and maintained an open source project. Let alone one that is used. Let alone one that is widely used. People in your position should rightfully do that, you have all the leverage. Other people not so much, especially new graduates. I agree that having tasks that simulate the real environment is the goal but we also work in an industry that it is common place for productivity to only really start after 6 months. Thus, any interview is going to be some simulacrum of the real thing. Personally, I would like to see something like "Make 10 commits to X open source project and we'll talk". Make the world a better place while actually evaluating the candidate for job readiness. Both technically and behaviorally as you see how they behave in the community.
Not every developer has the time and resources to dedicate to FOSS. Some of us spend most of their working time writing software for, well, the company we work for, which won't suddenly open source their IP.
Making commits to FOSS a general requirement in interviews would exclude a whole lot of highly skilled and experienced engineers, limit the chances of beginners, and lead to loads and loads of BS open source projects that exist solely to fatten resumes.
Making commits to FOSS a general requirement in interviews would exclude a whole lot of highly skilled and experienced engineers, limit the chances of beginners, and lead to loads and loads of BS open source projects that exist solely to fatten resumes.
First, 10 commits to an open source project is not "dedication". It feels like table stakes if we want a meritocratic process. Second, where do people get the time to work on leetcode problems? Replace that with open source contributions and/or open blog posts. I do agree that it would probably lead to a glut but that makes the interviewer's job incredibly easy if a commit is BS or not.
It seems you are asking for a miracle: an interview that properly assesses the candidate's job readiness without time involvement. How much time do people spend preparing for interviews and then have to do it again the next time they want to interview? At least with this system, you have a history that is reusable.
It seems you are asking for a miracle: an interview that properly assesses the candidate's job readiness without time involvement. How much time do people spend preparing for interviews and then have to do it again the next time they want to interview? At least with this system, you have a history that is reusable.
People never talk about the most important metric for interviews: your target rejection rate. Google and Facebook need to do these interviews because of the number of applicants they get. Any process they design must definitionally reject 99% of people who go through it, and they found it is essentially impossible to hit that kind of rejection rate with just 'tell me about your work' questions.
If your startup operates with a 50% or 75% target rejection rate, you absolutely can and should operate differently, and this could be an important competitive advantage for you.
If your startup operates with a 50% or 75% target rejection rate, you absolutely can and should operate differently, and this could be an important competitive advantage for you.
Doesn't this work like Harvard/Yale/Princeton/etc admissions, where if they selected a completely different set of applicants everyone there would just as likely be qualified and deserving of attending?
I swear sometimes it might just be easier or even fairer to throw all resumes/applications in the air and see which ones land inside of a predetermined circle.
I swear sometimes it might just be easier or even fairer to throw all resumes/applications in the air and see which ones land inside of a predetermined circle.
> I swear sometimes it might just be easier or even fairer to throw all resumes/applications in the air and see which ones land inside of a predetermined circle.
When I was a responsible for hiring interns, we used to joke about this all the time. Just throw away 75% of applicants, everyone that got thrown out is unlucky, and we don't want unlucky people bringing the company down.
When I was a responsible for hiring interns, we used to joke about this all the time. Just throw away 75% of applicants, everyone that got thrown out is unlucky, and we don't want unlucky people bringing the company down.
Yes, but that would destroy the illusion of a meritocracy and the idea that these companies are basing their hiring decisions on rational, objective metrics and not simply making it up as they go along.
Steve Yegge made that observation many years ago regarding Google. For every person working at Google, there is a panel of interviewers that would have rejected that person.
Steve Yegge made that observation many years ago regarding Google. For every person working at Google, there is a panel of interviewers that would have rejected that person.
The Monte Carlo hiring technique?
They don't have a target rejection rate. They have a target quality threshold. More software engineers that can productively employed is good for them. It's not like Harvard or other institutions where the point is to be elite, not to do something. The more SEs Google can hire the better. If they could reduce their false negative rate in half that would materially impact their bottom line. False negative for Harvard is basically undefined. They could easily dectuple the number of students they admit without compromising on academic quality or family wealth and making only very small declines in average power level (dictator, princes, senator, governor children)
Can't they simply hire the first eligible person? One can go through resumes and filter out people who don't meet objective criteria(e.g. work experience, location). This should be trivial. Then once they have 30 good resumes, they can call these people for interview. Assuming 10% chance of each candidate meeting the job requirements, they have >95% chance of finding a good candidate.
With automated resume pruning, it shouldn't take more than two hours for initial pruning. Actually, the higher number of applications means they can put in stricter pruning conditions, leading to lesser manual pruning later on. Assuming 2 hours/person interviewing, it should take less than a 40 hours-person to close a position.
I'd guess there's also some benefit w.r.t company loyalty when hiring a good-enough candidate instead of the best-available. They would be less likely to switch jobs compared to someone who can solve a leetcode hard problem in 15 minutes.
With automated resume pruning, it shouldn't take more than two hours for initial pruning. Actually, the higher number of applications means they can put in stricter pruning conditions, leading to lesser manual pruning later on. Assuming 2 hours/person interviewing, it should take less than a 40 hours-person to close a position.
I'd guess there's also some benefit w.r.t company loyalty when hiring a good-enough candidate instead of the best-available. They would be less likely to switch jobs compared to someone who can solve a leetcode hard problem in 15 minutes.
Rejection rate has been discussed many times. It is not the most important metric since it's just an observation. You can't start with a desired rejection rate and come up with a good interview process. What you can do is know what kind of person you want to reject or select for, design a process to do that, and then adjust other factors (listed compensation, recruiter talent, marketing site, etc.) so that you have enough candidates to which you want to extend offers.
Few companies are actually in a position, given their knowledge, stability of their industry, and marginal quality of hiring success, to succeed at deliberately loosening or tightening their interview process to arrive at just the right number of final candidates.
Few companies are actually in a position, given their knowledge, stability of their industry, and marginal quality of hiring success, to succeed at deliberately loosening or tightening their interview process to arrive at just the right number of final candidates.
I totally agree that asking people who created/maintain/make significant contributions to open source projects is a waste of time.
However, for hiring people who don't have a public body of work to point to - it remains pretty effective. (I am not affiliated with hackerrank/hackerearth/codility - just been on the hiring side for the last many years)
Also, in my current org - we've been able to trust people lacking the "expected" graduate degrees - solely because they were able to prove their acumen in the interview by solving algo-data-structure problems. (Of course, we have to be creative here - and avoid like the plague the same bunch of questions which are present all over the web)
However, for hiring people who don't have a public body of work to point to - it remains pretty effective. (I am not affiliated with hackerrank/hackerearth/codility - just been on the hiring side for the last many years)
Also, in my current org - we've been able to trust people lacking the "expected" graduate degrees - solely because they were able to prove their acumen in the interview by solving algo-data-structure problems. (Of course, we have to be creative here - and avoid like the plague the same bunch of questions which are present all over the web)
On HN and elsewhere, there are so much deeply personalized emotions regarding LeetCode-style interviews. Instead of focusing on personal specifics, I am more curious about it from a social and systemic perspective.
1. If LeetCode problems are not only irrelevant, but actually turning away good employees, why aren't more companies shunning LeetCode problems? Tech is a very competitive field, and in the current climate of low interest rate, we (at least in the West) have more money than talent available. An employer with non-LeetCode interviews would have a significant market advantage. Is Google (and Netflix) being completely stupid even with their rigorous introspective take on hiring? If LeetCode is terrible for both the employer and the employee, why is it so popular? Are we all cargo-culting?
2. What is a better solution? How do we know it's better? There are proposed solutions, here and elsewhere, that make the candidates feel better about themselves, but for employers interested in hiring technically competent engineers, what is the solution? Requiring everyone to have a popular git repository?
1. If LeetCode problems are not only irrelevant, but actually turning away good employees, why aren't more companies shunning LeetCode problems? Tech is a very competitive field, and in the current climate of low interest rate, we (at least in the West) have more money than talent available. An employer with non-LeetCode interviews would have a significant market advantage. Is Google (and Netflix) being completely stupid even with their rigorous introspective take on hiring? If LeetCode is terrible for both the employer and the employee, why is it so popular? Are we all cargo-culting?
2. What is a better solution? How do we know it's better? There are proposed solutions, here and elsewhere, that make the candidates feel better about themselves, but for employers interested in hiring technically competent engineers, what is the solution? Requiring everyone to have a popular git repository?
> why aren't more companies shunning LeetCode problems
Because its easier than talking to candidates and because many companies emulate FAANG, regardless of wheter that makes sense or not. If I am that big a deal, I NEED a system to filter, because I will always have a surplus of applicants, no matter what the market looks like. If I am a small company, or even a medium sized shop that has difficulty filling positions anyway, then doing the same doesn't make sense.
Not because my standards are lower, but the amount of applications I get are. Same as I don't build a webservices architecture to handle 10E6 req./h when it will serve less than a 1000, it doesn't make sense to apply methods designed for "hiring at scale" if I am not operating at that scale.
> What is a better solution?
Checking credentials, talking to people, about their past work, about how they would tackle example problems in the real world, their view on solutions.
aka. doing actual interviews instead of taking the easy route and just reading a puzzle question from a sheet of paper and waiting behind them with a stopwatch.
If I want competent people, I need to do competent hiring.
Because its easier than talking to candidates and because many companies emulate FAANG, regardless of wheter that makes sense or not. If I am that big a deal, I NEED a system to filter, because I will always have a surplus of applicants, no matter what the market looks like. If I am a small company, or even a medium sized shop that has difficulty filling positions anyway, then doing the same doesn't make sense.
Not because my standards are lower, but the amount of applications I get are. Same as I don't build a webservices architecture to handle 10E6 req./h when it will serve less than a 1000, it doesn't make sense to apply methods designed for "hiring at scale" if I am not operating at that scale.
> What is a better solution?
Checking credentials, talking to people, about their past work, about how they would tackle example problems in the real world, their view on solutions.
aka. doing actual interviews instead of taking the easy route and just reading a puzzle question from a sheet of paper and waiting behind them with a stopwatch.
If I want competent people, I need to do competent hiring.
> 1. If LeetCode problems are not only irrelevant, but actually turning away good employees, why aren't more companies shunning LeetCode problems?
At the margin, most companies (probably close to 100% for many sectors) do not make money from code quality. I'd argue it's actually pretty far from the margin for most companies. See the overwhelming shittiness of almost every software product that's been around for more than a couple of years. Almost all products get worse over time but still some companies make more money. IP and market position are much more important. I think this can also explain why so many very successful companies never launch a successful second product: they didn't initially succeed because they're so much better at development than everyone else, they succeeded because of the serendipity of the initial product.
At the margin, most companies (probably close to 100% for many sectors) do not make money from code quality. I'd argue it's actually pretty far from the margin for most companies. See the overwhelming shittiness of almost every software product that's been around for more than a couple of years. Almost all products get worse over time but still some companies make more money. IP and market position are much more important. I think this can also explain why so many very successful companies never launch a successful second product: they didn't initially succeed because they're so much better at development than everyone else, they succeeded because of the serendipity of the initial product.
For junior developers algorithm questions is a test of their technical talent. For senior developers algorithm questions is a test of their soft skills. Most companies are happy that it helps them filter out diva behavior.
Not going to lie, I actually like Leetcode style interviews. What you put in is what you get - you know how to control your own outcomes, and that's by putting in the time and work to study for the questions.
People shouldn't have to cram little tricks and algorithms into their brain just to pass a Leetcode challenge; that's the point. No real-world programming career is driven by such pointless exercises.
I would think that real world programming careers are affected by someone's ability to remember information and put in effort, and that's demonstrated by the time spent to study for the little tricks and algorithms that go into passing an interview. I think it makes things a lot less subjective - you put in the time, you win.
I don't think that expecting prospective developers to spend lots of unpaid time on a skill that's ultimately not going to be useful to the vast majority of them should be normalized.
Do you expect medical doctors to do an unpaid "trial" surgery as part of their interview? Do you expect lawyers to litigate a case for free before they'll be considered for a job?
As far as I know software engineering is the only field where in addition to experience & education you have to waste time gaming the interview process. Every other field seems to get away just fine hiring based on experience and/or education.
Do you expect medical doctors to do an unpaid "trial" surgery as part of their interview? Do you expect lawyers to litigate a case for free before they'll be considered for a job?
As far as I know software engineering is the only field where in addition to experience & education you have to waste time gaming the interview process. Every other field seems to get away just fine hiring based on experience and/or education.
Yes...but the bar to entry here is also a major factor. Doctors and lawyers in part have already been vetted by lots of crap before even qualifying for an interview.
How much of the content on the MCAT is "useful" for a doctor's daily work? I would argue that the majority of it is moot. In the same vein, the LSAT has a lot of "useless" logic puzzles that aren't applicable to their daily work as a lawyer. Fact of the matter is, we're lucky in tech to have this system that ANYONE can study for "useless" shit, doesn't matter your background, what college you went to, etc. If you get the interview and nail it, you're in.
How much of the content on the MCAT is "useful" for a doctor's daily work? I would argue that the majority of it is moot. In the same vein, the LSAT has a lot of "useless" logic puzzles that aren't applicable to their daily work as a lawyer. Fact of the matter is, we're lucky in tech to have this system that ANYONE can study for "useless" shit, doesn't matter your background, what college you went to, etc. If you get the interview and nail it, you're in.
I prefer interviews that ask leetcode questions because they are standard. I can prepare once and interview with multiple companies. The alternatives are take-home projects, pair-programming sessions which in my opinion are very subjective and more time-consuming.
This reminds me of a blog posted a while a go
https://joelgrus.com/2016/05/23/fizz-buzz-in-tensorflow/
https://joelgrus.com/2016/05/23/fizz-buzz-in-tensorflow/
Yup, there are countless employers looking to hire coders. If your spidey senses tingle, end the interview and move on to the next one.
I've got a little anecdote on an interview that I should have ended, but I was a junior so I didn't really know better at the time.
At this interview, one of the things they wanted me to do was to do a series of one minute coding challenges. They weren't leet code or anything, but just difficult enough that they required a lot of knowledge, and I wasn't allowed to look up documentation. This one guy stood behind me and looked over my shoulder with a stop watch. After a minute was up, I had to put down the code I was writing and move on to the next one, and the process would repeat. I think I did about 12 of those. It was some of the most ridiculous bullshit I ever did in an interview. If you didn't remember the exact array method off the top of your head then you were just screwed. Mind you, they were hiring for a junior position. These days there's a possibility I'd survive it, but I wouldn't even bother at this point.
I've got a little anecdote on an interview that I should have ended, but I was a junior so I didn't really know better at the time.
At this interview, one of the things they wanted me to do was to do a series of one minute coding challenges. They weren't leet code or anything, but just difficult enough that they required a lot of knowledge, and I wasn't allowed to look up documentation. This one guy stood behind me and looked over my shoulder with a stop watch. After a minute was up, I had to put down the code I was writing and move on to the next one, and the process would repeat. I think I did about 12 of those. It was some of the most ridiculous bullshit I ever did in an interview. If you didn't remember the exact array method off the top of your head then you were just screwed. Mind you, they were hiring for a junior position. These days there's a possibility I'd survive it, but I wouldn't even bother at this point.
I think OP implied a better solution when mentioning interviews that mimic the job: present interviewees with a small take-home case study based on a real on-the-job problem. I have done a few of these and they have been much more enlightening experiences than any paired programming boilerplate live coding exercise. Even in situations in which my proposed solution weren’t up to snuff to land the job, the conversation with the team about my solution felt like an actual representation of what it would be like to be a member of the team.
> present interviewees with a small take-home case study based on a real on-the-job problem
In my experience these are terrible. It becomes an arms race, what the company intended to become a 5 hour take home assignment becomes a 20 hour assignment because everyone else that's doing the take home assignment is also spending 20 hours. Take homes are a trap.
In my experience these are terrible. It becomes an arms race, what the company intended to become a 5 hour take home assignment becomes a 20 hour assignment because everyone else that's doing the take home assignment is also spending 20 hours. Take homes are a trap.
Personally, I like the idea of objective based take homes rather than time based. Then just have them check everything in on git so you can see how they work.
I have interviews lined up with google and coinbase which both are going to ask leet code style questions. I'm going to go through it just for the promise of more $.
Even if it is annoying to spend hours and not get an offer, I can see why they ask these types of questions to see your problem solving ability first hand.
I would expect any decent engineer to be able to solve leet code style question, or at the very least be able to model the problem reasonably well and give a recurrence. Even if the code contains errors or there's missing edge cases then it allows you to see how someone approaches a problem.
Leet code style questions are basically a bunch of combinatorial algorithms. Even if they seem like a different trick each time there's general techniques that can be used to approach each problem.
Even if it is annoying to spend hours and not get an offer, I can see why they ask these types of questions to see your problem solving ability first hand.
I would expect any decent engineer to be able to solve leet code style question, or at the very least be able to model the problem reasonably well and give a recurrence. Even if the code contains errors or there's missing edge cases then it allows you to see how someone approaches a problem.
Leet code style questions are basically a bunch of combinatorial algorithms. Even if they seem like a different trick each time there's general techniques that can be used to approach each problem.
You get to choose between a lot of factors when deciding where to apply:
Location, pay, workload, social aspects, stocks. Not to mention work environment, chance at promotion, career progression. Deciding on interviewing types can be part of the equation.
Changing employer's tactics is not going to happen at the interview feedback level. This movement needs a gurus/books and a cargo cult mantra which filters into magazine articles and popular culture.
You are better off going in a different direction and using this information to hire the best employees for your own company or offering a service that places your candidates in roles.
Location, pay, workload, social aspects, stocks. Not to mention work environment, chance at promotion, career progression. Deciding on interviewing types can be part of the equation.
Changing employer's tactics is not going to happen at the interview feedback level. This movement needs a gurus/books and a cargo cult mantra which filters into magazine articles and popular culture.
You are better off going in a different direction and using this information to hire the best employees for your own company or offering a service that places your candidates in roles.
I understand and empathize with the sentiment here. Interviews have a skewed power-dynamic. A handful of people, or sometimes even a single person, can make or break your chances at what you feel is your dream job, and there is nothing you can do about it. It is an extremely frustrating place to be in, indeed.
I've had people with half my years of experience, ask questions that were likely from an algorithms course they took during their (under)grad school days, with little regard for its applicability to the role. These are the kinds of red flags I look for, and while I am yet to walk out of an interview, I do make it a point to provide feedback to the recruiter, along with my decision to withdraw from the process.
It is also pretty telling when a company fails to account for the candidate's experience. At best, it shows a lack of experience with recruiting; at its worst, it is a good indicator of poor culture. Having a person with 10+ years of experience write a quick-sort algorithm is unlikely to produce a good result; mostly because any engineer with that kind of experience would not really write a sort function by hand, unless working on a very low-level system (and even those are mature enough to have an optimized sort function readily available for use).
Like interviewing, conducting an interview is a skill that needs to be developed, and any good company would take measures to ensure it does a good job of it. Recruiting, after all, is a pretty expensive affair.
You've likely dodged a bullet there. Count your blessings, and move on to greener pastures. :)
I've had people with half my years of experience, ask questions that were likely from an algorithms course they took during their (under)grad school days, with little regard for its applicability to the role. These are the kinds of red flags I look for, and while I am yet to walk out of an interview, I do make it a point to provide feedback to the recruiter, along with my decision to withdraw from the process.
It is also pretty telling when a company fails to account for the candidate's experience. At best, it shows a lack of experience with recruiting; at its worst, it is a good indicator of poor culture. Having a person with 10+ years of experience write a quick-sort algorithm is unlikely to produce a good result; mostly because any engineer with that kind of experience would not really write a sort function by hand, unless working on a very low-level system (and even those are mature enough to have an optimized sort function readily available for use).
Like interviewing, conducting an interview is a skill that needs to be developed, and any good company would take measures to ensure it does a good job of it. Recruiting, after all, is a pretty expensive affair.
You've likely dodged a bullet there. Count your blessings, and move on to greener pastures. :)
I still like to ask somebody if they can code a basic algorithm just to see if they can code at all.
It was common ten years ago. It was still possible in 2015 or so, at least in "B-round startups". I was still asked about my relevant previous experience. In hindsight, I believe people like me who ended up in companies like that were already trading down significantly. Or I have never been as smart as I thought.
My experience in the last year or so categorically states that leetcoding is the only game in town in CA. I hate it, I am bad at it, I find it humiliating and irrelevant. I was told by Asian teenagers that I am not good at coding after 20 years building software. I have failed multiple interviews because of dynamic programming&Co even at companies where I had a decade of highly relevant experience. Nobody ever wanted to hire me for my production experience with all of Scala/Kotlin/Java even though I routinely meet companies where they don't use Scala/Kotlin because of hiring difficulties.
It's just the way it is nowadays. The world is getting progressively crazy. If you're in a position to not accept such interviews I envy you big time. I don't see it happening even in smaller (i.e. not-FAANG compensation) companies though. Some European/remote companies occasionally give you a home coding challenge. But they are an exception.
My experience in the last year or so categorically states that leetcoding is the only game in town in CA. I hate it, I am bad at it, I find it humiliating and irrelevant. I was told by Asian teenagers that I am not good at coding after 20 years building software. I have failed multiple interviews because of dynamic programming&Co even at companies where I had a decade of highly relevant experience. Nobody ever wanted to hire me for my production experience with all of Scala/Kotlin/Java even though I routinely meet companies where they don't use Scala/Kotlin because of hiring difficulties.
It's just the way it is nowadays. The world is getting progressively crazy. If you're in a position to not accept such interviews I envy you big time. I don't see it happening even in smaller (i.e. not-FAANG compensation) companies though. Some European/remote companies occasionally give you a home coding challenge. But they are an exception.
I think this is a matter for the interviewer to solve. Smart ones should take advantage of this situation and grab the better developers that don't pass those stupid interviews. I posted this in comments before since it pretty much echos my thoughts on the matter: https://talktotheduck.dev/debugging-the-technical-interview-...
Reminds me of people who complain about dating. Dating was bad, at least before tinder, yet complaining didn’t improve much about your situation.
I myself am not pro-leetcode, but I see the advantages of leetcode from both the interviewer and the interviewee side.
From the interviewee side: It gives you a known set of material to prepare for (ofc there may be newer questions). But this alone helps a lot. With the explosion of frameworks, libraries, languages it is very hard to prepare for everything.
From the interviewer side: Again it will give you a standardized set of questions which can be asked and you can compare across candidate due to this.
>> about half of them admitted they already use my open-source software
Similar to how people don't have time to prepare for leetcode, there may also be people who don't have github repos with open source code etc. Take my case, I work for a company where I have to get permissions if i am gonna start a open source project, even though it may not be related at all to my current work.
From the interviewee side: It gives you a known set of material to prepare for (ofc there may be newer questions). But this alone helps a lot. With the explosion of frameworks, libraries, languages it is very hard to prepare for everything.
From the interviewer side: Again it will give you a standardized set of questions which can be asked and you can compare across candidate due to this.
>> about half of them admitted they already use my open-source software
Similar to how people don't have time to prepare for leetcode, there may also be people who don't have github repos with open source code etc. Take my case, I work for a company where I have to get permissions if i am gonna start a open source project, even though it may not be related at all to my current work.
> I work for a company where I have to get permissions if i am gonna start a open source project, even though it may not be related at all to my current work.
Is this common ? Or legal ?
Is this common ? Or legal ?
“You agree that everything you write is work for hire owned by us, in return for your salary” is pretty common in full-time employment contracts. Certain jurisdictions have chosen to pass laws limiting it, so I suppose it would be legal otherwise.
https://leginfo.legislature.ca.gov/faces/codes_displaySectio...
https://apps.leg.wa.gov/rcw/default.aspx?cite=49.44.140
For morale and tech brand reasons, some companies have a process to officially bless releases of work they might have had a good case for owning.
https://leginfo.legislature.ca.gov/faces/codes_displaySectio...
https://apps.leg.wa.gov/rcw/default.aspx?cite=49.44.140
For morale and tech brand reasons, some companies have a process to officially bless releases of work they might have had a good case for owning.
More efficent? maybe. More effective? debatable.
What you're describing are not advatanges as much as ways to make the job easier, without accounting for if it produces better results.
What you're describing are not advatanges as much as ways to make the job easier, without accounting for if it produces better results.
I'm curious about the legality of such practices. Your employer needs to authorize what you do on your own time and dime? What country/state are you located in?
I somewhat agree. I've been through a lot of terrible interviews. The last year or so I've started walking away when they get too annoying. For example, I refuse take home exercises. I used to do them, I don't anymore. Too much work to do on spec.
But everything involves trade offs. By being very selective of what I will put up with during the process, I've lost some opportunities. If I NEEDED a job, I would suddenly be much more tolerant.
Either way, I don't see the industry as a whole learning to interview better. When I refuse a particular exercise, or drop right out of the process, they don't blame themselves. They just move on to hazing the next candidate and hope for the best.
But everything involves trade offs. By being very selective of what I will put up with during the process, I've lost some opportunities. If I NEEDED a job, I would suddenly be much more tolerant.
Either way, I don't see the industry as a whole learning to interview better. When I refuse a particular exercise, or drop right out of the process, they don't blame themselves. They just move on to hazing the next candidate and hope for the best.
This would basically preclude anybody from interviewing at the FANGs. I believe they have all adopted the Google style leetcode college level trivia questions, completely divorced from the realities of the job. Their processes are so tightly controlled and the duties of the job so far removed from the interview process that I can't imagine what would happen even if you successfully navigated to a job offer.
"Oh congratulations, you made it! You won't see any of the people you interviewed with ever again. You'll be placed on 1 of 3 teams that do things we've never spoken to you about. I'm sure you are going to love it here!"
"Oh congratulations, you made it! You won't see any of the people you interviewed with ever again. You'll be placed on 1 of 3 teams that do things we've never spoken to you about. I'm sure you are going to love it here!"
I sympathize but I'm not sure this is the hill to die on. As a potential employee I'm looking for the best company to join for myself, and the likelihood that I will enjoy working somewhere has very little correlation to their recruiting/interviewing practices. So if I want to work somewhere I'll jump through their hoops (within reason), but I'll also be keeping an eye out for red flags from engineers and managers I talk to, because the downside of an annoying interview that is not to my taste pales in comparison to the downside of accepting a job and then discovering the work environment is a tire fire.
Too bad you didn't record your interviews. I bet if you bailed on a shitty interview from a high profile company, it'd go viral. Could be much more effective than pleading with people to stop accepting shitty interviews.
What is a "shitty"interview? I certainly hope you don't mean a whiteboard interview where you are asked to solve and code a small problem? Probably the best type of interview our industry can give
What is a "mimic the job"interview? How do you do the in 45 minutes? Using tools the interviewee may never have seen?
I think asking a few coding questions is the best interview.
I used to believe in this dogma, I thought I was "too good" to practice brain teasers. It set my career back by years. We don't live in a utopia. Tech jobs aren't a meritocracy. If you want to work on cool stuff and get paid proportionally then start your own company. Otherwise it's best if you accept that tech recruiting is really fucked up, and just play this shitty game.
[deleted]
I accept very few interviews at all, so sniffing out shitty ones ahead of time isn’t particularly intuitive for me. But I can say of the two worst ones, I very nearly walked out on one, and did walk out on the other. I regretted not walking on the first one and didn’t regret walking the next one. The first was a FAANG, the second … should be in the acronym.
The point of a leetcode style question is to test someones' ability to explain and adapt a tricky problem on the fly. Given most of a software engineer's day to day is about communication and adapting existing solutions to related problems, leetcode problems are a very powerful tool for analyzing candidates.
>> The point of a leetcode style question is to test someones' ability to explain and adapt a tricky problem on the fly.
This is just nonsense, since the only reason the developer can answer the question is because they crammed it in preparation for the interview. Leetcode questions are not tricky, they're just tricks.
This is just nonsense, since the only reason the developer can answer the question is because they crammed it in preparation for the interview. Leetcode questions are not tricky, they're just tricks.
While there is a lot of nuance to interviewing people, I can't say that solving leetcode problems in an interview setting is similar to any problem solving I've done on the job.
The closest might be white-boarding potential solutions in meetings, but there, most of the time, I have a lot of context wrt. the company and situation.
In the past I haven't minded presenting solutions to contrived problems in interviews, but these days I wouldn't. I also wouldn't take a job without significant equity attached, but I'm probably not representative of most devs.
The closest might be white-boarding potential solutions in meetings, but there, most of the time, I have a lot of context wrt. the company and situation.
In the past I haven't minded presenting solutions to contrived problems in interviews, but these days I wouldn't. I also wouldn't take a job without significant equity attached, but I'm probably not representative of most devs.
Better advice: Leetcode is easy as shit. Suck up your pride and be good at it. You’ll rake in money getting companies to fight over you. I had a $670k offer that’s appreciated to $1m/yr. The company I joined didn’t ask LC, but it still mentally prepared me for the questions I got.
You are absolutely right from your angle, but this is the perfect example of why the process is wrong. Nobody should be using something as trivial as leetcode to choose people they are willing to pay $1M.
How else do they decide? They’re doing a behavioral round, system design, manager interview, etc. But being able to implement, traverse, and analyze complex data structures and apply them to a problem is essential.
There’s a class of delusional people who think a GitHub profile is sufficient. Outside of a minority of technical influencers that is absolutely wrong and unreasonable.
There’s a class of delusional people who think a GitHub profile is sufficient. Outside of a minority of technical influencers that is absolutely wrong and unreasonable.
Bingo!
This comment must be satire, or peek hubris. Please tell me you are joking. I hope N-gate writes a bit about this comment, or about me for not clearly knowing if this is a joke or if it's real.
If you're not kidding, you are so delusional that I don't even know where to start.
If you are kidding than to did a damn good job!
If you're not kidding, you are so delusional that I don't even know where to start.
If you are kidding than to did a damn good job!
It's nice if you don't have to but I'm a mediocre engineer with no degree and no open source work. I have to take what I can get.
> One of the recurring themes is the "leet code" interviews
What makes these "shitty" interviews? They're very simple and they test for your ability to write code.
They set the bar clearly and you have the ability to study for them with ease. Solve 200 of these problems once in your life and you'll never have an issue again. You'll breeze through all of these interviews.
Compared to the alternative, where people ask insane unrelated questions and that are incredibly subjective, these aren't all that bad.
What makes these "shitty" interviews? They're very simple and they test for your ability to write code.
They set the bar clearly and you have the ability to study for them with ease. Solve 200 of these problems once in your life and you'll never have an issue again. You'll breeze through all of these interviews.
Compared to the alternative, where people ask insane unrelated questions and that are incredibly subjective, these aren't all that bad.
Because in OP's case, they indicate that the company either doesn't care that OP wrote software the company uses and thus has already demonstrated competence and then some, or they indicate that the company's hiring process is too inflexible to avoid insulting someone who they already know is beyond qualified.
Either one is a red flag, and a perfectly great reason to bail from an interview. If hiring/HR is rotten, the rest of the company is likely similar.
Either one is a red flag, and a perfectly great reason to bail from an interview. If hiring/HR is rotten, the rest of the company is likely similar.
> they test for your ability to write code.
Do crossword puzzles test my ability to write novels? No they don't, because remembering some obscure name for a medieval dignitarys hat of office doesn't mean I can write compelling immersive stories.
By the same logic, memorizing how some obscure algorithm is implemented doesn't test my ability to think about real world applications or code in the context of a large project.
Do crossword puzzles test my ability to write novels? No they don't, because remembering some obscure name for a medieval dignitarys hat of office doesn't mean I can write compelling immersive stories.
By the same logic, memorizing how some obscure algorithm is implemented doesn't test my ability to think about real world applications or code in the context of a large project.
There's a compromise in here that's worth looking into. Leet code has multiple difficulty levels for problems, and FAANG generally set their bar at the medium level. By contrast, I've had success asking senior candidates an "easy" question.
It's a small portion of the interview, the majority of which is asking them to talk about problems, technologies, strategies, etc., but the easy question (not fizzbuzz, but also not a DP question) makes sure they actually know how to code. My general issue with leet code interviews is that once I get nervous, my brain stops being able to come up with clever solutions to problems, and if a problem seems hard, this only compounds.
When you give someone an "easy" question, they loosen up and can actually start thinking creatively. You can see how they think it through, verify their solutions, and talk to you about requirements and approaches. As opposed to giving them a hard problem and watching the amygdala do its thing for an hour.
It's a small portion of the interview, the majority of which is asking them to talk about problems, technologies, strategies, etc., but the easy question (not fizzbuzz, but also not a DP question) makes sure they actually know how to code. My general issue with leet code interviews is that once I get nervous, my brain stops being able to come up with clever solutions to problems, and if a problem seems hard, this only compounds.
When you give someone an "easy" question, they loosen up and can actually start thinking creatively. You can see how they think it through, verify their solutions, and talk to you about requirements and approaches. As opposed to giving them a hard problem and watching the amygdala do its thing for an hour.
Thank you for your application to extend our web forum built with Next.js. We query APIs from X, Y, and Z to offer Foo to 500 users. I see you have three years experience with Next.js and eight years working with X, Y, and Z! Excellent. We only have one question before we hire you - can you implement a Trie from scratch in an hour on this whiteboard without the internet?
...
...
I guess I understand the intent but I think you're off base for 2 reasons.
1. The question you are posing isn't hard. It's easy to borderline trivial, I would think so even for junior engineers who are recent grads.
2. Being skilled in a specific FE framework or any library/API isn't what you should be testing for unless you're hiring for simple task work. You really want to uncover ability (IQ) not familiarity. You want people that will grow with the company, not drag you down.
1. The question you are posing isn't hard. It's easy to borderline trivial, I would think so even for junior engineers who are recent grads.
2. Being skilled in a specific FE framework or any library/API isn't what you should be testing for unless you're hiring for simple task work. You really want to uncover ability (IQ) not familiarity. You want people that will grow with the company, not drag you down.
> Being skilled in a specific FE framework or any library/API isn't what you should be testing for
Knowing how to implement some arcane algorithm is also not what you should be testing for. It tells me ZERO about this persons ability to code in the real world.
It may not even tell me they understand the algorithm...they may have just memorized the pseudocode implementation well enough.
> You really want to uncover ability (IQ) not familiarity.
a) Cognitive ability isn't accurately measureable. Not with IQ tests, and certainly not with standardized puzzle questions the internet is making lists of which one to learn to crack interview @ company X
b) Familiarity is EXTREMELY important when working with complicated systems.
Knowing how to implement some arcane algorithm is also not what you should be testing for. It tells me ZERO about this persons ability to code in the real world.
It may not even tell me they understand the algorithm...they may have just memorized the pseudocode implementation well enough.
> You really want to uncover ability (IQ) not familiarity.
a) Cognitive ability isn't accurately measureable. Not with IQ tests, and certainly not with standardized puzzle questions the internet is making lists of which one to learn to crack interview @ company X
b) Familiarity is EXTREMELY important when working with complicated systems.
a) sure and that's why you aren't literally administering an IQ test. it's highly subjective and flawed. so?
b) of course it is! no dispute whatsoever. that's why it's 6 months to become productive in any new job in the first place. But you want someone that can BECOME familiar and can UNDERSTAND the framework du jour.
When I hire an appliance repairman I want a guy that understands how the machine WORKS and can troubleshoot it from that knowledge base. Not someone that can swap in a fuse as long as the machine tells him which fuse is bad. I want someone who is knowledgeable with DISHWASHERS not someone that is an expert in MIELE seeing as I have a BOSCH or I've modified my miele beyond recognition anymore.
If you need seat warmers the story is different. Then sure, you just want a miele certified repairman.
b) of course it is! no dispute whatsoever. that's why it's 6 months to become productive in any new job in the first place. But you want someone that can BECOME familiar and can UNDERSTAND the framework du jour.
When I hire an appliance repairman I want a guy that understands how the machine WORKS and can troubleshoot it from that knowledge base. Not someone that can swap in a fuse as long as the machine tells him which fuse is bad. I want someone who is knowledgeable with DISHWASHERS not someone that is an expert in MIELE seeing as I have a BOSCH or I've modified my miele beyond recognition anymore.
If you need seat warmers the story is different. Then sure, you just want a miele certified repairman.
I think we are completely in agreement about what constitutes skill, and what companies should look for in candidates.
The point where I think we disagree, and please correct me if I'm wrong, is: are LC style questions an effective testing method for these skills.
I say no, they are not, for 2 reasons:
a) implementing something like a non-trivial graph algorithm doesn't come up often in day-2-day work, and if it comes up, even some experienced developers look it up, just to make sure they got it right from memory. To borrow your dishwasher-analogy: I want the repairmen to know how a generalized dishwasher works, so he can repair Miele, Bosch, whathaveyou, but I don't need him to be able to program the DWs microcontroller on a whiteboard, or forge one of the pipes with nothing but a firepit, a hammer, anvil, crucible and some copper-scrap.
b) since all these questions are known problems, both in their solution AND their usage as test questions, we are facing a "learning for the test" dilemma: people can, and many do, just memorize these answers to pass interviews, same as many students in standardized test environments just learn how to pass the tests.
This is fundamentally different from understanding the subject. Again, I agree with you: I want someone who can answer these questions because he can apply generalized knowledge about algos and data-structures.
But I am not confident tests which rely on pre-fabricated and well known puzzles will give me that.
The point where I think we disagree, and please correct me if I'm wrong, is: are LC style questions an effective testing method for these skills.
I say no, they are not, for 2 reasons:
a) implementing something like a non-trivial graph algorithm doesn't come up often in day-2-day work, and if it comes up, even some experienced developers look it up, just to make sure they got it right from memory. To borrow your dishwasher-analogy: I want the repairmen to know how a generalized dishwasher works, so he can repair Miele, Bosch, whathaveyou, but I don't need him to be able to program the DWs microcontroller on a whiteboard, or forge one of the pipes with nothing but a firepit, a hammer, anvil, crucible and some copper-scrap.
b) since all these questions are known problems, both in their solution AND their usage as test questions, we are facing a "learning for the test" dilemma: people can, and many do, just memorize these answers to pass interviews, same as many students in standardized test environments just learn how to pass the tests.
This is fundamentally different from understanding the subject. Again, I agree with you: I want someone who can answer these questions because he can apply generalized knowledge about algos and data-structures.
But I am not confident tests which rely on pre-fabricated and well known puzzles will give me that.
Everything is trivial once you know it. Good engineering principles like software architecture, for example being able to produce a sound Onion architecture with an accurate and well tested domain model, could be missed entirely by focusing on leetcode but is more important.
Edit: If you're looking for puzzle solving skills then give someone a puzzle
Edit: If you're looking for puzzle solving skills then give someone a puzzle
> They're very simple and they test for your ability to write code
Do they? Or do they test for your ability to memorize a set of common problems?
I don't have another solution, but just because they are the best we have come up with doesn't mean they aren't shitty.
Do they? Or do they test for your ability to memorize a set of common problems?
I don't have another solution, but just because they are the best we have come up with doesn't mean they aren't shitty.
> They're very simple and they test for your ability to write code.
Do you have objective evidence that this is an effective test of "the ability to write code", or is that just your belief?
Do you have objective evidence that this is an effective test of "the ability to write code", or is that just your belief?
> They're very simple and they test for your ability to write code.
This statement is an oxymoron.
This statement is an oxymoron.
"Past performance is no guarantee of future results."
I'll predicate this with: there are lots of shitty interviews, but I don't think it's because they ask you to reverse linked lists. It's because the interviewers don't know what questions they're trying to answer.
I'm coming up on 15 years of experience. I also feel like linked lists isn't really that relevant for my day-to-day. However, I still think coding interviews can make sense no matter the level.
If I only asked you about the past, I would miss out on at least three crucial pieces of information for the general full-time software engineer:
1) Are you still able to do the gritty work? Perhaps you wrote a great piece of FOSS ten years ago, but you completely lost it after that got going. Now you're managing PRs, but externally, it looks like you're still doing it yourself, because you're announcing the releases. This is useful even if I'm hiring you as a senior, and expecting you to mentor/guide those who are just starting out with the gritty work.
2) How* did you do the work? Was it luck, experience, genious? (Did you call a friend?) Can I give you a problem to solve, and feel confident you'll ask if you don't understand or think it's a bad idea? I.e. how do you react to the unknown?
3) Can you communicate with others around you about new topics? Are we speaking the same language? Can I feel confident you understand when I tell you a linked list is a bad solution. If I hire you as a domain expert, this is obviously less crucial. Again, the question is about the daily unknown, not the story you've had the chance to rehearse. (And if your job is simply to reiterate the same things over and over, I should probably just buy a book.)
You can use technical expertise in two ways: getting things done yourself, or building rapport with those who do. I have interviewed "seniors" who I wouldn't have wanted as mentors myself, because all they could do was talk. Lacking reasoning skills, or inclination to work together to get something solved.
That said, the focus should never be on the problem itself. It's ephemeral, as everyone keeps saying. There shouldn't be any gotchas or trick questions. The problem is just a way to get the conversation going. It must be small enough, and easily explained. When I only have 45 min at a Google interview, I don't have time to explain a problem I'm actually working on. There's too much domain knowledge required. Instead, I have to distill a problem I had in the past to something that's interesting, but simple. That's why we end up with reversing linked lists over and over again.
It shouldn't matter if the problem is fully solved or not in the end, as long as the method is sound, and we understand each other. It doesn't matter if we go on a tangent, as long as it's enlightening. Also, I'd add that I don't think programming interviews should be the only thing that's happening, especially not for people with experience. They just happen to be the most contentious, and come up a lot in discussions.
(As someone who helped build Spotify's tech recruitment pipeline, and having interviewed SRE/SWE candidates at Google.)
I'll predicate this with: there are lots of shitty interviews, but I don't think it's because they ask you to reverse linked lists. It's because the interviewers don't know what questions they're trying to answer.
I'm coming up on 15 years of experience. I also feel like linked lists isn't really that relevant for my day-to-day. However, I still think coding interviews can make sense no matter the level.
If I only asked you about the past, I would miss out on at least three crucial pieces of information for the general full-time software engineer:
1) Are you still able to do the gritty work? Perhaps you wrote a great piece of FOSS ten years ago, but you completely lost it after that got going. Now you're managing PRs, but externally, it looks like you're still doing it yourself, because you're announcing the releases. This is useful even if I'm hiring you as a senior, and expecting you to mentor/guide those who are just starting out with the gritty work.
2) How* did you do the work? Was it luck, experience, genious? (Did you call a friend?) Can I give you a problem to solve, and feel confident you'll ask if you don't understand or think it's a bad idea? I.e. how do you react to the unknown?
3) Can you communicate with others around you about new topics? Are we speaking the same language? Can I feel confident you understand when I tell you a linked list is a bad solution. If I hire you as a domain expert, this is obviously less crucial. Again, the question is about the daily unknown, not the story you've had the chance to rehearse. (And if your job is simply to reiterate the same things over and over, I should probably just buy a book.)
You can use technical expertise in two ways: getting things done yourself, or building rapport with those who do. I have interviewed "seniors" who I wouldn't have wanted as mentors myself, because all they could do was talk. Lacking reasoning skills, or inclination to work together to get something solved.
That said, the focus should never be on the problem itself. It's ephemeral, as everyone keeps saying. There shouldn't be any gotchas or trick questions. The problem is just a way to get the conversation going. It must be small enough, and easily explained. When I only have 45 min at a Google interview, I don't have time to explain a problem I'm actually working on. There's too much domain knowledge required. Instead, I have to distill a problem I had in the past to something that's interesting, but simple. That's why we end up with reversing linked lists over and over again.
It shouldn't matter if the problem is fully solved or not in the end, as long as the method is sound, and we understand each other. It doesn't matter if we go on a tangent, as long as it's enlightening. Also, I'd add that I don't think programming interviews should be the only thing that's happening, especially not for people with experience. They just happen to be the most contentious, and come up a lot in discussions.
(As someone who helped build Spotify's tech recruitment pipeline, and having interviewed SRE/SWE candidates at Google.)
The process appears to more like a hazing ritual. A form of tribalism.
What open source software did you write out if curiosity?
My most used in finance is a few functions in pandas, like merge_asof(). It's pretty industry standard now.
https://pandas.pydata.org/docs/reference/api/pandas.merge_as...
The biggest open-source thing I've ever written is Empirical, a language for time-series analysis.
https://www.empirical-soft.com
I'm also top 1% on Stack Overflow. A lot of companies have found me through there.
https://stackoverflow.com/users/478288/chrisaycock
https://pandas.pydata.org/docs/reference/api/pandas.merge_as...
The biggest open-source thing I've ever written is Empirical, a language for time-series analysis.
https://www.empirical-soft.com
I'm also top 1% on Stack Overflow. A lot of companies have found me through there.
https://stackoverflow.com/users/478288/chrisaycock
Why should I have to make a good resume when that doesn't mimic anything I'll be doing on the job?
I suspect the answer to this question is the same as to why people do the same for interview questions.
I suspect the answer to this question is the same as to why people do the same for interview questions.
I personally like these questions a lot and I've used them in the past when I was tasked with hiring people because it basically boils down to a few things. One is that people who have a good grasp on algorithms, math or academic problems generally are pretty smart, and you can teach them practical coding skills very quickly while the reverse isn't true. so it's sort of like a IQ test.
Secondly it rewards people who at least put the effort in to prepare for an interview. These questions are a very reliable way to eliminate candidates.
And to take the journalism example, I actually think that'd make a pretty solid test because if a journalist doesn't have a basic grasp on formal language I would have to reconsider how strongly they're committed to their craft. It's honestly only the case in software, an engineering discipline to boot, that people question the necessity of being able to solve basic theoretical problems. Would anyone trust a civil engineer who can't answer first semester math questions?
Secondly it rewards people who at least put the effort in to prepare for an interview. These questions are a very reliable way to eliminate candidates.
And to take the journalism example, I actually think that'd make a pretty solid test because if a journalist doesn't have a basic grasp on formal language I would have to reconsider how strongly they're committed to their craft. It's honestly only the case in software, an engineering discipline to boot, that people question the necessity of being able to solve basic theoretical problems. Would anyone trust a civil engineer who can't answer first semester math questions?
I guess it all boils down to who you are hiring (the OP is talking about hiring senior developers). It is ok to ask more technical questions to fresh graduates as they don't have any other experience anyway.
> it rewards people who at least put the effort in to prepare for an interview
Is this a good thing? Would you rather hire smart people, or people who studied a few leetcode questions the night before the interview?
> Would anyone trust a civil engineer who can't answer first semester math questions?
Civil engineers use that knowledge in everything they do. A really small subset of developers need to implement tree traversal algorithms from scratch on a daily basis.
> it rewards people who at least put the effort in to prepare for an interview
Is this a good thing? Would you rather hire smart people, or people who studied a few leetcode questions the night before the interview?
> Would anyone trust a civil engineer who can't answer first semester math questions?
Civil engineers use that knowledge in everything they do. A really small subset of developers need to implement tree traversal algorithms from scratch on a daily basis.
Take a step back and see how this leads you to miss out on plenty of candidates who could do the job, at the end of the day. These sorts of tests really just optimize for hiring people with time on their hands to study this stuff. Right off the bat having a heavy leetcode requirement probably excludes people with families or dependents, people with hobbies, or people who want to do anything after the bell that isn't staring at more code on an electronic screen.
I also think its a little backward to expect the candidate to be doing very much prep work in advance of the interview. They are in all likelyhood currently employed fulltime and busy with their life outside of that.
Instead I would focus on simply seeing if the candidate would be interested in the sort of work they would be doing, and giving as much detail as possible on what the job would be like for them. Job listings and corporate websites are often very opaque. It's one thing to get someone competent, but if they hate the job they will be gone in a year and you will be in the exact same place as you are today. If you get someone who might not know all the required tooling, but is very interested in the work being done, they will step up and be one of your most prolific employees before long and contribute to lasting institutional knowledge within your organization, rather than running off with their skills as soon as someone dangles a better paycheck in front of them.
I also think its a little backward to expect the candidate to be doing very much prep work in advance of the interview. They are in all likelyhood currently employed fulltime and busy with their life outside of that.
Instead I would focus on simply seeing if the candidate would be interested in the sort of work they would be doing, and giving as much detail as possible on what the job would be like for them. Job listings and corporate websites are often very opaque. It's one thing to get someone competent, but if they hate the job they will be gone in a year and you will be in the exact same place as you are today. If you get someone who might not know all the required tooling, but is very interested in the work being done, they will step up and be one of your most prolific employees before long and contribute to lasting institutional knowledge within your organization, rather than running off with their skills as soon as someone dangles a better paycheck in front of them.
> One is that people who have a good grasp on algorithms, math or academic problems
And can you guarantee that this is what LC questions test for?
We are talking about questions here were the answers are known, their usage for interviews is known, and curated lists of questions commonly used for interviews exist .
So yeah, they may test for a good grasp of algorithms, data structures and math.
But they can also just be memorized and replayed at will given enough time.
And can you guarantee that this is what LC questions test for?
We are talking about questions here were the answers are known, their usage for interviews is known, and curated lists of questions commonly used for interviews exist .
So yeah, they may test for a good grasp of algorithms, data structures and math.
But they can also just be memorized and replayed at will given enough time.
BTW, IQ tests are, even if occasionally legal (seems they aren't necessarily illegal, but sometimes illegal in the USA depending on circumstances), are pretty unethical given that IQ is poorly correlated with what you're trying to measure (G), but it you don't trust me, trust Nasim Taleb:
https://medium.com/incerto/iq-is-largely-a-pseudoscientific-...
You should not be trying to approximate an IQ test (and leetcode does exactly this).
You should not be trying to approximate an IQ test (and leetcode does exactly this).
Asking people to stop accepting interviews because they're bad (which I agree they are) is a bit like asking people to stop using combustion-based car because it is bad for the environment. On an individual level, sure -- a few people might flat out decline such interviews as a matter of principle. But as a whole this kind of appeal hardly has any impact.
I rue the day that leetcode-style interviews became the norm, but I don't think they're going away any time soon. The incentives are just too strong. Leetcode style quizzes are a (yes, flawed, biased etc) somewhat reasonable proxy to coding skills, they cost very little to implement, scale easily etc. Personally I prefer take-home half-day projects, but its easy to see how absurdly impractical those are for larger companies.
I rue the day that leetcode-style interviews became the norm, but I don't think they're going away any time soon. The incentives are just too strong. Leetcode style quizzes are a (yes, flawed, biased etc) somewhat reasonable proxy to coding skills, they cost very little to implement, scale easily etc. Personally I prefer take-home half-day projects, but its easy to see how absurdly impractical those are for larger companies.
https://news.ycombinator.com/item?id=29892437
One of the recurring themes is the "leet code" interviews. I will go further and say that any interview that doesn't mimic the job is a guaranteed way for me to turn the company down.
Stop accepting shitty interviews
When I was talking to trading firms about coming back to the finance industry, about half of them admitted they already use my open-source software. And yet, only one firm bothered to ask me about how some of my stuff was implemented. That's the firm I'm working at now.
Instead, I got plenty of firms that still insisted on asking me questions that are not remotely related to the job. Tons of Hacker Rank and Codility, even though I've never had a time limit when building something in real life. Tons of intern-level questions as if I was a college freshman. And tons of detailed questions about how a specific programming language works, as if we're going to build a compiler for that language.
And to reiterate, those were the firms that already admitted to using my software!
Stop accepting shitty interviews
The stupid assessment practices won't change unless employers suffer the consequences for this nonsense. I have always made it my mission to reject any firm that asks me syntax questions or brain teasers. (Seriously, the New York Times doesn't assess journalists by having them conjugate verbs; it's just weird to ask this.)
Yes, people get mad when I terminate the interview on the spot. I'm sure I've been blacklisted from a few places because of my stubbornness. But damn if I'm going to allow stupid shit like this.
Stop accepting shitty interviews