Why coding interviews aren't all that bad(eli.thegreenplace.net)
eli.thegreenplace.net
Why coding interviews aren't all that bad
https://eli.thegreenplace.net/2022/why-coding-interviews-arent-all-that-bad/
53 comments
> Testing for simple algorithms knowledge biases for recently graduated people who have these in their active memory.
I have never understood this argument. With 29 years of experience, I am about as far from being a "recently graduated person" as anyone could be - in fact, I never graduated at all! Not only did I drop out after three semesters, I never took so much as one computer science class.
I have, according to your assumptions, never had algorithms knowledge in my active memory - and yet, these sorts of questions have never been an obstacle for me. How can this be?
I think many people must mistakenly believe that interview questions are the same sort of questions one might find on a university test. They appear to believe that the interviewer expects them to regurgitate a correct answer, on the spot, from memory, and that their suitability for the job is rated based on the speed of recall.
That would be absurd, of course. If anyone is actually interviewing people that way, they're doing a terrible job.
I have never answered these questions that way - and, when I ask such questions in an interview, I never expect them to be answered that way.
Rather, the point of an algorithms problem is that you have to think your way through it. I solve them in the same way I would solve any problem I encountered in my day-to-day programming work: by thinking about it, poking around at different approaches, roughing out a simplest-possible approach, then refining toward a more efficient process.
If I am interviewing you, and I pose you an algorithms problem, I'm hoping you will not have memorized an answer. If you already know the answer, your skill at writing it out on the whiteboard tells me nothing. The whole point of asking you to think your way through an algorithms problem is that I want to see how you think. I don't even care whether you find your way through to a correct answer in the time allotted, so long as I can see that you know how to think about a problem, work your way toward a solution, and communicate about your process as you go.
The common idea that experienced devs ought to struggle with algorithms questions dismays me. What are people doing that does not require them to maintain basic problem-solving skills? Or... perhaps... is this all a big misunderstanding...
I have never understood this argument. With 29 years of experience, I am about as far from being a "recently graduated person" as anyone could be - in fact, I never graduated at all! Not only did I drop out after three semesters, I never took so much as one computer science class.
I have, according to your assumptions, never had algorithms knowledge in my active memory - and yet, these sorts of questions have never been an obstacle for me. How can this be?
I think many people must mistakenly believe that interview questions are the same sort of questions one might find on a university test. They appear to believe that the interviewer expects them to regurgitate a correct answer, on the spot, from memory, and that their suitability for the job is rated based on the speed of recall.
That would be absurd, of course. If anyone is actually interviewing people that way, they're doing a terrible job.
I have never answered these questions that way - and, when I ask such questions in an interview, I never expect them to be answered that way.
Rather, the point of an algorithms problem is that you have to think your way through it. I solve them in the same way I would solve any problem I encountered in my day-to-day programming work: by thinking about it, poking around at different approaches, roughing out a simplest-possible approach, then refining toward a more efficient process.
If I am interviewing you, and I pose you an algorithms problem, I'm hoping you will not have memorized an answer. If you already know the answer, your skill at writing it out on the whiteboard tells me nothing. The whole point of asking you to think your way through an algorithms problem is that I want to see how you think. I don't even care whether you find your way through to a correct answer in the time allotted, so long as I can see that you know how to think about a problem, work your way toward a solution, and communicate about your process as you go.
The common idea that experienced devs ought to struggle with algorithms questions dismays me. What are people doing that does not require them to maintain basic problem-solving skills? Or... perhaps... is this all a big misunderstanding...
> That, of course, would be absurd. If anyone is actually interviewing people that way, they're doing a terrible job.
Perhaps you have never experienced that sort of interview, but it doesn't mean it doesn't exist.
I was once asked a question where the solution involved implementing a trie. The naive design involves a 26-element array to hold all the possible next-letter entries, which obviously takes a huge amount of memory. The interviewer expected me to know the trivia answer on how to avoid allocating all that memory, and was clearly disappointed that I didn't know (using a linked list instead of a fixed-size array: https://en.wikipedia.org/wiki/Trie#Implementation_strategies). This was a principal engineer interviewing another principal engineer, FWIW. The company was Qualtrics, a survey/customer satisfaction company, not a bare-metal algorithms-heavy company.
Obviously you've managed to have a successful career without encountering interview questions that hinge on algorithmic trivia, and I'm not doubting your experience. But saying "it's never happened to me so therefore it can't exist" isn't an especially strong argument.
Perhaps you have never experienced that sort of interview, but it doesn't mean it doesn't exist.
I was once asked a question where the solution involved implementing a trie. The naive design involves a 26-element array to hold all the possible next-letter entries, which obviously takes a huge amount of memory. The interviewer expected me to know the trivia answer on how to avoid allocating all that memory, and was clearly disappointed that I didn't know (using a linked list instead of a fixed-size array: https://en.wikipedia.org/wiki/Trie#Implementation_strategies). This was a principal engineer interviewing another principal engineer, FWIW. The company was Qualtrics, a survey/customer satisfaction company, not a bare-metal algorithms-heavy company.
Obviously you've managed to have a successful career without encountering interview questions that hinge on algorithmic trivia, and I'm not doubting your experience. But saying "it's never happened to me so therefore it can't exist" isn't an especially strong argument.
I’ve had this happen to me too. When I had only 2 years of experience, I was asked to code parallel dijkstra's in an interview. That startup shut its doors only 2 years later - guessing a hiring issue.
But at the same time, saying “it happened to me once therefore it’s generally the case” isn’t a strong argument.
I agree with the GP. I’m also a dropout, but am generally pretty good at navigating FANG-style questions, which are leetcode medium at worst. Often it feels like they’re ultimately testing for depth of understanding, flexibility, and creativity when coming up with an answer.
It’s possible to brute-force your way through questions by rote memorization, but that’s going about it in the wrong spirit. And it feels like that's the gap when this topic comes up.
Interviewing is a skill to a certain extent, with maybe a few days/interviews of warmups. But if you believe that your ability to land jobs is directly correlated to regurgitating leetcode hard questions, the outcome is almost predestined.
If I had to sum up from personal experience:
- Not every good leetcoder I know is a top engineer.
- Every top engineer I know is a good (but not necessarily great!) leetcoder.
But at the same time, saying “it happened to me once therefore it’s generally the case” isn’t a strong argument.
I agree with the GP. I’m also a dropout, but am generally pretty good at navigating FANG-style questions, which are leetcode medium at worst. Often it feels like they’re ultimately testing for depth of understanding, flexibility, and creativity when coming up with an answer.
It’s possible to brute-force your way through questions by rote memorization, but that’s going about it in the wrong spirit. And it feels like that's the gap when this topic comes up.
Interviewing is a skill to a certain extent, with maybe a few days/interviews of warmups. But if you believe that your ability to land jobs is directly correlated to regurgitating leetcode hard questions, the outcome is almost predestined.
If I had to sum up from personal experience:
- Not every good leetcoder I know is a top engineer.
- Every top engineer I know is a good (but not necessarily great!) leetcoder.
That’s how it should work but it seems to have largely gravitated toward, not rewarding memorization exactly, but definitely pattern recognition gained from drilling on a wide arrangement of leetcode style problems, and as much as they say they want to understand your thought process your performance is heavily weighted towards whether or not you arrive at the optimal solution in the time given, not how well you might explain how you might get there even if your code is ultimately lacking due to time constraints.
How often do you interview and how often are you doing LC interviews?
There’s no real time to “see how one thinks” when the expectation is to pump out two medium answers in <45 minutes. You’re basically just reading the question, going through a couple hoops, and then implementing/explaining the algorithm. There’s barely any time to “see how one thinks” in these modern interviews.
Maybe you can show three problems you were asked recently and how long you had to solve them.
There’s no real time to “see how one thinks” when the expectation is to pump out two medium answers in <45 minutes. You’re basically just reading the question, going through a couple hoops, and then implementing/explaining the algorithm. There’s barely any time to “see how one thinks” in these modern interviews.
Maybe you can show three problems you were asked recently and how long you had to solve them.
I have never encountered what I would consider to be a "leetcode interview"; that is the essence of my bafflement when people talk as though it is simply The Way Tech Interviews Are Done.
Perhaps I have simply been lucky, but my luck has held for a very long time, and that makes me want a better explanation.
When I hear people describe these "leetcode interviews", they often describe a process which sounds very much like being given a normal interview problem, except they assume, for reasons which have never been clear to me, that they must have a memorized answer at hand in order to succeed.
From my non-college-trained perspective, this seems like the sort of misunderstanding one might easily make if one attempted to apply years of practice with school exams to a new environment. Perhaps I have encountered "leetcode interviews", and simply didn't experience them the same way?
Or perhaps the "leetcode thing" genuinely is a different style of interview, which has grown up in a sector of the industry I don't care for. That might explain why it seems ubiquitous to some and nonexistent to others...
How often do I interview - ? Well, whenever I might want a new job, I suppose. I'm not sure what you mean; you seem to be implying that people interview recreationally? Last time would have been three-odd years ago, and I certainly did not bother to write down the problems which would have been involved.
Perhaps I have simply been lucky, but my luck has held for a very long time, and that makes me want a better explanation.
When I hear people describe these "leetcode interviews", they often describe a process which sounds very much like being given a normal interview problem, except they assume, for reasons which have never been clear to me, that they must have a memorized answer at hand in order to succeed.
From my non-college-trained perspective, this seems like the sort of misunderstanding one might easily make if one attempted to apply years of practice with school exams to a new environment. Perhaps I have encountered "leetcode interviews", and simply didn't experience them the same way?
Or perhaps the "leetcode thing" genuinely is a different style of interview, which has grown up in a sector of the industry I don't care for. That might explain why it seems ubiquitous to some and nonexistent to others...
How often do I interview - ? Well, whenever I might want a new job, I suppose. I'm not sure what you mean; you seem to be implying that people interview recreationally? Last time would have been three-odd years ago, and I certainly did not bother to write down the problems which would have been involved.
Maybe you just haven’t interviewed for tech companies in Silicon Valley/the Bay Area or in Seattle before?
That would be a convenient explanation! - but I began my career in CA and moved to Seattle in '99. I've worked for a handful of startups, a few mid-size companies, and three of the megas.
If they’re asking you to explain the time and/or space complexity of your solution in terms of “Big O”, which is something you’d learn in an algorithms class, that’s a good clue. The most recent interview I had asked me to determine the depth of a given binary tree which is about as textbook algorithms class as you can get. Others were a bit more nebulous in terms of solutions but they all span various algorithmic topics you learn in school.
Big O is a useful vocabulary for comparing efficiency between different approaches. Binary trees are abstract, yes, but I prefer them to spending an extra 5-10 minutes listening to an interviewer describe a scenario that ultimately still boils down to a binary tree.
From the other side of the table, when I’ve interviewed candidates without a solid understanding of runtimes I’ve seen some very unique code. One candidate wrote a triple-nested for loop and couldn’t understand why it wasn’t an answer we were hoping for. i.e. O(n^3) instead of O(nlgn). Running that code in production honestly would have been a disaster for our oncall.
Interviews being such a hot button topic always struck me as odd. On HN we love evaluating new frameworks of the month, or discussing the intricacies of Elixir fundamentals - all of which also require time investment to learn. Why draw the line at learning Big O? Try looking at it through the lens of a useful skill rather than something to memorize for interviews. Even if you never learned it in school, you can still learn it now and it has real life practical value.
From the other side of the table, when I’ve interviewed candidates without a solid understanding of runtimes I’ve seen some very unique code. One candidate wrote a triple-nested for loop and couldn’t understand why it wasn’t an answer we were hoping for. i.e. O(n^3) instead of O(nlgn). Running that code in production honestly would have been a disaster for our oncall.
Interviews being such a hot button topic always struck me as odd. On HN we love evaluating new frameworks of the month, or discussing the intricacies of Elixir fundamentals - all of which also require time investment to learn. Why draw the line at learning Big O? Try looking at it through the lens of a useful skill rather than something to memorize for interviews. Even if you never learned it in school, you can still learn it now and it has real life practical value.
Isn’t binary tree depth just simple recursion? Pretty straight forward from programming first principles without memorizing algorithms.
> Technological fields are blue oceans where only junior people spend significant time near the safe harbors. Experienced people have sailed far away and seen lands you don't know about yet.
I really like this analogy.
I really like this analogy.
Yes, exactly-this is what I wanted to get across in my comment down post.
The discourse around algorithms/data structures coding interviews (as opposed to domain-relevant pair programming coding challenges) does seem to be universally either about abolishing them or keeping the status quo.
Then what about keeping those topics but improving the process? How about reducing other pain points? Such as:
* Transparent rubrics about how an applicant is to be judged
* Disclosure of what specific topics are to be tested or not, e.g. avoiding candidates stressing over dynamic programming problems only to be stumped by simple string manipulation
* Actual training of interviewers so that they can be more collaborative and helpful during the interview
* Disclosing to candidates just how many questions they are expected to answer during a round
Companies want to keep their Leetcode question banks, fine. But the coding interview process are painful and scary to candidates not only because of any difficulty in problem content, but because of the apparent subjectivity and opaqueness of everything involved. Maybe the current process can be reformed and systemized a bit so neither party wastes their time.
And another subject worth considering: how could current algorithmic pedagogy be improved? How do you take the teaching of runtime costs and complexity, specific data structures, and general principles like dynamic programming, and be able to solve specific problems like, say, merge intervals, without having seen such problems previously? Maybe there’s a layer of “now that you have a toolset, this is a good set of heuristics for matching your methods to actual problems.”
Then what about keeping those topics but improving the process? How about reducing other pain points? Such as:
* Transparent rubrics about how an applicant is to be judged
* Disclosure of what specific topics are to be tested or not, e.g. avoiding candidates stressing over dynamic programming problems only to be stumped by simple string manipulation
* Actual training of interviewers so that they can be more collaborative and helpful during the interview
* Disclosing to candidates just how many questions they are expected to answer during a round
Companies want to keep their Leetcode question banks, fine. But the coding interview process are painful and scary to candidates not only because of any difficulty in problem content, but because of the apparent subjectivity and opaqueness of everything involved. Maybe the current process can be reformed and systemized a bit so neither party wastes their time.
And another subject worth considering: how could current algorithmic pedagogy be improved? How do you take the teaching of runtime costs and complexity, specific data structures, and general principles like dynamic programming, and be able to solve specific problems like, say, merge intervals, without having seen such problems previously? Maybe there’s a layer of “now that you have a toolset, this is a good set of heuristics for matching your methods to actual problems.”
You have to treat it like a creative field, albeit one which has logical rigors to it. I believe the correct way of hiring in this structure is to hire people for the Apprentice/Journeyman/Master artisan categories used in medieval times.
There are clearly master coders out there e.g. Linus Torvalds, John Carmack, etc.
I really strongly feel having the master/apprentice structure is the correct way to hire and consider coders, and that many things business wants to do and hires “new programmers” for needs to be considered in terms of how many people you want to hire who can complete the task on their own, and how much help they will need which can be provided in terms of junior/senior positions below the artisan/architecture level programmer.
There are clearly master coders out there e.g. Linus Torvalds, John Carmack, etc.
I really strongly feel having the master/apprentice structure is the correct way to hire and consider coders, and that many things business wants to do and hires “new programmers” for needs to be considered in terms of how many people you want to hire who can complete the task on their own, and how much help they will need which can be provided in terms of junior/senior positions below the artisan/architecture level programmer.
The core interview of this type is still fairly terrible IMO. You're just masking it with a few nicities as many big companies already do. (Those rich enough to have proper interviewer training)
Those big company interviews are still completely opaque, fraught with uncertainty.
What the article says makes sense but he's not disproving doing a coding test similar to what you may encounter on the job vs algorithms you'll implement once in a lifetime.
When I'm interviewing, I ask about their projects to see if they're passionate about something and then I test them on something I need to implement for work (or similar enough). It's often something like, implement an API endpoint that does X with Y database, implement this UI component. I'm happy to be flexible on language and technologies the candidate is familiar with.
If you think the job will be much more than that, you're simply wrong.
The closest I got to implement a leetcode exercise at work was when I had to implement pathfinding for a 3d game in a language that is not commonly used for games. Turns out my naive solution was poor, I duckduckgoed 5 minutes and implemented a way better algorithm from Wikipedia.
Besides: If I need to prepare for two months for your interview, I won't apply for your company. Even doing a take home exercise takes way less time than that, I can crank most of those in a couple of hours. Still, I'd prefer a real world exercise at the interview, so I don't need to prepare anything in advance.
When I'm interviewing, I ask about their projects to see if they're passionate about something and then I test them on something I need to implement for work (or similar enough). It's often something like, implement an API endpoint that does X with Y database, implement this UI component. I'm happy to be flexible on language and technologies the candidate is familiar with.
If you think the job will be much more than that, you're simply wrong.
The closest I got to implement a leetcode exercise at work was when I had to implement pathfinding for a 3d game in a language that is not commonly used for games. Turns out my naive solution was poor, I duckduckgoed 5 minutes and implemented a way better algorithm from Wikipedia.
Besides: If I need to prepare for two months for your interview, I won't apply for your company. Even doing a take home exercise takes way less time than that, I can crank most of those in a couple of hours. Still, I'd prefer a real world exercise at the interview, so I don't need to prepare anything in advance.
The questions are designed to allow an interviewer to get as much signal in the 45 minutes spent in the interview.. that's why they're contrived rather than your typical piece of work.
They're not designed to test you on something you would never encounter though.
If a company is asking these kinds of questions and the job only ever requires dealing with npm packages that shields the engineer from this complexity then they shouldn't be doing that in my opinion.
But most of the companies asking these questions are in the business of building software that requires this lower-level knowledge
I can appreciate the author’s reservations around take-home assignments given how easy it is to cheat, but why not as an interviewer simply protect against that by asking the interviewee to detail certain aspects of their implementation on a follow-up call?
Whenever I submit a project for review, I’m prepared to answer questions about why I did X, why I used this tool/library/approach versus some other tool/library/approach, and what I could do to improve the implementation further. On the latter subject, I’ll raise that discussion even before being asked. If your applicant panics on questions like this — or is obviously clearly making shit up on the spot — there’s your indicator to reject.
If you ask me to implement a B-tree, I’ll describe what a B-tree is and tell you I don’t off-hand remember how to do it or what an ideal approach would be in a given language. I’d tell you that I’d research it a bit before diving face-first into it. Which is what almost everyone would do in real-world contexts anyway.
Whenever I submit a project for review, I’m prepared to answer questions about why I did X, why I used this tool/library/approach versus some other tool/library/approach, and what I could do to improve the implementation further. On the latter subject, I’ll raise that discussion even before being asked. If your applicant panics on questions like this — or is obviously clearly making shit up on the spot — there’s your indicator to reject.
If you ask me to implement a B-tree, I’ll describe what a B-tree is and tell you I don’t off-hand remember how to do it or what an ideal approach would be in a given language. I’d tell you that I’d research it a bit before diving face-first into it. Which is what almost everyone would do in real-world contexts anyway.
Ah, to me that sounds like you are a good engineer and what they actually are seeking is a good “follower/code-servant”.
You see it’s good to filter applicants like you out because you may do nasty things like “question management decisions” and “think creatively”.
We want obedient slaves with social media accounts, Damn it!
You see it’s good to filter applicants like you out because you may do nasty things like “question management decisions” and “think creatively”.
We want obedient slaves with social media accounts, Damn it!
Yes, this is one of the things that arduous processes filter on.
bzzzt Wrong! No, you’re filtering for the tolerance of stupidity. When you hire idiots who are complacent to your stupidity then you are guaranteed a progressively more stupid environment.
That coding interviews are objective is no argument in their favor. The author provides no evidence that what's being "objectively" evaluated matters at all. They could just as soon choose the candidate whose name comes first in the alphabet or who's the youngest. Those are also objective, but not meaningful hiring criteria.
My only real problem with coding interviews is that you never know whether or not people lack knowledge, or if they're just folding under pressure. I've seen very competent people start to fumble, when they're being put on the spot - which in turn makes a bad first impression, for the people that have just met them.
Some people are very good at whiteboard interviews, others at take-home exams.
Some people are very good at whiteboard interviews, others at take-home exams.
If the folding under pressure is episodic/random, then those very competent people will likely make a correctly good impression in the next 1 or 2 interviews they attend and end up landing a job soon. From their perspective, it's still fine.
From a company perspective, if you really can't tell whether they lack competence or just folded under pressure, you pretty much have to pass on them and try again with the next candidate.
From a company perspective, if you really can't tell whether they lack competence or just folded under pressure, you pretty much have to pass on them and try again with the next candidate.
Why would it be random? I have seen people describe this as related to performance anxiety, and maybe some part of it is that.
I have myself come to conclusion that this is largely due to brain architecture. Notably the same part of the brain is responsible for coding as for speaking. Asking people to explain their thinking to an unknown person with unknown criteria takes a huge amount of brain capacity exactly from the parts of the brain one needs to program.
Additionally, the normal way experienced programmers approach these types of problems is to utilize background processing in the brain. They first encounter the problem, then they aim to understand it well. Then they go take a coffee, or even sleep on it, while the brain is working in the background unconsciously, and then at some point an innovation comes almost unbidden. The coding interview set up doesn't measure this real problem solving capability in any way. This is why experienced people become stumped, because it is not the way they actually solve the problems in the real life, and they might not even be aware of why their brain seems to not work suddenly.
This might even flare up their impostor syndrome which is a common professional hazard if they think they have failed in something they should have succeeded in. It is not only a bad way to interview and screen people, it is discriminatory and materially abusive.
I have myself come to conclusion that this is largely due to brain architecture. Notably the same part of the brain is responsible for coding as for speaking. Asking people to explain their thinking to an unknown person with unknown criteria takes a huge amount of brain capacity exactly from the parts of the brain one needs to program.
Additionally, the normal way experienced programmers approach these types of problems is to utilize background processing in the brain. They first encounter the problem, then they aim to understand it well. Then they go take a coffee, or even sleep on it, while the brain is working in the background unconsciously, and then at some point an innovation comes almost unbidden. The coding interview set up doesn't measure this real problem solving capability in any way. This is why experienced people become stumped, because it is not the way they actually solve the problems in the real life, and they might not even be aware of why their brain seems to not work suddenly.
This might even flare up their impostor syndrome which is a common professional hazard if they think they have failed in something they should have succeeded in. It is not only a bad way to interview and screen people, it is discriminatory and materially abusive.
How do they handle pager duty?
If you're asking how people solve problems with systems they know from head to toe, with which they have worked for ages and they know common issue types, and procedures how to diagnose and debug by endless repetition, you're right, they don't need to sleep on it, they just do it by muscle memory.
That's not unlike math olympiad competitors or competitive coders who by repetition generate a skill for those types of problems exactly.
That's not unlike math olympiad competitors or competitive coders who by repetition generate a skill for those types of problems exactly.
I used "if" in the sense of "if (i < 95)..." It's not arguing that i is always less than 95, just saying what the consequent is in the event that it is.
For many people, it is episodic/random; performance is variable by day, hour, and situation.
For many people, it is episodic/random; performance is variable by day, hour, and situation.
I personally implemented major APIs and libraries that are currently being used in deployment by over 25m+ systems with 99.999% uptime. When I get called into interviews and some guy starts trying to get me to implement or work on some random crap in some random IDE and doesn’t want me to use a search engine to do so?
Guess what, I just turn off the laptop and go hiking.
For a year. And then after that year I’ll try again. You’re not playing the game you think you are.
Guess what, I just turn off the laptop and go hiking.
For a year. And then after that year I’ll try again. You’re not playing the game you think you are.
Hot take: Having false negatives is perfectly fine as long as you’re minimizing false positives.
That is a bullcrap phrase all interviewers and HR say.
Look, how would you know your interview process isn't filtering critical numbers of good engineers?
You have no control group for that assertion. None.
And engineers are known to be neurodivergent more than your usual meatsack.
For example, your assertion works really well if the subconscious goal is to only hire white people. After all, black people sue for discrimination to better to be safe and avoid them.
Look, how would you know your interview process isn't filtering critical numbers of good engineers?
You have no control group for that assertion. None.
And engineers are known to be neurodivergent more than your usual meatsack.
For example, your assertion works really well if the subconscious goal is to only hire white people. After all, black people sue for discrimination to better to be safe and avoid them.
What’s missing from all these speculations about the best way to hire is any tracking of the applicants performance after the hire. If you showed me data that said that the highest percentage of candidates that do well at the company all did well on their algorithm interview vs any other screening technique then I would be more convinced. Google claims this is the case and the industry seems to have just followed suit without questioning it. I personally think the best way is probably a short simple fizzbuzz type problem to screen out people who simply can’t code at all then a take-home or in-house coding assignment with a discussion after or (or during if in-house). If the claim that avoiding a bad hire is far better than rejecting a good candidate is to be believed then it behooves the hirer to get as much signal around something that closely resembles the actual works as possible and actually discussing the code at hand would probably eliminate most problems around cheating.
> behooves the hirer to get as much signal around something that closely resembles the actual works as possible
Is there any reason why closely simulating the work environment should be the most efficient way to glean whether a candidate will perform well once hired?
Is there any reason why closely simulating the work environment should be the most efficient way to glean whether a candidate will perform well once hired?
I wouldn’t think it would be the most efficient but if you’re worried more about accepting a bad hire than rejecting a good one efficiency isn’t the primary metric.
Kind of. Long, convoluted interview processes are expensive in the opportunity cost of not having the role filled and in the actual work that needs to happen to conduct the interviews. That gets weight against the (presumed large) cost of a bad hire, but hiring is expensive, and practices that slow it down across the board to reduce a few bad hires might not be worth it.
I've personally experienced three kinds of coding interviews.
1. Trivial questions that can be done very quickly; e.g. does this list of integers contain any duplicate integers?
2. Non-trivial questions that require a large amount of time to solve if you haven't memorized the solution; e.g. the skyline problem https://leetcode.com/problems/the-skyline-problem/
3. Practical questions; e.g. a question that more-or-less represents something you might actually encounter in your day to day job, for instance querying multiple APIs and joining the data together into a particular expected format
---
1 isn't bad due to its level of ease. 2 is awful since you mostly just need to grind many hours of Leetcode to be able to consistently solve those kinds of problems. 3 is enjoyable since the problems given might actually teach you something you'll use day-to-day.
1. Trivial questions that can be done very quickly; e.g. does this list of integers contain any duplicate integers?
2. Non-trivial questions that require a large amount of time to solve if you haven't memorized the solution; e.g. the skyline problem https://leetcode.com/problems/the-skyline-problem/
3. Practical questions; e.g. a question that more-or-less represents something you might actually encounter in your day to day job, for instance querying multiple APIs and joining the data together into a particular expected format
---
1 isn't bad due to its level of ease. 2 is awful since you mostly just need to grind many hours of Leetcode to be able to consistently solve those kinds of problems. 3 is enjoyable since the problems given might actually teach you something you'll use day-to-day.
As an interviewee, I like coding interviews for another reason: they are so straightforward to prepare for. I have no idea what they say about my abilities as a SWE, but 4 coding interviews loops beat open ended loops where I don’t know what to expect. Yes, they have made intense practice of coding problems and prepping mandatory, but it isn’t a mystery.
But I don’t think they yield quality hires much more than the other ways, which all seem to be crapshoots anyways. Credentials and referrals seem to still give the best results, but they are necessarily biased to those who network.
But I don’t think they yield quality hires much more than the other ways, which all seem to be crapshoots anyways. Credentials and referrals seem to still give the best results, but they are necessarily biased to those who network.
The tree mirror problem is funny. You don't even need to recurse. Just define a wrapper for the node object that gets left when right is accessed and right when left is accessed. Voila!
Right and left are arbitrary anyway.
Right and left are arbitrary anyway.
Actually in our problem we wanted you to do it without a wrapper so while this works, 0/100
Can I rate your question? Why am I asking permission. 0/100
I'm going to be honest, I'm a backend developer and I've never written a tree in production. I understand the structure, I've written them for practice, but for real world code it's just never come up.
It seems that for the overwhelming majority of developer roles these data structures are outsourced to a database, which is usually closer to the data than my service anyway.
It's not that this knowledge is irrelevant, but there are so many aspects of writing, and maintaining, business logic with a team of others that are skipped entirely.
It seems that for the overwhelming majority of developer roles these data structures are outsourced to a database, which is usually closer to the data than my service anyway.
It's not that this knowledge is irrelevant, but there are so many aspects of writing, and maintaining, business logic with a team of others that are skipped entirely.
The author asserts that simple coding interviews
> involving nothing more than simple and fundamental data structures
is the objective way to evaluate a candidate. Because those
> do require a good understanding of programming fundamentals
But in my experience, someone who can memorize those basics can also memorize their implementations. So it is quite possible to have someone code a perfect BFS, but have no grasp of programming fundamentals.
> involving nothing more than simple and fundamental data structures
is the objective way to evaluate a candidate. Because those
> do require a good understanding of programming fundamentals
But in my experience, someone who can memorize those basics can also memorize their implementations. So it is quite possible to have someone code a perfect BFS, but have no grasp of programming fundamentals.
Honestly, they are really great points he makes.
There absolutely are flaws with interviews. But if you stick with coding questions using simple datastructures non-esoteric data structures, they are the most objective, scalable and accurate method of evaluating a candidate.
There absolutely are flaws with interviews. But if you stick with coding questions using simple datastructures non-esoteric data structures, they are the most objective, scalable and accurate method of evaluating a candidate.
Except that skews the candidates that pass younger. If you’re fresh out of school, you remember binary trees or sorting algorithms.
After doing software development for 20+ years, you’ve been using those structures as a small part of the larger problems you now solve. If you go looking for a new job, you don’t necessarily remember all these tricks to invert a binary tree, and you fail. Even though you have all that experience in what you actually will be asked to do on the job.
After doing software development for 20+ years, you’ve been using those structures as a small part of the larger problems you now solve. If you go looking for a new job, you don’t necessarily remember all these tricks to invert a binary tree, and you fail. Even though you have all that experience in what you actually will be asked to do on the job.
I’m a fairly new interviewer at FAANG. I wouldn’t consider “invert a binary tree” to be a good question.
If a question did require “invert a binary tree” to be part of the optimal answer, then identifying this would be the most important thing. If the candidate then coded an suboptimal version, whilst mentioning that a more optimal one could exist, that’s almost max points.
I think there is a slight focus on “tricks” for coding interviews, but from the questions I’ve done and seen, these are more general problem solving strategies, rather than the slightly esoteric algorithmic tricks that get brought up on HN.
I haven’t seen a current question where a very specific algorithmic trick is required to get a good rating (but that could very well be my lack of experience). I guess dynamic programming may be the closest thing. But I would say that’s a more general strategy and the questions I’ve seen that use it are fairly rare. I’ve never seen a question require it for a good rating, the ones I’ve seen have it as extension type stuff.
If a question did require “invert a binary tree” to be part of the optimal answer, then identifying this would be the most important thing. If the candidate then coded an suboptimal version, whilst mentioning that a more optimal one could exist, that’s almost max points.
I think there is a slight focus on “tricks” for coding interviews, but from the questions I’ve done and seen, these are more general problem solving strategies, rather than the slightly esoteric algorithmic tricks that get brought up on HN.
I haven’t seen a current question where a very specific algorithmic trick is required to get a good rating (but that could very well be my lack of experience). I guess dynamic programming may be the closest thing. But I would say that’s a more general strategy and the questions I’ve seen that use it are fairly rare. I’ve never seen a question require it for a good rating, the ones I’ve seen have it as extension type stuff.
Your mentality is “throw them at a hacker rank problem and rate who had the best American Idol performance.”
Which is braindead. You can claim that’s not your process but ultimately you’re trying to get someone to present an output based on your input while also not accepting that the process is not always the same.
Let’s explain by analogy here: I’m hiring a great /Painter/ for a portrait of my family. Would it be reasonable to call in 10 painters and show them a photo of my family and say “show me what you can paint in 10 minutes!”?
Do you think that’s a good metric to hire creative talent? Or, perhaps you don’t think they’re creative, and you make your portrait a paint by numbers, and then say “quick, take this color and mix it, then paint in the numbers, let’s see who can mix the color really fast!”
Or maybe you call the 10 painters in and say “show me your personal gallery, that will be all!” So you dismiss 7/10 painters because they sold their work and it doesn’t ride around with them.
Do you feel that you’re hiring a good painter?
Which is braindead. You can claim that’s not your process but ultimately you’re trying to get someone to present an output based on your input while also not accepting that the process is not always the same.
Let’s explain by analogy here: I’m hiring a great /Painter/ for a portrait of my family. Would it be reasonable to call in 10 painters and show them a photo of my family and say “show me what you can paint in 10 minutes!”?
Do you think that’s a good metric to hire creative talent? Or, perhaps you don’t think they’re creative, and you make your portrait a paint by numbers, and then say “quick, take this color and mix it, then paint in the numbers, let’s see who can mix the color really fast!”
Or maybe you call the 10 painters in and say “show me your personal gallery, that will be all!” So you dismiss 7/10 painters because they sold their work and it doesn’t ride around with them.
Do you feel that you’re hiring a good painter?
Firstly, I never said whether algorithmic interviews were good or not (still unsure tbh), I wanted to provide a counter-point against a common trope that it is just memorising algorithmic tricks to do the interview.
I’m not sure I understand your point. My guess is that you’re saying that I’m expecting a very specific answer whilst the difference in process might result in different answers? My binary tree question is a made up example to illustrate a point, but it is a bit misleading. There’s no expectation that people will identify the “invert a binary tree”, as long as they can describe the optimal solution some other way. However, in case the candidate doesn’t immediately spot the full optimal solution, they can break it down into steps, one of which might be to identify a good data structure (a binary tree) and a useful operation (inversion) and go from there.
I’m not sure how to even start engaging the painter analogy, but I will point out a flaw in the problem. A large reason FAANG does the interviews this way is because they can quantise it better and distribute not just the interviewing between interviewers, but also the training of new interviewers.
So I guess a slightly more relevant analogy would be you trying to hire 100 painters worldwide for your large painting company after interviewing 1000 people whilst not having the time to interview them all personally.
I’m not sure I understand your point. My guess is that you’re saying that I’m expecting a very specific answer whilst the difference in process might result in different answers? My binary tree question is a made up example to illustrate a point, but it is a bit misleading. There’s no expectation that people will identify the “invert a binary tree”, as long as they can describe the optimal solution some other way. However, in case the candidate doesn’t immediately spot the full optimal solution, they can break it down into steps, one of which might be to identify a good data structure (a binary tree) and a useful operation (inversion) and go from there.
I’m not sure how to even start engaging the painter analogy, but I will point out a flaw in the problem. A large reason FAANG does the interviews this way is because they can quantise it better and distribute not just the interviewing between interviewers, but also the training of new interviewers.
So I guess a slightly more relevant analogy would be you trying to hire 100 painters worldwide for your large painting company after interviewing 1000 people whilst not having the time to interview them all personally.
>A large reason FAANG does the interviews this way is because they can quantise it better and distribute not just the interviewing between interviewers, but also the training of new interviewers.
Almost! They do it this way because they are stupid individuals who are paid to hire individuals more intelligent than they are.
If anyone finds out they aren’t necessary then you and your job go away.
To wit: if your hiring process is transparent and commands no respect why would your daily operations be different? Lol! Oh no!
Almost! They do it this way because they are stupid individuals who are paid to hire individuals more intelligent than they are.
If anyone finds out they aren’t necessary then you and your job go away.
To wit: if your hiring process is transparent and commands no respect why would your daily operations be different? Lol! Oh no!
> They do it this way because they are stupid individuals who are paid to hire individuals more intelligent than they are.
This is partly inevitable, not every interviewer will be smarter than the candidates they are interviewing.
I’m curious why your distribution is so skewed though (all interviewers are dumber than all candidates). I’m not sure how satisfiable this is, considering that some of the candidates become employees and then future interviewers. To keep this constraint, there must continue to be an unbounded growth of intelligence at FAANG. If that’s the case, it seems like a great place to learn from smart people.
> To wit: if your hiring process is transparent and commands no respect
Would definitely say the hiring process is not as transparent as it could be. Not sure about “commands no respect”. Considering how popular algorithmic interviews have become for non-FAANG companies, there must be some people who decided to value it. Perhaps you don’t consider the opinion of anyone who does value it important enough to include. But then your sample becomes slightly biased right?
This is partly inevitable, not every interviewer will be smarter than the candidates they are interviewing.
I’m curious why your distribution is so skewed though (all interviewers are dumber than all candidates). I’m not sure how satisfiable this is, considering that some of the candidates become employees and then future interviewers. To keep this constraint, there must continue to be an unbounded growth of intelligence at FAANG. If that’s the case, it seems like a great place to learn from smart people.
> To wit: if your hiring process is transparent and commands no respect
Would definitely say the hiring process is not as transparent as it could be. Not sure about “commands no respect”. Considering how popular algorithmic interviews have become for non-FAANG companies, there must be some people who decided to value it. Perhaps you don’t consider the opinion of anyone who does value it important enough to include. But then your sample becomes slightly biased right?
Agreed that it's not perfect.
But asking to talk about technical topics is what is essentially trivia - is an absolutely worse approach.
But asking to talk about technical topics is what is essentially trivia - is an absolutely worse approach.
Testing for simple algorithms knowledge biases for recently graduated people who have these in their active memory. Experienced people rarely encounter these details in practice and are more likely to have their heads full of deep knowledge no one in your organization knows about. If you screen these people out, it will be quite literally your loss.
Technological fields are blue oceans where only junior people spend significant time near the safe harbors. Experienced people have sailed far away and seen lands you don't know about yet.
People also have this inherent bias of treating everything they know well as some common knowledge everyone else knows as well, or at least should know. This bias of underestimating the vastness of the field is largely because of inexperience. People who haven't seen how deep it goes are liable to think that the basic university exercises are the important things.