In defense of coding interviews(biggestfish.substack.com)
biggestfish.substack.com
In defense of coding interviews
https://biggestfish.substack.com/p/in-defense-of-coding-interviews
387 comments
I've just interviewed for Google and Meta. In total I went through 8 individual coding interviews.
With an exception of one, which was a bit too much like a puzzle, I think the assignments were completely fair examples of what I might experience on daily basis. Not at all tailored to people doing competition style programming, which was what I expected after being exposed to HN for years.
I have to say I enjoyed both preparing for the interview as well as the interviews themselves. And I believe that overall the interviewers were able to grasp how I would code.
With an exception of one, which was a bit too much like a puzzle, I think the assignments were completely fair examples of what I might experience on daily basis. Not at all tailored to people doing competition style programming, which was what I expected after being exposed to HN for years.
I have to say I enjoyed both preparing for the interview as well as the interviews themselves. And I believe that overall the interviewers were able to grasp how I would code.
If you have to prepare for the interview, doesn't that suggest the assessment isn't representative of day-to-day programming skills? In fact, it suggests the exact opposite.
With zero preparation, I could go to an interview with you right now and tell you anything you wanted to know about designing a database schema, how to get any kind of data out of that schema, how to make that data available via an API, how to cache the data and perform permissions checks, how to deploy code to a cloud environment, how to set up a build pipeline, or any other question you might want to ask about the day-to-day process of developing and deploying web applications. I can answer those questions because I already work in the industry and I broadly know what I'm doing.
However, I probably couldn't pass a FAANG interview without months of prep work, precisely because whiteboard interviews bear almost no resemblance to the reality of software development.
With zero preparation, I could go to an interview with you right now and tell you anything you wanted to know about designing a database schema, how to get any kind of data out of that schema, how to make that data available via an API, how to cache the data and perform permissions checks, how to deploy code to a cloud environment, how to set up a build pipeline, or any other question you might want to ask about the day-to-day process of developing and deploying web applications. I can answer those questions because I already work in the industry and I broadly know what I'm doing.
However, I probably couldn't pass a FAANG interview without months of prep work, precisely because whiteboard interviews bear almost no resemblance to the reality of software development.
The items you list may not be as universal in software development as you seem to think they are. There are plenty of highly competent programmers who haven’t thought about designing a database schema in many years, or who have never deployed code to a cloud environment (at least not in the way you have in mind).
Having a set of common knowledge to interview about is in fact very useful.
Having a set of common knowledge to interview about is in fact very useful.
Interviewing is assessing how a person would perform, not an actual day to day collaboration. There is a gap. You do not talk like you would to a colleague. You have a limited time and it's more one-sided to even the playing field. It's worth preparing for to overcome the gap.
It's like a dance. Your are supposed to lead deriving the solution, interviewer is supposed to follow closely and judge your performance.
If you expect to ace an interview just like that then you also need to be lucky not accidentally getting asked to stuff, which you always need to Google an answer to, basic algorithms and patterns, which you usually kind of know but could slow you down during the precious 40 minutes you have to show off your skills.
And lastly, people do prepare for these. If you do not do that, you might simply make a slightly worse impression than someone who did.
That being said, I mostly did not prepare for the coding interviews. I focused on the design and behavioral interviews.
It's like a dance. Your are supposed to lead deriving the solution, interviewer is supposed to follow closely and judge your performance.
If you expect to ace an interview just like that then you also need to be lucky not accidentally getting asked to stuff, which you always need to Google an answer to, basic algorithms and patterns, which you usually kind of know but could slow you down during the precious 40 minutes you have to show off your skills.
And lastly, people do prepare for these. If you do not do that, you might simply make a slightly worse impression than someone who did.
That being said, I mostly did not prepare for the coding interviews. I focused on the design and behavioral interviews.
I would never change the way I speak to someone just because they're interviewing me. If you're interviewing me it's with a view to working with me on your team a few months later. You would surely want to see the real me, not some constrained version. Maybe it's an age thing, but I'm comfortable enough with my skills and my knowledge that I don't feel the need to put on an act. And remember that you're interviewing the interviewers as much as they're interviewing you.
Also, I would also be quite surprised if Google didn't allow candidates to Google information during the interview...
Also, I would also be quite surprised if Google didn't allow candidates to Google information during the interview...
> Also, I would also be quite surprised if Google didn't allow candidates to Google information during the interview
I did a Google onsite several years ago (probably 7 or 8 years ago at this point). Each interview was just me in a room with a whiteboard and a person sitting at a table across from me with a notepad. No computer present.
I suppose I could have dug out my phone and Googled, but that feels unprofessional to me, moreso than if I had access to a computer.
Also the part I got stuck on was coming up with the trick needed to solve the problems. Most I was fine, but one in particular would have taken some decent searching to find something equivalent if I just Googled, and would have taken some time to grok the answer. Also that guy was not very forthcoming with hints or information when I asked him, so I doubt he'd have appreciated a Google search.
I don't remember the exact question, but it had to do with converting a 3D city model into a 2D skyline by drawing a single line. I got stuck on trying to visualize how to deal with partial overlapping models and irregular building shapes and ran out of time.
I did a Google onsite several years ago (probably 7 or 8 years ago at this point). Each interview was just me in a room with a whiteboard and a person sitting at a table across from me with a notepad. No computer present.
I suppose I could have dug out my phone and Googled, but that feels unprofessional to me, moreso than if I had access to a computer.
Also the part I got stuck on was coming up with the trick needed to solve the problems. Most I was fine, but one in particular would have taken some decent searching to find something equivalent if I just Googled, and would have taken some time to grok the answer. Also that guy was not very forthcoming with hints or information when I asked him, so I doubt he'd have appreciated a Google search.
I don't remember the exact question, but it had to do with converting a 3D city model into a 2D skyline by drawing a single line. I got stuck on trying to visualize how to deal with partial overlapping models and irregular building shapes and ran out of time.
Did you pass the interview?
I might not have been clear. It's not an act. You are not usually guiding your peer trough your thought process our loud step by step. That's inefficient. Yet that is what you should do to maximize the amount of information passed onto the interviewer. It's just a different setup to which most people need to adjust to.
You just described a CRUD app.
At FAANG, most people are not doing any of these things. Think of people working on EC2, Ads or Maps.
I only worked at multiple FAANG and I never had to design a database schema or make data available via API.
At FAANG, most people are not doing any of these things. Think of people working on EC2, Ads or Maps.
I only worked at multiple FAANG and I never had to design a database schema or make data available via API.
Google publishes many APIs. I know, because we consume them. My company in turn publishes APIs that are consumed by other companies. You're probably right that most people working at FAANG are not allowed to touch a database or set up deployments, however.
As a sibling commenter pointed out, you just described a CRUD app. Web/mobile applications.
It's not hard to do those.
And I suppose for most business cases out there, writing data to the DB and reading it back later is all it takes.
However, there's plenty of other problems where the skillset you mentioned falls short.
For instance, you talked about designing a database schema. Do you know how to design and build a database itself from the ground up? What if that DB has to be distributed, with thousands of writers? How do you ensure consistency and consensus? (These are all solved problems).
Think about Linux kernel developers. Do you think DB schema design is an especially relevant skill for them to have?
Can you come up with a compiler, or your own programming language?
Try building your own text editor, with features like syntax highlighting, undo/redo etc (it takes clever algorithms).
I'm not claiming I can do all of that; just mentioning that the whole scene is much more interesting beyond the horizon.
It's not hard to do those.
And I suppose for most business cases out there, writing data to the DB and reading it back later is all it takes.
However, there's plenty of other problems where the skillset you mentioned falls short.
For instance, you talked about designing a database schema. Do you know how to design and build a database itself from the ground up? What if that DB has to be distributed, with thousands of writers? How do you ensure consistency and consensus? (These are all solved problems).
Think about Linux kernel developers. Do you think DB schema design is an especially relevant skill for them to have?
Can you come up with a compiler, or your own programming language?
Try building your own text editor, with features like syntax highlighting, undo/redo etc (it takes clever algorithms).
I'm not claiming I can do all of that; just mentioning that the whole scene is much more interesting beyond the horizon.
I once worked on a system that included a SQL-ish interpreter (starting from an ANTLR-based parser and all the way to distributed plan execution). I don't remember discussing it in interviews in the last five years. Even at places such as Dremio or Imply. In all fairness, I _once_ got an offer when one interview in the process included sketching an ANTLR grammar for a calculator in a plain text editor.
As someone who has done the interview rounds at FAANG and many dozens of other companies in SV - you either work in algorithmic work or got lucky with your questions.
The questions that are asked have little to no basis in my daily work as an engineer. Same is true for the hundreds of other engineers I’ve worked with who have gone through the same hoops that I have.
The questions that are asked have little to no basis in my daily work as an engineer. Same is true for the hundreds of other engineers I’ve worked with who have gone through the same hoops that I have.
Interesting. Might have been just lucky.
> I think the assignments were completely fair examples of what I might experience on daily basis.
In my experience, you have a lot more than an hour to do it in the real job. And while perhaps 1% of SW folks will deal with in on a daily basis, most will not. I certainly have gone years without needing to use graph algorithms (including even DFS/BFS). And I've needed a BST only once in a decade - and I didn't need to code it, just needed to understand the data structure and complexity to know that "binary_search" in C++ was a superior option than using a map for my use case (even though the complexity is the same).
Some context: At work I recently solved a problem where I had to build a graph (DAG), and do some queries on it mildly efficiently[1]. I did it quickly thanks to wasting time on Leetcode, but no one knew how quick, and people were impressed that I "got it done in less than a day". The reality in most SW work is that whether it takes you a day or an hour to solve this problem, the overall rate of progress will not be impacted.
[1] Mildly efficiently because for the first POC pass, I didn't memoize, and then I got lazy and decided I'd optimize it only after getting a real world data set and seeing if it was slow. It wasn't, even for a decently large data set. The reason? Business logic dictated that the max distance between two nodes did not exceed 8, so from a complexity stand point, you're not increasing the work N-fold by not memoizing, but it's just changing the constant factor.
Somehow, I suspect that in half of the FAANG interviews, had I coded this and argued that the complexity didn't change, I would not get a pass.[2]
[2] Not trying to be cynical: I did argue this for a different problem in a Google phone screen - I had been asked to write some code + data structures to handle an Manager/Report relationship, and the interviewer pointed out my data structure was less than optimal. I countered with "In most companies, the number of employees a manager has under him/her is usually bounded by a small number." Surprisingly, the interviewer gave in.
In my experience, you have a lot more than an hour to do it in the real job. And while perhaps 1% of SW folks will deal with in on a daily basis, most will not. I certainly have gone years without needing to use graph algorithms (including even DFS/BFS). And I've needed a BST only once in a decade - and I didn't need to code it, just needed to understand the data structure and complexity to know that "binary_search" in C++ was a superior option than using a map for my use case (even though the complexity is the same).
Some context: At work I recently solved a problem where I had to build a graph (DAG), and do some queries on it mildly efficiently[1]. I did it quickly thanks to wasting time on Leetcode, but no one knew how quick, and people were impressed that I "got it done in less than a day". The reality in most SW work is that whether it takes you a day or an hour to solve this problem, the overall rate of progress will not be impacted.
[1] Mildly efficiently because for the first POC pass, I didn't memoize, and then I got lazy and decided I'd optimize it only after getting a real world data set and seeing if it was slow. It wasn't, even for a decently large data set. The reason? Business logic dictated that the max distance between two nodes did not exceed 8, so from a complexity stand point, you're not increasing the work N-fold by not memoizing, but it's just changing the constant factor.
Somehow, I suspect that in half of the FAANG interviews, had I coded this and argued that the complexity didn't change, I would not get a pass.[2]
[2] Not trying to be cynical: I did argue this for a different problem in a Google phone screen - I had been asked to write some code + data structures to handle an Manager/Report relationship, and the interviewer pointed out my data structure was less than optimal. I countered with "In most companies, the number of employees a manager has under him/her is usually bounded by a small number." Surprisingly, the interviewer gave in.
This comment reminds me of a comic with two vultures who question Mr. Mouse’s claims about the Owl being a predator. It’s not like he ever bothered them. See here: https://twitter.com/nathanwpyle/status/999294987195035649
I appreciate the reference.
But, I don't think those are vultures. They are bald eagles. I'm not sure if owls scavenge, but I do know they primarily hunt. Eagles both hunt and scavenge. Vultures primarily scavenge.
But, I don't think those are vultures. They are bald eagles. I'm not sure if owls scavenge, but I do know they primarily hunt. Eagles both hunt and scavenge. Vultures primarily scavenge.
The comic could definitely use some annotations :)
Yeah. I might be generalizing a bit, even though I made sure to emphasize that it is my isolated experience.
On the other hand I felt like a sample of 8 different interviewers from different offices and backgrounds felt representative. If it is true that the interviews are overwhelmingly negatively received, what is the probability that I would get more than half of reasonable interviews? Let alone 7/8.
On the other hand I felt like a sample of 8 different interviewers from different offices and backgrounds felt representative. If it is true that the interviews are overwhelmingly negatively received, what is the probability that I would get more than half of reasonable interviews? Let alone 7/8.
You're overlooking the fact that you're just one person. Each one of us reacts differently under such conditions and only a minority (which is likely small) happen to excel at this.
The interactions and power dynamics between people differ wildly from person to person. While an interviewer might be the kindest person in the world for candidate A, they might be a total bully for candidate B. I was bullied constantly throughout school snd, whatever cues prompted those kids to pick on me are probably still there, but adults are a bit more subtle, because they can hurt you in other ways. And this is on top of the day-to-day variations, because they might be hungry and the current interview is sitting between them and lunch or maybe they just got slapped down by their boss and want to punch someone or whatever. Based on my own experience, out of the typical 5 rounds of in-person interviews that I went through many times for FAANG companies, one or two ended up being a total shitshow each time, where the interviewer was condescending at best and I don't think they realised it or cared. I was told several times "how easy / trivial" the problem is and that "they're sure I'll have an easy time solving it based on my CV". Maybe the worst ones are when the interviewer smirks at me when they spot dome issue and I can't cope with that.
Now add to this a long streak of never ever getting an offer after dozens of such interviews in spite of putting myself through training sessions and mock interviews over and over. I'm sure I'd get in eventually, if I just kept trying, but do I want a job where I'll be asked to interview people in a similar manner? Even if I refuse to interview people as part of my job, will I really thrive in there? I'm certain this process permeates in one form or another throughout the company culture in various internal management, decisionmaking and promotion processes, but that's a separate discussion.
The interactions and power dynamics between people differ wildly from person to person. While an interviewer might be the kindest person in the world for candidate A, they might be a total bully for candidate B. I was bullied constantly throughout school snd, whatever cues prompted those kids to pick on me are probably still there, but adults are a bit more subtle, because they can hurt you in other ways. And this is on top of the day-to-day variations, because they might be hungry and the current interview is sitting between them and lunch or maybe they just got slapped down by their boss and want to punch someone or whatever. Based on my own experience, out of the typical 5 rounds of in-person interviews that I went through many times for FAANG companies, one or two ended up being a total shitshow each time, where the interviewer was condescending at best and I don't think they realised it or cared. I was told several times "how easy / trivial" the problem is and that "they're sure I'll have an easy time solving it based on my CV". Maybe the worst ones are when the interviewer smirks at me when they spot dome issue and I can't cope with that.
Now add to this a long streak of never ever getting an offer after dozens of such interviews in spite of putting myself through training sessions and mock interviews over and over. I'm sure I'd get in eventually, if I just kept trying, but do I want a job where I'll be asked to interview people in a similar manner? Even if I refuse to interview people as part of my job, will I really thrive in there? I'm certain this process permeates in one form or another throughout the company culture in various internal management, decisionmaking and promotion processes, but that's a separate discussion.
> Even if I refuse to interview people as part of my job, will I really thrive in there? I'm certain this process permeates in one form or another throughout the company culture in various internal management, decisionmaking and promotion processes, but that's a separate discussion.
Are you sure you want to work at a FAANG company? You sure you would be happy? What motivates you to try so hard?
Are you sure you want to work at a FAANG company? You sure you would be happy? What motivates you to try so hard?
Bingo! I now know I don't, but it took me years to come to this conclusion. When I started this career, everyone around was telling me that getting into one of these companies will lead to meaningful and challenging work which is complete nonsense. I now know that a large number of people who join these companies are unable to find any meaningful work there.
Anyhow, several years ago, I added a disclaimer to my public profile on LinkedIn which states that I refuse any form of live / timed coding assessments during interviews and I'm not going to make any exceptions on it.
Anyhow, several years ago, I added a disclaimer to my public profile on LinkedIn which states that I refuse any form of live / timed coding assessments during interviews and I'm not going to make any exceptions on it.
> preparing for the interview
Not sure what is wrong with preparing for a challenging interview. What is the point you are making?
To add some thought to this.
The interview results need to be reasonably repeatable and comparable to assess interviewer skew and bias.
The result is that the interviews themselves inevitably end up being somewhat predicable. And what is predicable you can prepare for, giving you predictably better results.
If you are top 0.1%, sure you will nail them without much prep, but most of the candidates would spend some time preparing.
The interview results need to be reasonably repeatable and comparable to assess interviewer skew and bias.
The result is that the interviews themselves inevitably end up being somewhat predicable. And what is predicable you can prepare for, giving you predictably better results.
If you are top 0.1%, sure you will nail them without much prep, but most of the candidates would spend some time preparing.
You mentioned that it is not about Competitive Programming/leetcode stuff but a daily real world work. So how do you prepare? Just continue doing your daily work?
You prepare by making sure all your bases are covered. In my daily work discussions, for example, I tend to be very hand-wavy about concurrency, calling everything “locks” and hoping people understand that I can dig down into the details when needed. When I’m interviewing I don’t want to just hope, so I review my concurrency primitives to try and make sure I won’t put my foot in my mouth.
The fact that people prepare for an interview doesn't say per se it's a failed model.
What type of questions were there?
> I've just interviewed for Google and Meta. In total I went through 8 individual coding interviews.
Curious, were there any questions about what you think of their company ethics, and if you have a problem with it?
Curious, were there any questions about what you think of their company ethics, and if you have a problem with it?
One thing to realize is that interviews are a two-way street. Do you think you want to work with the interviewers? Do you like the interview process? If the interview process is poor, what other kind of hiring decisions is the company making? Are you going to want to work with the people they end up hiring?
You learn a bit about the company during the interview, and if that bit is bad, then move on. If they ask shitty leetcode questions and the interviewers are hostile or hung up on unimportant details - move on to another company.
You learn a bit about the company during the interview, and if that bit is bad, then move on. If they ask shitty leetcode questions and the interviewers are hostile or hung up on unimportant details - move on to another company.
The problem is when a significant number of companies all cargo cult the same process- that way you can’t even write them off as a one-off bad place.
Do they cargo cult the same process? I do see similarities in the interview process, so maybe that's somewhat true, but in my experience there's a ton of variety. Different companies have different formats, different questions, focus on different aspects of my responses, and I'm sure make hiring decisions wildly differently.
Some places have phone screens, and some don't. One company gave an easy leetcode problem as part of the phone screen, while in another case, one person actually spoke characters over the phone to me ("LinkedList", "left angle bracket", "String", "right angle bracket"). Some give take home questions. Some do a code review interview. Some dig into my resume during an interview, and some don't. Some companies had multiple interview sessions where I was asked the same question about my resume in each session by different people (wasting time). And so on.
So in general, I can and have passed over companies based on their hiring process.
Some places have phone screens, and some don't. One company gave an easy leetcode problem as part of the phone screen, while in another case, one person actually spoke characters over the phone to me ("LinkedList", "left angle bracket", "String", "right angle bracket"). Some give take home questions. Some do a code review interview. Some dig into my resume during an interview, and some don't. Some companies had multiple interview sessions where I was asked the same question about my resume in each session by different people (wasting time). And so on.
So in general, I can and have passed over companies based on their hiring process.
I’ve interviewed at hundreds of places all within SV - I can say quite confidently that they’re almost all the same. There are some minor variations but they’re almost all completely forgettable by the end of the day. If they have something truly novel - it’s usually very off putting and requires intense amounts of unpaid labor… which you can’t take with you to other interviews.
The variations you’re talking about are quite minute. Whether someone asks you about your resume or asks you about your most technically challenging project is all relatively mundane stuff.
The variations you’re talking about are quite minute. Whether someone asks you about your resume or asks you about your most technically challenging project is all relatively mundane stuff.
You can still write off cargo-culting companies who cargo-cult bad processes off. I give you permission.
Every company cargo-cults or should we say - follows best practices or industry wide processes - in something.
No company has thought through every little bit of their system and processes because no company has the time to be completely original, just like no human idea only has ideas that they thought up themselves instead of borrowing a bunch of ideas from school, books, and overheard conversation because they sound right.
No company has thought through every little bit of their system and processes because no company has the time to be completely original, just like no human idea only has ideas that they thought up themselves instead of borrowing a bunch of ideas from school, books, and overheard conversation because they sound right.
It is better to not know and explore than adopt a bad process because you overheard it in a coffee shop.
It's ok to not have an answer.
It's ok to not have an answer.
You have personally used at least one tool, language, or framework without reading its code or understanding it beyond a surface level. You've done this because some text on some website claims it gets the job done. And you probably do this every day (e.g. your OS, your phone, your microwave).
People cargo cult because there isn't enough time to exhaustively research every topic from scratch. Civilization exists because we blindly trust whatever methods seem to work, and then iterate on them.
People cargo cult because there isn't enough time to exhaustively research every topic from scratch. Civilization exists because we blindly trust whatever methods seem to work, and then iterate on them.
I'm going to put the definition of cargo culting here since we seem to have lost it:
> Cargo cult programming can also refer to the practice of applying a design pattern or coding style blindly without understanding the reasons behind that design principle. Some examples are adding unnecessary comments to self-explanatory code, overzealous adherence to the conventions of a programming paradigm, or adding deletion code for objects that garbage collection automatically collects.
Installing an app from the store is not cargo culting. Playing with a new framework is not cargo culting. Trying a new order at the donut shop is not cargo culting.
> Cargo cult programming can also refer to the practice of applying a design pattern or coding style blindly without understanding the reasons behind that design principle. Some examples are adding unnecessary comments to self-explanatory code, overzealous adherence to the conventions of a programming paradigm, or adding deletion code for objects that garbage collection automatically collects.
Installing an app from the store is not cargo culting. Playing with a new framework is not cargo culting. Trying a new order at the donut shop is not cargo culting.
several points, I agree installing an app is not cargo culting, however following the same interview process as many other companies do is not 'cargo cult programming' while I would agree it is cargo culting of a sort.
the problem really is that results for interview process does not seem to reliably add up to anything, there are no reasons behind any interview methodology in the traditional cargo culting sense because there is no interview methodology really founded on reason, it's mumbo jumbo all the way down.
But back to my original point some posts up, everybody has at some point taken something on faith as a best practice that works without understanding why.
Some examples where cargo cult programming are concerned are adding unnecessary comments to self-explanatory code - but how about also one I've had requesting the removal of explanatory comments from code because the code should explain itself (comment was something like, have done it this way instead of what might seem like the more reasonable way ..example code.. because of a bug in Safari that means X happens, etc. etc, if you want to removal or change to better method please check that Safari works now and this place actually wanted it removed because they preferred not to have comments in code!)
Or how about this one which I am old enough to have done: putting your script tags in the head of the document because this was a best practice (before async and defer attributes, I'm talking 1999-2000 here)
Hell many of the best practices around CSS usage have depending on the year been the exact opposite of some other earlier best practice. And many frontend developers do it because they don't have the time or evidently the inclination to really learn CSS.
Hey, many best practices regarding CSS at various times are based around the fact that frontend developers don't want to learn to use it.
Now you might say those are not really examples of cargo culting, but I think they are somewhat close - if you are using a naming convention to avoid learning how something works then you don't really know why you are using the naming convention because the purpose of that convention is to keep you from that knowledge.
anyway getting off track here, I just think that saying you are not going to work at a place that has cargo culted its interview process does not tell you anything about the overall quality of the place, because everyone has cargo culted something, just like many fine developers when setting up their React projects have said oh, everyone uses Webpack we'll do that as our build process - they know they need a build, but why Webpack? They don't really know (me neither, every time I've ever encountered Webpack I've always ended up wondering why!? Generally the answer is because that's what the tutorials had, everyone used it, they didn't give any thought, but now we're stuck with it)
the problem really is that results for interview process does not seem to reliably add up to anything, there are no reasons behind any interview methodology in the traditional cargo culting sense because there is no interview methodology really founded on reason, it's mumbo jumbo all the way down.
But back to my original point some posts up, everybody has at some point taken something on faith as a best practice that works without understanding why.
Some examples where cargo cult programming are concerned are adding unnecessary comments to self-explanatory code - but how about also one I've had requesting the removal of explanatory comments from code because the code should explain itself (comment was something like, have done it this way instead of what might seem like the more reasonable way ..example code.. because of a bug in Safari that means X happens, etc. etc, if you want to removal or change to better method please check that Safari works now and this place actually wanted it removed because they preferred not to have comments in code!)
Or how about this one which I am old enough to have done: putting your script tags in the head of the document because this was a best practice (before async and defer attributes, I'm talking 1999-2000 here)
Hell many of the best practices around CSS usage have depending on the year been the exact opposite of some other earlier best practice. And many frontend developers do it because they don't have the time or evidently the inclination to really learn CSS.
Hey, many best practices regarding CSS at various times are based around the fact that frontend developers don't want to learn to use it.
Now you might say those are not really examples of cargo culting, but I think they are somewhat close - if you are using a naming convention to avoid learning how something works then you don't really know why you are using the naming convention because the purpose of that convention is to keep you from that knowledge.
anyway getting off track here, I just think that saying you are not going to work at a place that has cargo culted its interview process does not tell you anything about the overall quality of the place, because everyone has cargo culted something, just like many fine developers when setting up their React projects have said oh, everyone uses Webpack we'll do that as our build process - they know they need a build, but why Webpack? They don't really know (me neither, every time I've ever encountered Webpack I've always ended up wondering why!? Generally the answer is because that's what the tutorials had, everyone used it, they didn't give any thought, but now we're stuck with it)
A simple test: Do they hire like Google but manage their programmers like a sweatshop (ie Scrum)? Then they are just cargo culting. Companies like Google at least gives its engineers a lot of responsibility and freedom you'd never get in an Agile shop.
a) plenty of agile shops give you more freedom than Google does. Following Agile methodology really doesn't mean what you think it means; and
b) please don't casually "ie sweatshops" when talking about something that is clearly not that.
Do your developers have to work in 2 week sprints and forced to work on tasks split into sub 1-day part so they never have any real autonomy? Then you aren't giving your developers more freedom than Google. And if you aren't working like that then you aren't doing Scrum.
Google gives even junior developers multi month projects to own, manage and complete. No Scrum shop gives developers that level of autonomy, not even to senior developers.
So to me the time I've spent working on Scrum projects feels like a sweatshop compared to how I could plan and structure development at Google. At Google I am free to collaborate with stakeholders, build prototypes and get feedback etc, as I see fit to complete the project. Or not do it when I don't feel it is needed, the important part isn't how I run the project but that I run it well. If you don't give your developers that level of autonomy then I'm not sure why you'd care much about developer competence at all. (I quit Google a few years ago though due to how the company was changing, but I'll never join a Scrum development team again)
Google gives even junior developers multi month projects to own, manage and complete. No Scrum shop gives developers that level of autonomy, not even to senior developers.
So to me the time I've spent working on Scrum projects feels like a sweatshop compared to how I could plan and structure development at Google. At Google I am free to collaborate with stakeholders, build prototypes and get feedback etc, as I see fit to complete the project. Or not do it when I don't feel it is needed, the important part isn't how I run the project but that I run it well. If you don't give your developers that level of autonomy then I'm not sure why you'd care much about developer competence at all. (I quit Google a few years ago though due to how the company was changing, but I'll never join a Scrum development team again)
"And if you aren't working like that then you aren't doing Scrum"
I kindly disagree. You seem to have worked at a very stressfull place. In my experience that has nothing to do with it being "true scrum TM" or not. In any case, happy for you that you got out.
I kindly disagree. You seem to have worked at a very stressfull place. In my experience that has nothing to do with it being "true scrum TM" or not. In any case, happy for you that you got out.
Always this "if you have any complaints about Scrum then you aren't doing it correctly" with no explanation how you'd fix the problems I talked about.
Also the leetcode interview is cargo-culted from big tech companies like Google and Scrum is cargo-culted from big tech consultants. They don't fit well together, having both means your company just picked the most popular/simple way to do each part without considering why Google uses that interview or tech consultants uses Scrum. If you want your engineers to work on problems similar to at Google then you wouldn't use Scrum, and if you want your engineers to work on similar problems as big tech consultants then you wouldn't use the leetcode interview.
Also the leetcode interview is cargo-culted from big tech companies like Google and Scrum is cargo-culted from big tech consultants. They don't fit well together, having both means your company just picked the most popular/simple way to do each part without considering why Google uses that interview or tech consultants uses Scrum. If you want your engineers to work on problems similar to at Google then you wouldn't use Scrum, and if you want your engineers to work on similar problems as big tech consultants then you wouldn't use the leetcode interview.
Scrum does not mean "no freedom" or "no autonomy" it just says lets think about what we're going to do and try to follow a plan for the next week or two. Of course the individual software developer should be the one responsible to forming, adapting, and updating this plan (and it definitely should contain " collaborate with stakeholders, build prototypes and get feedback etc,") (of course, there is some level of 'commitment'. As in other project management systems, you should try and do what you said you'll do).
If as you say, you had no autonomy/freedom, this has nothing to with scrum, that's just regular bad management.
If as you say, you had no autonomy/freedom, this has nothing to with scrum, that's just regular bad management.
Sanity check: Would your manager complain if you stopped doing Scrum? If yes then you don't got that much freedom. If no, then I can see it being fine. It can work fine if nobody outside of it depends or uses the Scrum process, but then the company isn't doing Scrum, just the team doing it internally. This lets the team adapt and stop doing Scrum when it no longer fits their problem etc, that is the agile way. My point was just that if a company uses both the leetcode interview and the tightly integrated Scrum process mandated on teams then that doesn't make sense.
Edit: And a job ad where they say you should be well versed in Scrum I take it as they will force Scrum on me regardless if it makes sense or not, because I doubt a team just doing Scrum for themselves would put that in a job ad. Companies loves putting this in their job ads though so it is very easy to spot which companies works like this.
Edit: And a job ad where they say you should be well versed in Scrum I take it as they will force Scrum on me regardless if it makes sense or not, because I doubt a team just doing Scrum for themselves would put that in a job ad. Companies loves putting this in their job ads though so it is very easy to spot which companies works like this.
"You can't stop using scrum without your manager complaining" is not the same as "zero autonomy".
In every Scrum team I’ve personally worked on (admittedly not a huge sample size), it was OK for a senior engineer to say that their work for the next two weeks isn’t scrum-able so they’re going to mark themselves as having no capacity this sprint.
Yes, and my point was that juniors at Google have more freedom to go and work on different things than seniors have in a typical Scrum team. The description for the junior role (level 3) at Google says that they go off and work on individual projects lasting no longer than 3 months. Having a project for a month or two isn't an exception, it is expected from juniors. Then as you advance levels you get more and more freedom, and as a senior IC you are basically a free agent expected to be able to find things to improve, write a design for that, sell it to the higher ups on the product, and then go and build it, possibly get a junior to build it instead as a few months project, possibly get headcount and set up a team to maintain it etc, on your own.
I'm not saying that this way is strictly superior, but you undeniably have way more freedom under it than Scrum.
I'm not saying that this way is strictly superior, but you undeniably have way more freedom under it than Scrum.
You're in the scrum cult dude, get over it.
Scrum is like communism, it doesn't work, stop obsessing over it.
If a company is really following scrum, then the developers are creating and managing their own tickets
Yes, nobody else could write tickets that reliably takes less than a day other than the team. The problem is that your day to day structure gets decided by a brainstorming session held every other week, that might make sense in some contexts but mandating that every developer has to work that way is exactly what I mean that you aren't allowed to have any responsibility.
To me it seems like Scrum was designed to make developers have as little individual ownership/responsibility as possible while still being able to produce relevant code. I see the merit in that, you don't have to tell me why minimizing individual ownership/responsibility could be a good thing. But I strongly believe that individual ownership is necessary to properly capitalize on the creativity of individual talent, otherwise its mostly wasted.
To me it seems like Scrum was designed to make developers have as little individual ownership/responsibility as possible while still being able to produce relevant code. I see the merit in that, you don't have to tell me why minimizing individual ownership/responsibility could be a good thing. But I strongly believe that individual ownership is necessary to properly capitalize on the creativity of individual talent, otherwise its mostly wasted.
The false negative is the obvious downside of the coding interview, but I wouldn't say that it means there's no merit to the excercise.
Take for example the "can't define an array" crowd. Are there people virtually sitting in front of me who just forget the basic syntax? Absolutly, but given we a) tell them that correct syntax isn't dreafully important, and b) let them pick the language, a [] would probably suffice, I don't think it's a completely unreasonble ask.
Moreso, when we added a prestep of basic questions that one can google, the "can't define an array" crowd mostly disappeared. Sure, there may be some other causation there, but it's hard to hire a coder that you know, can't demonstrate coding.
Finally, anecdata, since we started coding interviews, the quality of developer is simply better. We have had internal debates about this, but the truth is, everyone is overall happier with what we're doing, and I'm a lot happier that false positives are way down.
I'm sure there's other ways of course. :)
Take for example the "can't define an array" crowd. Are there people virtually sitting in front of me who just forget the basic syntax? Absolutly, but given we a) tell them that correct syntax isn't dreafully important, and b) let them pick the language, a [] would probably suffice, I don't think it's a completely unreasonble ask.
Moreso, when we added a prestep of basic questions that one can google, the "can't define an array" crowd mostly disappeared. Sure, there may be some other causation there, but it's hard to hire a coder that you know, can't demonstrate coding.
Finally, anecdata, since we started coding interviews, the quality of developer is simply better. We have had internal debates about this, but the truth is, everyone is overall happier with what we're doing, and I'm a lot happier that false positives are way down.
I'm sure there's other ways of course. :)
The style you propose probably has a high correlation with intelligence and adaptability without requiring too much upfront knowledge / experience. It's open enough to allow for mistakes or answers which show intelligence without requiring perfection. Comparatively, LC-style interviews and "do you know the compiler of this language"-style questions are looking more for knowledge than figuring things out on the spot.
Knowledge generally isn't a great indicator of future job performance, intelligence is. There might be some correlation between the two (you probably need a level of intelligence to acquire the knowledge for LC-style questions), but most jobs are layering proxy upon proxy to test something that has little to do with job performance. Being able to fuddle syntax or search basic solutions and mutate, is leagues different from what LC in an isolated environment is.
Then you add onto that the alternatives tend to be even worse. Personality testing works when done right, but the vast majority of interviewers play pretend-psychiatrist and make the entire thing way too subjective to be effective beyond filtering obvious red flags. Take-homes work, but most companies won't compensate for time and there's no framework to show which take-homes individuals have succeeded (kind of like a certification process), making them an incredible time sink.
And then there are always the cases who have little experience yet manage to pick everything up in 2-3 months and become anywhere from average to top performers.
Knowledge generally isn't a great indicator of future job performance, intelligence is. There might be some correlation between the two (you probably need a level of intelligence to acquire the knowledge for LC-style questions), but most jobs are layering proxy upon proxy to test something that has little to do with job performance. Being able to fuddle syntax or search basic solutions and mutate, is leagues different from what LC in an isolated environment is.
Then you add onto that the alternatives tend to be even worse. Personality testing works when done right, but the vast majority of interviewers play pretend-psychiatrist and make the entire thing way too subjective to be effective beyond filtering obvious red flags. Take-homes work, but most companies won't compensate for time and there's no framework to show which take-homes individuals have succeeded (kind of like a certification process), making them an incredible time sink.
And then there are always the cases who have little experience yet manage to pick everything up in 2-3 months and become anywhere from average to top performers.
> The false negative is the obvious downside of the coding interview, but I wouldn't say that it means there's no merit to the excercise.
Tell that to the people who are always the false negative.
Tell that to the people who are always the false negative.
I notice there seems to be a "No true Scottman" issue here where the critics is stating one problem and the proponent is like "yeah but what if we change it to this". I'm guessing the process at your place is much more tolerable than what people have been complaining.
I wonder at what point does it stop being "whiteboard interview" and just a "few sprinkle of algorithmic questions to test logical skill".
Also when the talents pool is large enough, obviously you can do as many brutal questions as you can and still fill the headcounts with good developers. So I think saying "but it still works" is not very productive.
I wonder at what point does it stop being "whiteboard interview" and just a "few sprinkle of algorithmic questions to test logical skill".
Also when the talents pool is large enough, obviously you can do as many brutal questions as you can and still fill the headcounts with good developers. So I think saying "but it still works" is not very productive.
It seems to me that a false negative is worse than a false positive if the goal is to hire talent. Have never understood the mentality that it is better to mistakenly turn away a high skill individual than to mistakenly hire a low skill one, you're effectively guaranteeing the former to avoid the latter. Would you write a search algorithm that behaved this way?
> Do coding interviews work in the actual world in which we live in?" No, fundamentally not.
I look forward to seeing you disrupt every major tech company in the world via your superior hiring strategy.
> EVEN IF THE CANDIDATE CAN IN FACT PROGRAM JUST FINE.
You don’t seem to have thought very hard about what employers are optimising for. Everyone realises that programming interviews produce lots of false negatives. Even the strongest programmers have bad days. From the employer’s point of view, that’s okay - minimizing false negatives is not the goal. The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.
I look forward to seeing you disrupt every major tech company in the world via your superior hiring strategy.
> EVEN IF THE CANDIDATE CAN IN FACT PROGRAM JUST FINE.
You don’t seem to have thought very hard about what employers are optimising for. Everyone realises that programming interviews produce lots of false negatives. Even the strongest programmers have bad days. From the employer’s point of view, that’s okay - minimizing false negatives is not the goal. The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.
The question is whether they are actually decreasing the number of false positives. So far I've not really seen evidence of this beyond some mere anecdotes. For such a bold claim, one would expect evidence to be readily available.
Which becomes even worse when realizing "false positive/true positive" is too binary of a definition for a spectrum such as job performance.
And I say this as someone who's in favor of giving people a quick check on whether they can actually do FizzBuzz.
Which becomes even worse when realizing "false positive/true positive" is too binary of a definition for a spectrum such as job performance.
And I say this as someone who's in favor of giving people a quick check on whether they can actually do FizzBuzz.
> For such a bold claim, one would expect evidence to be readily available.
The only way to provide this evidence would be to to conduct an RCT in which you hire some percentage of people who fail the test as a control group. The nascent psychologist in me loves this idea, but it seems unethical, and presumably this is why nobody does it.
But I disagree that we need ”evidence.” It’s flat out obvious. If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job.
The only way to provide this evidence would be to to conduct an RCT in which you hire some percentage of people who fail the test as a control group. The nascent psychologist in me loves this idea, but it seems unethical, and presumably this is why nobody does it.
But I disagree that we need ”evidence.” It’s flat out obvious. If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job.
>If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job.
You're also maximizing the odds they will get bored and feel baited. With all due respect, you could have picked a better example to illustrate your point.
This is exactly why you can't make such bold claims. You're looking at this from one perspective without considering all the variables which go into the workplace equation. So far, the best correlator to job performance is giving people work corresponding to what they will be doing at the work place. For various reasons, this isn't really happening in software development, making people all to eager to grasp at proxies which they swear will do the same thing. Maybe whiteboarding is in line with the day-to-day at FAANGs, but tons of companies ask questions which are not in line with the day-to-day at all. Both vertically and horizontally. It creates dissatisfaction, burnout, bad cultures, sometimes even animosity, as if hiring managers are playing pranks on candidates.
If you're only looking at this from the perspective of "get someone who can do the job", you're going to miss the forest for the trees. As others say, the company is also being interviewed.
You're also maximizing the odds they will get bored and feel baited. With all due respect, you could have picked a better example to illustrate your point.
This is exactly why you can't make such bold claims. You're looking at this from one perspective without considering all the variables which go into the workplace equation. So far, the best correlator to job performance is giving people work corresponding to what they will be doing at the work place. For various reasons, this isn't really happening in software development, making people all to eager to grasp at proxies which they swear will do the same thing. Maybe whiteboarding is in line with the day-to-day at FAANGs, but tons of companies ask questions which are not in line with the day-to-day at all. Both vertically and horizontally. It creates dissatisfaction, burnout, bad cultures, sometimes even animosity, as if hiring managers are playing pranks on candidates.
If you're only looking at this from the perspective of "get someone who can do the job", you're going to miss the forest for the trees. As others say, the company is also being interviewed.
> You're also maximizing the odds they will get bored and feel baited.
I like this point. This is an especially big problem in my area (data science), in which people think they are being hired to do deep representation learning, and are subsequently mortified to learn that the job is principally about transpiling ”product manager” to ”SQL”. The cure is to be up front with job candidates. Tell them the job involves no deep learning. Explain that you give hard interview problems because you need to know that they can reason about eg asymptotic complexity on the rare times where it arises.
> So far, the best correlator to job performance is giving people work corresponding to what they will be doing at the work place.
I don’t necessarily disagree, but I believe you want to optimize for the hardest day rather than the average day. If your team is optimized for who can best solve an average problem, you will eventually have a problem that nobody can solve, and the business will suffer greatly as a result. Hence the discrete math problems.
I like this point. This is an especially big problem in my area (data science), in which people think they are being hired to do deep representation learning, and are subsequently mortified to learn that the job is principally about transpiling ”product manager” to ”SQL”. The cure is to be up front with job candidates. Tell them the job involves no deep learning. Explain that you give hard interview problems because you need to know that they can reason about eg asymptotic complexity on the rare times where it arises.
> So far, the best correlator to job performance is giving people work corresponding to what they will be doing at the work place.
I don’t necessarily disagree, but I believe you want to optimize for the hardest day rather than the average day. If your team is optimized for who can best solve an average problem, you will eventually have a problem that nobody can solve, and the business will suffer greatly as a result. Hence the discrete math problems.
> I believe you want to optimize for the hardest day rather than the average day.
I think this is a huge mistake. I work in data science, and nearly universally my colleagues are motivated by opportunities to learn. If you're testing for the very rare day, so that your employee has the right answer in their front pocket when that happens they are never going to feel challenged, feel like they're growing, and they're going to jump ship.
I think you're doing snobby filtering, and it's not to your benefit.
I think this is a huge mistake. I work in data science, and nearly universally my colleagues are motivated by opportunities to learn. If you're testing for the very rare day, so that your employee has the right answer in their front pocket when that happens they are never going to feel challenged, feel like they're growing, and they're going to jump ship.
I think you're doing snobby filtering, and it's not to your benefit.
> If you're testing for the very rare day, so that your employee has the right answer in their front pocket
On the hard days, the answer has never been seen before, and therefore can’t be in anyone’s pocket. You need to know that you have employees who are smart enough to discover it. Technical interviews are how you find them.
The observation that your employees are motivated by learning opportunities is useful for how you manage them, but this doesn’t tell us how to hire people who can do the job.
On the hard days, the answer has never been seen before, and therefore can’t be in anyone’s pocket. You need to know that you have employees who are smart enough to discover it. Technical interviews are how you find them.
The observation that your employees are motivated by learning opportunities is useful for how you manage them, but this doesn’t tell us how to hire people who can do the job.
>but I believe you want to optimize for the hardest day rather than the average day
There are a few problems with this. If your "hardest day" deviates far from your "average day", while being very infrequent, you're still not doing a good job enticing the candidate. Worse, by looking for people who have already acquired the skills to combat the worst you can throw at them, you're severely stagnating their growth.
Then comes the cherry on top, if your company is doing it while not providing absolutely stellar benefits to function as external motivation, you're further incentivizing people to work on ditching your place. Many companies do not give total comps in line with the skill level tested for, setting themselves up for failure in this regard. Altogether, you're still looking at this primarily from the job interviewer's perspective, not the perspective of the candidate which translates into the company's long term (lack of) health.
>you will eventually have a problem that nobody can solve
No. Someone has to be the first person to solve a question. This strategy doesn't take into account the capacity of individuals to learn and overcome challenges. What's more, most individuals will not be on the highest level, allowing mentors to train them to higher levels.
>I don’t necessarily disagree
It's not about disagreement. Schmidt and Hunter did a meta-analysis on this way back in 1998, and psychology has yet to find the silver bullet. Look at table 1[1]. Most studies checking the market find similar things. It is ridiculously difficult to find correlations between job performance and selection criteria. All corporates, including FAANG, are not exempt from making the very human mistake of doing something which seems great, but ends up being part of survivorship bias. In something as multi-faceted as the job market, combined with the young and everchanging nature of software development, making statements on correlations shouldn't be taken for granted.
Notice how the study above points out years of experience as one of the worst predictors. Most of the job market still uses YoE as if it is gospel. The market is nowhere near as rational as people like to believe it is.
[1]: https://www.researchgate.net/publication/232564809_The_Valid...
There are a few problems with this. If your "hardest day" deviates far from your "average day", while being very infrequent, you're still not doing a good job enticing the candidate. Worse, by looking for people who have already acquired the skills to combat the worst you can throw at them, you're severely stagnating their growth.
Then comes the cherry on top, if your company is doing it while not providing absolutely stellar benefits to function as external motivation, you're further incentivizing people to work on ditching your place. Many companies do not give total comps in line with the skill level tested for, setting themselves up for failure in this regard. Altogether, you're still looking at this primarily from the job interviewer's perspective, not the perspective of the candidate which translates into the company's long term (lack of) health.
>you will eventually have a problem that nobody can solve
No. Someone has to be the first person to solve a question. This strategy doesn't take into account the capacity of individuals to learn and overcome challenges. What's more, most individuals will not be on the highest level, allowing mentors to train them to higher levels.
>I don’t necessarily disagree
It's not about disagreement. Schmidt and Hunter did a meta-analysis on this way back in 1998, and psychology has yet to find the silver bullet. Look at table 1[1]. Most studies checking the market find similar things. It is ridiculously difficult to find correlations between job performance and selection criteria. All corporates, including FAANG, are not exempt from making the very human mistake of doing something which seems great, but ends up being part of survivorship bias. In something as multi-faceted as the job market, combined with the young and everchanging nature of software development, making statements on correlations shouldn't be taken for granted.
Notice how the study above points out years of experience as one of the worst predictors. Most of the job market still uses YoE as if it is gospel. The market is nowhere near as rational as people like to believe it is.
[1]: https://www.researchgate.net/publication/232564809_The_Valid...
> conduct an RCT in which you hire some percentage of people who fail the test as a control group
You might enjoy Work Rules by Laszlo Bock. They actually did do this at Google to validate their interview approach.
You might enjoy Work Rules by Laszlo Bock. They actually did do this at Google to validate their interview approach.
Lo and behold, they still do coding interviews. Who would have guessed that people who answer difficult questions correctly would outperform people who don’t.
In all seriousness, thank you for bringing this up - it’s an interesting story and I will probably look into it. But I also think it’s pretty obvious that coding interviews work extremely well, and any discussion of ”how can we create a procedure that is better for everyone” that doesn’t acknowledge this is hopeless.
In all seriousness, thank you for bringing this up - it’s an interesting story and I will probably look into it. But I also think it’s pretty obvious that coding interviews work extremely well, and any discussion of ”how can we create a procedure that is better for everyone” that doesn’t acknowledge this is hopeless.
> You don’t seem to have thought very hard about what employers are optimising for.
Have you ever seen a hiring process that's simply not effective?
The undercurrent of this irks me a bit. It assumes a kind of omniscience on part of the employer, and enough time for them to develop a good interview process. I think this is really hard to do.
I think you're generally right. And a lot of the time, employers are empathetic, pragmatic, self-aware and really are doing us a favor when they deny us. Some take into account I-O psychology stuff, too.
I (and others probably) are eager to hear what companies you feel do a good job at hiring.
> Everyone realises that programming interviews produce lots of false negatives.
Let's assume a bad scenario. A company can disregard evidence of competency, like having a portfolio, to basically have a rolling, year round tech Olympiad. To them, it's about fronting as a strong candidate in events, even at the expense of being good at the role.
You can be a pro at pole vault or discus throwing, but a poor carpenter. But there's a mentality out there - with some - that a strong, fit candidate can handle any technical challenge.
Can a chemist be a great chef? I bet, I also bet some could overthink and/or overengineer preparing a simple meal. Does the interview process do anything to check for soft skills, like curiosity, simplicity, etc? Hopefully.
> The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.
I think some companies - not all - can create a "squid game" contest out of candidates. Worse, I feel candidates can unduly suffer and be humiliated in these cases.
The company gets a superb interviewer, who is no doubt a very intelligent person, but not necessarily one that can adapt to technical reality, contingencies, ambiguities, etc. Some interview processes end up focusing on theoretical and rote knowledge, not integration and synthesis, when they deeply need that. Turnover happens.
But we're back to the beginning, did the company: 1. really know what they needed for that position, 2. interview process check for that? My guess is when you wrote that you were assuming situations where both were true?
Have you ever seen a hiring process that's simply not effective?
The undercurrent of this irks me a bit. It assumes a kind of omniscience on part of the employer, and enough time for them to develop a good interview process. I think this is really hard to do.
I think you're generally right. And a lot of the time, employers are empathetic, pragmatic, self-aware and really are doing us a favor when they deny us. Some take into account I-O psychology stuff, too.
I (and others probably) are eager to hear what companies you feel do a good job at hiring.
> Everyone realises that programming interviews produce lots of false negatives.
Let's assume a bad scenario. A company can disregard evidence of competency, like having a portfolio, to basically have a rolling, year round tech Olympiad. To them, it's about fronting as a strong candidate in events, even at the expense of being good at the role.
You can be a pro at pole vault or discus throwing, but a poor carpenter. But there's a mentality out there - with some - that a strong, fit candidate can handle any technical challenge.
Can a chemist be a great chef? I bet, I also bet some could overthink and/or overengineer preparing a simple meal. Does the interview process do anything to check for soft skills, like curiosity, simplicity, etc? Hopefully.
> The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.
I think some companies - not all - can create a "squid game" contest out of candidates. Worse, I feel candidates can unduly suffer and be humiliated in these cases.
The company gets a superb interviewer, who is no doubt a very intelligent person, but not necessarily one that can adapt to technical reality, contingencies, ambiguities, etc. Some interview processes end up focusing on theoretical and rote knowledge, not integration and synthesis, when they deeply need that. Turnover happens.
But we're back to the beginning, did the company: 1. really know what they needed for that position, 2. interview process check for that? My guess is when you wrote that you were assuming situations where both were true?
But the adversarial puzzles and whiteboard programming are not "the job".
Hire for strengths, not to avoid weaknesses.
Hire for strengths, not to avoid weaknesses.
> But the adversarial puzzles and whiteboard programming are not "the job".
This is true. They are closer to ”the rare, most intellectually difficult parts of the job.” A hockey player typically spends 70% of each game sitting on the bench; yet this isn’t what is emphasized in tryouts.
> Hire for strengths, not to avoid weaknesses.
I guess you didn’t read the part of my comment where I wrote:
1. ”Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.”
This is the default position of large tech companies who do coding interviews, and especially those like Google and Microsoft who interview with very difficult discrete math problems.
If we assume 1, the hiring practices of Google/Microsoft make perfect sense. They are not trying to hire all the extremely talented candidates. They are trying to ensure that all the hires are extremely talented. Put differently, they are optimising for precision rather than recall.
It seems like you disagree with 1. And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing.
But my default position is scepticism, because if it was possible it would probably already be happening.
This is true. They are closer to ”the rare, most intellectually difficult parts of the job.” A hockey player typically spends 70% of each game sitting on the bench; yet this isn’t what is emphasized in tryouts.
> Hire for strengths, not to avoid weaknesses.
I guess you didn’t read the part of my comment where I wrote:
1. ”Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.”
This is the default position of large tech companies who do coding interviews, and especially those like Google and Microsoft who interview with very difficult discrete math problems.
If we assume 1, the hiring practices of Google/Microsoft make perfect sense. They are not trying to hire all the extremely talented candidates. They are trying to ensure that all the hires are extremely talented. Put differently, they are optimising for precision rather than recall.
It seems like you disagree with 1. And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing.
But my default position is scepticism, because if it was possible it would probably already be happening.
> And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing.
Only if you view hiring as the competitive advantage of Google/Microsoft/Apple. But everyone cargo cults the hiring practices of Google/Microsoft/Apple without becoming Google/Microsoft/Apple, which shows that it's not actually some magical unique competitive advantage.
We can get into a discussion/argument about what is the competitive advantage of Google/Microsoft/Apple (which collectively own near 100% of the market for mobile OS, desktop OS, web browser, search, etc.), but that's kind of a tangent IMO.
I will say that Google/Microsoft/Apple have a competitive advantage specifically in hiring (that is, getting many of the top candidates to apply) because of their prestige (a product of the aforementioned market dominance) and vast wealth. The latter in particular allows them to offer more compensation, especially stock compensation, than other companies.
It's important to mention though that the formalized hiring practices of these companies are largely ex post facto and could only be put in place after they were already successful. It's unlikely they hired exactly in the same way during the early years. Steve Jobs was introduced to Steve Wozniack by a friend in high school! Nobody can plausibly claim that the success of the Apple II was due to "coding interviews". Larry Page and Sergey Brin met in college, where they were already working on the ideas that led to Google. Hiring doesn't just turn into success, that's not a reliable formula.
You can set the bar as high as you like, but you won't be able to hire anyone if candidates just walk away from your obstacle course. Talented people put themselves through a lot of crap to work at BigCos because there's a big payoff working for a BigCo. That's what tends to make the BigCo hiring pool better than for other companies, not the hiring bar itself. You can afford to be extremely picky if you have a wealth of options to pick from.
Only if you view hiring as the competitive advantage of Google/Microsoft/Apple. But everyone cargo cults the hiring practices of Google/Microsoft/Apple without becoming Google/Microsoft/Apple, which shows that it's not actually some magical unique competitive advantage.
We can get into a discussion/argument about what is the competitive advantage of Google/Microsoft/Apple (which collectively own near 100% of the market for mobile OS, desktop OS, web browser, search, etc.), but that's kind of a tangent IMO.
I will say that Google/Microsoft/Apple have a competitive advantage specifically in hiring (that is, getting many of the top candidates to apply) because of their prestige (a product of the aforementioned market dominance) and vast wealth. The latter in particular allows them to offer more compensation, especially stock compensation, than other companies.
It's important to mention though that the formalized hiring practices of these companies are largely ex post facto and could only be put in place after they were already successful. It's unlikely they hired exactly in the same way during the early years. Steve Jobs was introduced to Steve Wozniack by a friend in high school! Nobody can plausibly claim that the success of the Apple II was due to "coding interviews". Larry Page and Sergey Brin met in college, where they were already working on the ideas that led to Google. Hiring doesn't just turn into success, that's not a reliable formula.
You can set the bar as high as you like, but you won't be able to hire anyone if candidates just walk away from your obstacle course. Talented people put themselves through a lot of crap to work at BigCos because there's a big payoff working for a BigCo. That's what tends to make the BigCo hiring pool better than for other companies, not the hiring bar itself. You can afford to be extremely picky if you have a wealth of options to pick from.
> Only if you view hiring as the competitive advantage of Google/Microsoft/Apple. But everyone cargo cults the hiring practices of Google/Microsoft/Apple without becoming Google/Microsoft/Apple, which shows that it's not actually some magical unique competitive advantage.
You're conflating some things here. Without unpicking it all, the main thing I'd say is that you've only demonstrated that amazing hiring is not sufficient for competitive advantage. You have not demonstrated that it's not necessary.
You're conflating some things here. Without unpicking it all, the main thing I'd say is that you've only demonstrated that amazing hiring is not sufficient for competitive advantage. You have not demonstrated that it's not necessary.
I wouldn't describe it as "amazing" hiring. I'd describe it as assembly line hiring. The results are amazing: Google/Microsoft/Apple are 3 out of the 4 largest companies in the world by market cap. And there's a hiring process — though I doubt the process has remained exactly the same from company founding to present. The question is how much if any the specific hiring process contributes to the results.
I'd say there's a big difference between a company finding engineers and engineers throwing themselves at a company. The latter is what often occurs with BigCos. They have a big pool of candidates throwing themselves at the company, and the company gets to pick whomever they like from the pool. How much credit to due to the company for picking? Whereas I think you'd give a company more credit for going out and specifically recruiting/poaching one person they want, even if that person wasn't applying for a job with them.
I'd say there's a big difference between a company finding engineers and engineers throwing themselves at a company. The latter is what often occurs with BigCos. They have a big pool of candidates throwing themselves at the company, and the company gets to pick whomever they like from the pool. How much credit to due to the company for picking? Whereas I think you'd give a company more credit for going out and specifically recruiting/poaching one person they want, even if that person wasn't applying for a job with them.
> How much credit to due to the company for picking?
Lots. Firstly for becoming a place people want to work. Secondly for sifting through all of those people.
Lots. Firstly for becoming a place people want to work. Secondly for sifting through all of those people.
So how do I effectively test for the strengths I want with a low likelihood of false positives?
Ideally without an increase of time/cost requirements.
Ideally without an increase of time/cost requirements.
Maybe you're the enlightened one here or maybe you're just rationalizing an awful and lazy part of the hiring process for many companies.
If anyone actually believed this they wouldn’t complain about it on HackerNews. If it were true that Google, Microsoft, Apple, etc don’t know how to hire programmers effectively, this would be an incredibly powerful observation which you would want to keep top secret while silently beating the hiring market and building the most successful tech company of all time. We’re all waiting.
The reason people are upset by this issue is that it highlights a misalignment in the incentives of employers and job seekers. And with this I fully empathize: job hunting is hell in any industry, ours included.
But we can’t begin to have a conversation about improving the situation until we have a rudimentary understanding of what is happening today. And all these ”Google doesn’t know how to hire smart people” takes are brutally handicapping the discourse.
The reason people are upset by this issue is that it highlights a misalignment in the incentives of employers and job seekers. And with this I fully empathize: job hunting is hell in any industry, ours included.
But we can’t begin to have a conversation about improving the situation until we have a rudimentary understanding of what is happening today. And all these ”Google doesn’t know how to hire smart people” takes are brutally handicapping the discourse.
> If it were true that Google, Microsoft, Apple, etc don’t know how to hire programmers effectively, this would be an incredibly powerful observation which you would want to keep top secret while silently beating the hiring market and building the most successful tech company of all time.
Well, I'd like to have a word with the Apple Mac Mail app team. And the Disk Utility team. And some others...
Well, I'd like to have a word with the Apple Mac Mail app team. And the Disk Utility team. And some others...
> or asking obnoxious questions ("Why did you use a foreach and not a for loop?")
you should be able to explain why you wrote something the way you did; that's not an absurd ask.
perf characteristics of different iteration styles are often significant to the task at hand.
you should be able to explain why you wrote something the way you did; that's not an absurd ask.
perf characteristics of different iteration styles are often significant to the task at hand.
Your problem has gone off the rails if that level of performance difference is important.
For me it wouldn't be about performance but about readability.
try rendering a dynamic React dashboard with 30 line, scatter, and heatmap charts, each containing 3 series of 10k points each, several times per second. welcome to what Grafana can do :)
If someone suggested that I ought to be able to solve a problem like that in an interview, rather than solving it over the course of several years with a team of developers (like Grafana did) then I'd stop the interview and remove myself from the hiring process.
No doubt there are situations where hugely optimized code is necessary. An interview isn't one of them unless you're hiring someone to write exactly that code. Most companies aren't.
No doubt there are situations where hugely optimized code is necessary. An interview isn't one of them unless you're hiring someone to write exactly that code. Most companies aren't.
> No doubt there are situations where hugely optimized code is necessary. An interview isn't one of them unless you're hiring someone to write exactly that code. Most companies aren't.
knowing the difference, and consciously deciding that it doesn't matter for a given task is often a strong positive signal. hopefully a good interviewer will not actually ask you this question if it doesnt matter for the given task.
knowing the difference, and consciously deciding that it doesn't matter for a given task is often a strong positive signal. hopefully a good interviewer will not actually ask you this question if it doesnt matter for the given task.
I don't agree. If someone brought that up in an interview I'd just think "oh, they know a bit of JS trivia that won't impact any of the code they'd write here." and ignore it. It's not a signal at all, let alone a strong one.
If they mentioned the fact that React 18 batches updates in concurrent mode which would greatly improve the responsiveness of a dashboard app that's updating lots of small components frequently, or if they suggested the graphs might be better written using GLSL shaders to move the rendering to the GPU so JS only needs to update a uniform array, then I'd pay attention. Potential new ways to approach a problem or changes to architecture to avoid the problem entirely rather than micro-optimizations are much more interesting signals about someone's deeper knowledge of building things.
If they mentioned the fact that React 18 batches updates in concurrent mode which would greatly improve the responsiveness of a dashboard app that's updating lots of small components frequently, or if they suggested the graphs might be better written using GLSL shaders to move the rendering to the GPU so JS only needs to update a uniform array, then I'd pay attention. Potential new ways to approach a problem or changes to architecture to avoid the problem entirely rather than micro-optimizations are much more interesting signals about someone's deeper knowledge of building things.
[deleted]
Yes, a regular for loop is a bit faster at iterating over a large list of elements than foreach in Javascript, but why on earth would you care about that kind of optimization minutia in a coding interview?
The question you're asking involves 10k * 3 * 30 = 900,000 data points. If a candidate wants to simply loop over all of them multiple times a second, I'd say that's a much bigger problem.
The question you're asking involves 10k * 3 * 30 = 900,000 data points. If a candidate wants to simply loop over all of them multiple times a second, I'd say that's a much bigger problem.
> If a candidate want to simply loop over all of them multiple times a second, I'd say that's a much bigger problem.
unless the task is to animate them in a rAF loop (which runs at ~60Hz), right? the task matters. in some cases the question is stupid & irrelevant, and in the other case it means getting the job done, or not.
the difference between the brain-dead way and the fast way is literally 30x (53ms vs 1.8ms on my machine): https://jsfiddle.net/3ho6f5k1/
unless the task is to animate them in a rAF loop (which runs at ~60Hz), right? the task matters. in some cases the question is stupid & irrelevant, and in the other case it means getting the job done, or not.
the difference between the brain-dead way and the fast way is literally 30x (53ms vs 1.8ms on my machine): https://jsfiddle.net/3ho6f5k1/
It’s unrealistic to expect this level of optimization in a generalist whiteboard session. For a specialist, maybe. I’m not specialized in this area.
For an array of 5-10 items the performance hit of foreach is worth it for readability. Maybe an interesting discussion could occur about when to sacrifice readability for performance but you won’t get that by grilling people about minutiae.
For an array of 5-10 items the performance hit of foreach is worth it for readability. Maybe an interesting discussion could occur about when to sacrifice readability for performance but you won’t get that by grilling people about minutiae.
> For an array of 5-10 items the performance hit of foreach is worth it for readability
As always it depends, maybe that component is being called 500 times on the page making it actually 5,000 invocations.
I'd FULLY expect a candidate to understand the difference and would ask in a whiteboard session. There is a lot of bloated slow javascript libraries out there and they almost always stem from the fact that most devs don't realize how much overhead function calls introduce.
For the interview, there is no discussion necessary, the correct answer is simply:
"I used a foreach for readability, because performance doesn't matter in this case because X."
or
"I used a for because performance matters in this case because Y"
End of story.
As always it depends, maybe that component is being called 500 times on the page making it actually 5,000 invocations.
I'd FULLY expect a candidate to understand the difference and would ask in a whiteboard session. There is a lot of bloated slow javascript libraries out there and they almost always stem from the fact that most devs don't realize how much overhead function calls introduce.
For the interview, there is no discussion necessary, the correct answer is simply:
"I used a foreach for readability, because performance doesn't matter in this case because X."
or
"I used a for because performance matters in this case because Y"
End of story.
one could argue that it's a job for the compiler to make sure two equivalent language constructs (map and for loop) perform similarly...
Foreach with functions being passed being slower is an implementation detail. You don't see it in Rust, where the block body is inlined like a generic, and it's perfectly feasible for a dynamic runtime to specialize the pattern and inline based on it being a common practice, never mind polymorphic inline caches which in principle permit inlining in this case.
> For an array of 5-10 items the performance hit of foreach is worth it for readability
100% agree. my point is more about OP's example being used to prove that dumb questions get asked. if it's being asked by a good interviewer, it's usually relevant.
100% agree. my point is more about OP's example being used to prove that dumb questions get asked. if it's being asked by a good interviewer, it's usually relevant.
Not sure it proves anything in interview context though. It's a language specific trivia you'll quickly learn either from experience or from a good mentor. Proves nothing about your ability to code or to be a successful member in the team. Unless you're actively looking for a candidate that knows all those JS optimization tricks.
Take Rust. For each and map generally are faster because they often allow bound checks to be elided.
100% trivia.
100% trivia.
Constant factors don't normally matter in an interview algorithm context. Focus on them often results in ugly code and can be a distraction against bigger improvements.
I would care about the reading optimization. So if someone tells me "I chose a forEach instead of a for because it provides almost the same performance, but is more readable, and my code does not need the index", that's a very good sign of someone making conscious choices.
If you can't give me the o(n) performance of the algorithm that you just wrote, this is a pretty serious red flag about your ability to understand what you are writing.
I'm going to blow your mind but a ton of important software was written by people who have a hard time explaining these things.
Also a ton of important software was written as a giant mess of spaghetti code, or with horrible security vulnerabilities.
I've never been tested on my ability to not write spaghetti code or write security vulnerabilities in my 15 years doing many many interviews.
Unless you're suggesting that knowing the performance of a loop helps with that?
Unless you're suggesting that knowing the performance of a loop helps with that?
What I'm suggesting is that "Important software has been written by people who couldn't/didn't do X" is not a very strong indication that X is not a valuable quality that employers should want in their engineers.
Except for the fact that using foreach instead of for has no impact on your O(n) performance.
What would you do if a candidate said with a smile "I'm pretty sure it is O(n!)"?
[deleted]
Parent commenter asked for little o, not for big O. So to be safe, better say n^o(n^2).
Hah this is the best comment.
“So you want to put code into production without having a better estimation of its time complexity than O(n!)?”
If the choice is relevant to the task, then sure. If it’s an arbitrary choice then the question is meaningless. It’s like asking “why are you wearing a blue shirt?” when you would have asked “why are you wearing a black shirt?” had they been wearing a black shirt.
i'm not claiming that this is a good question for every case. but there are many times where the choice of loop makes a significant difference. asking this question for a task that only requires iterating 3 elements every 1000ms would indeed be quite stupid on the interviewer's part.
it always amuses me when people use these seemingly stupid questions (out of context) to prove how broken tech interviews are. but of course, if your job will be write high perf code, then suddenly they question is incredibly relevant.
to be clear, i agree that there are a lot of stupid questions that get asked which are not relevant to the actual job you'll be doing (exceedingly few SWEs will need to implement a merge sort as part of their job)
it always amuses me when people use these seemingly stupid questions (out of context) to prove how broken tech interviews are. but of course, if your job will be write high perf code, then suddenly they question is incredibly relevant.
to be clear, i agree that there are a lot of stupid questions that get asked which are not relevant to the actual job you'll be doing (exceedingly few SWEs will need to implement a merge sort as part of their job)
> perf characteristics of different iteration styles are often significant to the task at hand.
This ridiculously untrue.
It's also a great example of how dumb current interviews are. Someone like this poster could easily interview you, and now what? Do you just play along or begin to explain how incredibly wrong they are?
This ridiculously untrue.
It's also a great example of how dumb current interviews are. Someone like this poster could easily interview you, and now what? Do you just play along or begin to explain how incredibly wrong they are?
This is untrue in languages with proper optimizing compilers, designed for performance, e.g. C++, Rust, Zig. A functional iterator transformation chain is often just the same performance as a "traditional" for loop.
But try that in Java and you'll often see a 3x-10x difference, especially if you run into problems with boxing. Streams are known to be notoriously slower than for loops with index. I once have even doubled the speed of a for loop by unrolling it manually (Java 8). Not all compilers are equally strong at optimization.
Anyway, asking such questions in an interview for a junior/regular developer is just a distraction.
But try that in Java and you'll often see a 3x-10x difference, especially if you run into problems with boxing. Streams are known to be notoriously slower than for loops with index. I once have even doubled the speed of a for loop by unrolling it manually (Java 8). Not all compilers are equally strong at optimization.
Anyway, asking such questions in an interview for a junior/regular developer is just a distraction.
Even in java/c#: what kind of application has its performance dominated by a control structure? If you're doing so little work inside the loop then you should probably look at using fancy cpu instructions like SIMD
> This ridiculously untrue.
at least for JS VMs, this is true today. this answer will certainly vary for other languages/runtimes.
i profile and optimize a lot of JS/TS code that handles datasets with millions of datapoints at my day job. but you don't need to take my word for it; this claim is not exactly difficult to verify.
> Do you just play along or begin to explain how incredibly wrong they are?
in this case, i'd prefer to be proven wrong with code, rather than prose.
at least for JS VMs, this is true today. this answer will certainly vary for other languages/runtimes.
i profile and optimize a lot of JS/TS code that handles datasets with millions of datapoints at my day job. but you don't need to take my word for it; this claim is not exactly difficult to verify.
> Do you just play along or begin to explain how incredibly wrong they are?
in this case, i'd prefer to be proven wrong with code, rather than prose.
It's probably time to move into a faster language if you're getting caught up in language specifics on loop execution.
quite impossible to move into a faster language when it would
1. require a complete rewrite of an existing, massive codebase
2. where the primary task is rendering interactive charts for a React web app
1. require a complete rewrite of an existing, massive codebase
2. where the primary task is rendering interactive charts for a React web app
> at least for JS VMs, this is true today. this answer will certainly vary for other languages/runtimes.
Total nonsense. It is not true of JS today, not has it ever been true.
No modern web app's or website's performance is dominated by the performance of what loops you choose. It isn't even remotely meaningfully impacted by that; by this I mean, any differences are totally unmeasurable beyond some 3 line micro-benchmark.
You have never benchmarked or profiled your code.
Of course, tight inner loops for image processing or ML, etc. yes, this can make a difference. But we're only talking about a handful of loops that matter out of many tens of thousands an application may have, you are almost certainly never writing those loops, and that's a specialized field that basically has nothing to do with the web. And it isn't what you're talking about.
> in this case, i'd prefer to be proven wrong with code, rather than prose.
What absurdity. And this is exactly when interviews suck. Because people have strongly held beliefs that are ungrounded in reality, and since there are no objective measures by which to rate interviews you need to pretend that they're real, because a lot of interviewers are basically just testing "cultural fit".
But hey! I have to say. That if you brought this up in an interview. I would at least know to never take the job!
Total nonsense. It is not true of JS today, not has it ever been true.
No modern web app's or website's performance is dominated by the performance of what loops you choose. It isn't even remotely meaningfully impacted by that; by this I mean, any differences are totally unmeasurable beyond some 3 line micro-benchmark.
You have never benchmarked or profiled your code.
Of course, tight inner loops for image processing or ML, etc. yes, this can make a difference. But we're only talking about a handful of loops that matter out of many tens of thousands an application may have, you are almost certainly never writing those loops, and that's a specialized field that basically has nothing to do with the web. And it isn't what you're talking about.
> in this case, i'd prefer to be proven wrong with code, rather than prose.
What absurdity. And this is exactly when interviews suck. Because people have strongly held beliefs that are ungrounded in reality, and since there are no objective measures by which to rate interviews you need to pretend that they're real, because a lot of interviewers are basically just testing "cultural fit".
But hey! I have to say. That if you brought this up in an interview. I would at least know to never take the job!
> "Do coding interviews work in the actual world in which we live in?" No, fundamentally not.
See, that's just not right. I mean, we all agree there's a hard problem to solve here, right? The world is populated by engineers of widely varying skill levels. It just is.
So how do you detect them? Well, the ability to bang out a correct solution on a whiteboard is a verifiably correct way to do that. It may not be optimal (passing candidates with other issues that make them bad fits), and it may have undesired failure modes (rejecting people who have trouble performing in the high pressure environment). And those are problems worth discussing.
But... it works. It clearly works. We've all seen it work. Companies that use these techniques aggressively tend strongly to be organizations with a long track record of producing working software. The FAANG employers are desirable because FAANGs ship code and make money!
So I just don't get the absolutism here. If there's a clearly better way that produces a more effective workforce, where's the darwinian example? Who's winning by hiring better people than Apple or Meta? I just don't see it.
See, that's just not right. I mean, we all agree there's a hard problem to solve here, right? The world is populated by engineers of widely varying skill levels. It just is.
So how do you detect them? Well, the ability to bang out a correct solution on a whiteboard is a verifiably correct way to do that. It may not be optimal (passing candidates with other issues that make them bad fits), and it may have undesired failure modes (rejecting people who have trouble performing in the high pressure environment). And those are problems worth discussing.
But... it works. It clearly works. We've all seen it work. Companies that use these techniques aggressively tend strongly to be organizations with a long track record of producing working software. The FAANG employers are desirable because FAANGs ship code and make money!
So I just don't get the absolutism here. If there's a clearly better way that produces a more effective workforce, where's the darwinian example? Who's winning by hiring better people than Apple or Meta? I just don't see it.
And yet correlation is not causation. And a survivor bias is probably in play here. There are so many more factors contributing to the quality of FAANG work and their performance as businesses. Hell, most of them due to their monopoly-like status could easily use any hiring technique and still rake in tons of $$$.
To me the streamlined / cargo cult hiring process seems much more a symptoms of the bureaucratization of these large companies than an actual key to their success.
> And yet correlation is not causation
Well... yeah. To pedantically elaborate that, clearly the hypothesis is that the measured traits (doing well on a coding interview and doing well at coding) are co-causal, and that measuring one (which is comparatively cheap to do) is a useful proxy for measuring the other (extremely expensive!).
That's a really compelling hypothesis. The counter point requires, to my eyes, a clear working example: a company with a uniform hiring process involving some other technique which has a track record of product development as robust as all the existing leaders. And I don't see one. Calling them monopolies or cargo cults seems to be filling in for evidence, and I don't see that as persuasive.
Well... yeah. To pedantically elaborate that, clearly the hypothesis is that the measured traits (doing well on a coding interview and doing well at coding) are co-causal, and that measuring one (which is comparatively cheap to do) is a useful proxy for measuring the other (extremely expensive!).
That's a really compelling hypothesis. The counter point requires, to my eyes, a clear working example: a company with a uniform hiring process involving some other technique which has a track record of product development as robust as all the existing leaders. And I don't see one. Calling them monopolies or cargo cults seems to be filling in for evidence, and I don't see that as persuasive.
What do you mean by "robust"? There are only a handful of monopolists in the world (by definition), so in that respect no other comparable companies exist.
But there are plenty of smaller companies who do well for themselves, relatively speaking, without cargo culting on BigCo hiring. And there are plenty of companies who cargo cult on BigCo hiring but who have not become a market leader. In fact there are graveyards full of failed startups who cargo culted BigCo hiring. So the hiring process doesn't clearly explain the financial results of the BigCos.
But there are plenty of smaller companies who do well for themselves, relatively speaking, without cargo culting on BigCo hiring. And there are plenty of companies who cargo cult on BigCo hiring but who have not become a market leader. In fact there are graveyards full of failed startups who cargo culted BigCo hiring. So the hiring process doesn't clearly explain the financial results of the BigCos.
There are plenty of large companies with established hiring practices and standardized interview schemes out there that don't do FAANG-style coding interviews, though. And on the whole they don't do well at delivering software, and have a reputation as terrible places to do software work.
And on the other side: are there major employers who implement rigorous coding interviews who... don't have a history of producing good software? I really can't think of any.
That's my point: the evidence is really stacked in favor of the FAANG scheme. So pointing to a handful of small outfits doesn't really sway my impression; those are just expected outliers. I've worked at smaller companies with terrible hiring success, FWIW; they ended up successful because of a few lucky hires.
And on the other side: are there major employers who implement rigorous coding interviews who... don't have a history of producing good software? I really can't think of any.
That's my point: the evidence is really stacked in favor of the FAANG scheme. So pointing to a handful of small outfits doesn't really sway my impression; those are just expected outliers. I've worked at smaller companies with terrible hiring success, FWIW; they ended up successful because of a few lucky hires.
It's difficult to agree or disagree with this comment, because it's so vague. For example, "on the whole they don't do well at delivering software, and have a reputation as terrible places to do software work." Are we supposed to just accept this claim without question? I have no idea which companies you're talking about. And I'm not sure that Apple and Amazon for example don't have reputations as terrible places to do software work, but again I'm not entirely clear on what you mean by that.
Moreover, "good software" is also highly subjective and subject to dispute. I feel that the software coming from Apple and Google has gone downhill in the past decade. Are we to explain that by hiring?
The financial numbers are indisputable for Apple, Microsoft, Google, Amazon, and Facebook. (Not sure the term FAANG even makes sense anymore.) But something like "reputation" is another matter entirely.
Moreover, "good software" is also highly subjective and subject to dispute. I feel that the software coming from Apple and Google has gone downhill in the past decade. Are we to explain that by hiring?
The financial numbers are indisputable for Apple, Microsoft, Google, Amazon, and Facebook. (Not sure the term FAANG even makes sense anymore.) But something like "reputation" is another matter entirely.
> There are only a handful of monopolists in the world (by definition)
I don't think that's right. If a small town only has one gas station, and the next closest is 100 miles away, isn't that gas station owner a monopolist?
I don't think that's right. If a small town only has one gas station, and the next closest is 100 miles away, isn't that gas station owner a monopolist?
Fun part is that it can work like ideal world when you have 1-2 devs per year to hire and as a small company you get 20-30 CVs a month.
Once you get into scale where you need to hire 20 devs a year and get 100s of CVs per week it all breaks down.
Once you get into scale where you need to hire 20 devs a year and get 100s of CVs per week it all breaks down.
I've been at companies where we receive hundreds of applications per role posted, and one where we receive 10s. I've posted about this before but in both cases the pipeline was the same - initial pass eliminates about 70-80% of the candidates because they're just not worth following up on - think things like IT staff applying for mid level development jobs with no experience, education or anything to indicate they could write any code. Of the remaining we end up filtering out based on comparing applications - this bit sucks, but pretty much stack ranking the resumes and picking out the top 20 or so[0]. At that point, everyone gets a phone screen with a set of questions picked from a list and we proceed through that. The same process works mostly because the size of your interviewing group scales with the number of engineers you hire.
[0] side note - I work in games where we're often filtering resumes for specific roles, e.g. a gameplay programmer or an online programmer. The filtering here is most of the time bucketing candidates based on their experience and sorting that way.
[0] side note - I work in games where we're often filtering resumes for specific roles, e.g. a gameplay programmer or an online programmer. The filtering here is most of the time bucketing candidates based on their experience and sorting that way.
I do a lot of interviews with potential candidates, and worked with plenty of those we hired (and unfortunately some we fired afterwards).
If you have a better way on how to assess a software developer, I would love to hear it.
If you have a better way on how to assess a software developer, I would love to hear it.
Not OP, but I would love to share some: I relate strongly to the anxiety induced descriptions in OPs text and thus try to avoid these when I interview devs as an engineering manager. Of all the candidates I have interviewed over the years I have never done a live coding interview because it is essentially worthless for me as an interviewer and only serves to discards potentially very clever people whose thought process does not work this way.
Instead I try to find an isolated problem/task we have in our backlog that the candidate should try to implement on their own time, so I can review what kind of solution they will actually deliver once hired. We then go through the solution with them and do a code review as they would go through if they got hired. You then see both the problem-solving skills as well as how they take feedback.
I would recommend checking out DuckDuckGo's excellent "How we hire" guide[0] which describes some of the best hiring process I have come across (albeit it may be too extensive to do in full for some companies).
[0] https://duckduckgo.com/assets/hiring/how_we_hire.pdf
Instead I try to find an isolated problem/task we have in our backlog that the candidate should try to implement on their own time, so I can review what kind of solution they will actually deliver once hired. We then go through the solution with them and do a code review as they would go through if they got hired. You then see both the problem-solving skills as well as how they take feedback.
I would recommend checking out DuckDuckGo's excellent "How we hire" guide[0] which describes some of the best hiring process I have come across (albeit it may be too extensive to do in full for some companies).
[0] https://duckduckgo.com/assets/hiring/how_we_hire.pdf
So how do you know they did it on their own? Spouse is a developer, helps them out, explains everything. Then on the job there is no spouse, and it all goes to shit. Not saying this would be every time, but you might come across one of these.
We have an initial Hacker Rank 'filter', that if you don't pass that, we don't even do an interview. A few weeks ago we interviewed someone who passed that filter very well. But in the interview, it was really below par. Not even the basics were understood. So I'm very skeptical about these kinds of homework where they could get help.
What I currently do is go through a very simple coding example that I wrote (very generic). Then we go through the code and I ask questions like "can you explain what this line is doing", or "If I move this line over here, is the behavior still the same", etc. Some bugs are in there and I ask what is going wrong and how to solve them.
Not ideal, but the problem is that nothing really is :(.
We have an initial Hacker Rank 'filter', that if you don't pass that, we don't even do an interview. A few weeks ago we interviewed someone who passed that filter very well. But in the interview, it was really below par. Not even the basics were understood. So I'm very skeptical about these kinds of homework where they could get help.
What I currently do is go through a very simple coding example that I wrote (very generic). Then we go through the code and I ask questions like "can you explain what this line is doing", or "If I move this line over here, is the behavior still the same", etc. Some bugs are in there and I ask what is going wrong and how to solve them.
Not ideal, but the problem is that nothing really is :(.
I think this person has a better idea because your spouse scenario, while realistic, can easily be detected while you discuss the code being presented.
You may have good intentions but what do you equates to "I don't trust you, bro/sis". Not a great way to start a relationship imho
You may have good intentions but what do you equates to "I don't trust you, bro/sis". Not a great way to start a relationship imho
> can easily be detected while you discuss the code being presented.
That is possible of course. But if they wrote the code together, and know they will question later, you could teach the other person all the trade-offs etc. I could see myself doing that with a bad developer and get them through such an interview.
> Not a great way to start a relationship imho
Well, you could take the benefit of the doubt, and then when it seems they did cheat, just fire them. But firing is always such a big decision.
It's a difficult problem.
That is possible of course. But if they wrote the code together, and know they will question later, you could teach the other person all the trade-offs etc. I could see myself doing that with a bad developer and get them through such an interview.
> Not a great way to start a relationship imho
Well, you could take the benefit of the doubt, and then when it seems they did cheat, just fire them. But firing is always such a big decision.
It's a difficult problem.
> But firing is always such a big decision.
Isn't that why managers get paid the big dollars, to make the big decisions?
The whole rhetoric of the tech industry — taking risks, creative destruction, etc. — seems to completely fall apart when it comes to hiring, where everyone turns fearful and ultraconservative.
Isn't that why managers get paid the big dollars, to make the big decisions?
The whole rhetoric of the tech industry — taking risks, creative destruction, etc. — seems to completely fall apart when it comes to hiring, where everyone turns fearful and ultraconservative.
Have you ever worked with a really bad hire? Someone totally incompetent at doing the job? In my personal experience, that person completely tanks the productivity and moral of the entire team. I’ll also point out you are not doing that person any favors by hiring them and firing them one month later (if your org is even capable of doing that). Bad hires are disastrous.
> In my personal experience, that person completely tanks the productivity and moral of the entire team.
A bad manager completely tanks the productivity and morale of the entire team too. And you can't fire your boss, you can only quit your job.
> I’ll also point out you are not doing that person any favors by hiring them and firing them one month later (if your org is even capable of doing that).
Not sure how it's relevant whether you're doing the incompetent person a favor or not? Hiring isn't about favors, it's an exchange of money for work. If there's no useful work, then the money has to stop.
> Bad hires are disastrous.
Seems a bit overstated to me. Do you have any examples of companies put out of business by a bad programmer hire?
Coders come and go. Is it a "disaster" if your best coder leaves for another job? We're treating hiring as if it were like marriage and divorce, but it's not. When you hire someone, it's not "til death do us part". Breakups are expected.
Too many companies make hiring much more difficult than it needs to be. They overthink it, and also waste way too much time on it. Find someone who seems right for the job, and hire them. If they don't work out, get rid of them and hire someone else. You might get unlucky at times with a bad hire, but you might also get lucky with an unexpectedly great hire. If you can't correct your mistakes quickly, that's a problem with your company's culture, not a problem with job candidates. I would argue that it's not a bad hire that demoralizes a team but rather a bad company culture that's incapable of correcting mistakes.
A bad manager completely tanks the productivity and morale of the entire team too. And you can't fire your boss, you can only quit your job.
> I’ll also point out you are not doing that person any favors by hiring them and firing them one month later (if your org is even capable of doing that).
Not sure how it's relevant whether you're doing the incompetent person a favor or not? Hiring isn't about favors, it's an exchange of money for work. If there's no useful work, then the money has to stop.
> Bad hires are disastrous.
Seems a bit overstated to me. Do you have any examples of companies put out of business by a bad programmer hire?
Coders come and go. Is it a "disaster" if your best coder leaves for another job? We're treating hiring as if it were like marriage and divorce, but it's not. When you hire someone, it's not "til death do us part". Breakups are expected.
Too many companies make hiring much more difficult than it needs to be. They overthink it, and also waste way too much time on it. Find someone who seems right for the job, and hire them. If they don't work out, get rid of them and hire someone else. You might get unlucky at times with a bad hire, but you might also get lucky with an unexpectedly great hire. If you can't correct your mistakes quickly, that's a problem with your company's culture, not a problem with job candidates. I would argue that it's not a bad hire that demoralizes a team but rather a bad company culture that's incapable of correcting mistakes.
You're answering in a vacuum. The previous poster is saying that just hiring easily with a view to firing if they're bad is destructive. Talking about managers is irrelevant to this. It's not one or the other.
This, probation time + take home assignment + discussion = I can filter out almost all the false negatives of the take at home. Give the people a choice between take home assignment and more classic code interview so you can dodge problems like shitty home-offices, strange contracts clauses but the real deal remains the probation time.
> Not saying this would be every time
How often would you say it ever happens? This feels like a rather extreme level of paranoia.
How often would you say it ever happens? This feels like a rather extreme level of paranoia.
> Instead I try to find an isolated problem/task we have in our backlog that the candidate should try to implement on their own time,
And in this case you're eliminating people who aren't willing and/or able to spend time preparing for your interview process. One of my previous jobs had a clause in the contract that all my programming work belonged to my employer; is your company going to sign a waiver that says they take responsibility if a previous employer sues? Doesn't matter whether or not it's enforceable or not.
What if you use that solution in production, do you compensate for their time in helping solve the problem like you would if the task was assigned to you?
Also, you're not really setting a level playing field here. How do you gauge a candidates experience/seniority versus the other 10 candidates you hired if you're not asking them the same questions or looking for the same things. Maybe I've solved that bug in my current job.
All this is to say that this doesn't "just fix" hiring - I don't know if it's any better or worse than giving people on-site tests, but you're at the very least trading one set of problems for another.
And in this case you're eliminating people who aren't willing and/or able to spend time preparing for your interview process. One of my previous jobs had a clause in the contract that all my programming work belonged to my employer; is your company going to sign a waiver that says they take responsibility if a previous employer sues? Doesn't matter whether or not it's enforceable or not.
What if you use that solution in production, do you compensate for their time in helping solve the problem like you would if the task was assigned to you?
Also, you're not really setting a level playing field here. How do you gauge a candidates experience/seniority versus the other 10 candidates you hired if you're not asking them the same questions or looking for the same things. Maybe I've solved that bug in my current job.
All this is to say that this doesn't "just fix" hiring - I don't know if it's any better or worse than giving people on-site tests, but you're at the very least trading one set of problems for another.
It might not "just fix" hiring, but at least it doesn't blinding accept the status quo and attempt to engineer an alternative that can then be evaluated. We are supposed to be scientists, are we not?
This is really not rocket science. We have implemented in our company.
Step 1: An initial phone interview with the candidate regarding technologies of choice, war stories, etc
Step 2: A small coding exercise that is similar to what we work on. The candidate can solve this on their own time and email us the solution.
Step 3: Review the solution together and ask questions about their methods.
Step 4: Meet the boss & offer.
The coding exercise is only sent _after_ the initial phone call, so we don't waste their time or ours.
The only problem here is that we can't virtue signal that we follow FANG whiteboard interviews and pretend that everyone working at the company can invert a Trie Tree in a 30 minute coding session..
Step 1: An initial phone interview with the candidate regarding technologies of choice, war stories, etc
Step 2: A small coding exercise that is similar to what we work on. The candidate can solve this on their own time and email us the solution.
Step 3: Review the solution together and ask questions about their methods.
Step 4: Meet the boss & offer.
The coding exercise is only sent _after_ the initial phone call, so we don't waste their time or ours.
The only problem here is that we can't virtue signal that we follow FANG whiteboard interviews and pretend that everyone working at the company can invert a Trie Tree in a 30 minute coding session..
I would like to see the hiring process like this.
1st interview - Go over the role etc, both parties ask question to see if they are a good fit. If both parties are happy to move to the next stage then there is a take home exercise to do.
Tech Exercise - Something simple which touches the core competencies that the role requires.
2 Second interview - Go through the exercise discussing the design choices etc. If everyone is happy offer the job.
I think the probation period should be used to determine if the candidate is the right fit for the role. That way the business gets a much better idea of who it is they are hiring and the same goes for the candidate.
1st interview - Go over the role etc, both parties ask question to see if they are a good fit. If both parties are happy to move to the next stage then there is a take home exercise to do.
Tech Exercise - Something simple which touches the core competencies that the role requires.
2 Second interview - Go through the exercise discussing the design choices etc. If everyone is happy offer the job.
I think the probation period should be used to determine if the candidate is the right fit for the role. That way the business gets a much better idea of who it is they are hiring and the same goes for the candidate.
My favourite is "I'll be your google, I'll be your Stack Overflow, when you would usually search something, ask me instead". So you eventually ask them, and they give you a cryptic answer back.
I'm sure Google wouldnt be as successful as they are if their answer to "how do you define an array" was "well think about what you're trying to do here"
I'm sure Google wouldnt be as successful as they are if their answer to "how do you define an array" was "well think about what you're trying to do here"
This! Thank you for writing it. And let’s not forget about the wasted nights ruminating about the poor performance and putting up with people who get their dopamine when they are given authority over a peer. Most developers don’t study psychology and don’t have the ability or interest of creating a safe space for conducting the kind of interviews that their employer demands.
I can't tell if you think companies should hire everyone or hire no-one.
What’s the alternative?
If we have to give Leetcode questions I'd at least like there to be an option to do it by myself, no one watching me type or asking questions if I'm thinking more than a minute. This interview style is so unrealistic and stressful, and turning a normally stressful event to a real horror show.
Yes you'll have to somehow see to it that I don't cheat if you leave me alone - which makes it harder, but it will be much more humane.
You can just ask them to talk about code. You can ask them to compare and contrast some frameworks or languages of their choosing. You can ask them to do design tasks, or just talk about the work the company does and let them ask questions.
One of the things I like to do, especially when a candidate is super anxious, is just ask them to talk about a recent project they've done that they like and why they like it.
One of the things I like to do, especially when a candidate is super anxious, is just ask them to talk about a recent project they've done that they like and why they like it.
Isn’t that going to result in a highly biased interview process? I suspect extroverts will do disproportionately better than introverts, and those who are comfortable in technologies the interviewer is not will do worse than those who share a similar technical background.
I also think the skill cap on this sort of question is super low.
(I’d put design questions in a different category though)
I also think the skill cap on this sort of question is super low.
(I’d put design questions in a different category though)
What process do you think introverts are going to have an equal footing at that isn't take home homework?
And yes, people who know technology will do better.. in a technology interview. I suppose I could ask about gardening but that doesn't help much does it?
And yes, people who know technology will do better.. in a technology interview. I suppose I could ask about gardening but that doesn't help much does it?
Algorithmic interviews are huge equalizers. Anyone regardless of their social skills or technical background can show their talent.
I’m not talking about people who know technology generally. I’m talking about people who know different technology. The process you’ve proposed, if implemented at, say, PHP shop, will rule out plenty of great .NET developers since the interviewer won’t have the context.
I’m not talking about people who know technology generally. I’m talking about people who know different technology. The process you’ve proposed, if implemented at, say, PHP shop, will rule out plenty of great .NET developers since the interviewer won’t have the context.
Algorithmic interviews optimise for algorithmic knowledge, not on the job capability. It checks for rote learning and time spent preparing for the interview.
Knowing how to implement prefix tree or a hash table shows that you've learned solutions, but not that you can write good code in a team environment. Frankly I'd take a worse engineer that worked better in a team over a 10x that ends up silo'ed in 95% of cases.
Knowing how to implement prefix tree or a hash table shows that you've learned solutions, but not that you can write good code in a team environment. Frankly I'd take a worse engineer that worked better in a team over a 10x that ends up silo'ed in 95% of cases.
That’s not a fair characterization of how most companies (well, at least FAANG) do algorithmic interviews.
A good algorithmic interview does not test algorithm trivia. The question should be solvable with only strong structured thinking and fundamentals.
For example, the much-derided “reverse a binary tree” question (I don’t ask it but I think it’s reasonable) doesn’t really require a ton of algorithm trivia. The data structure can be explained in a few minutes and the problem definition can be made super clear and crisp.
A good algorithmic interview does not test algorithm trivia. The question should be solvable with only strong structured thinking and fundamentals.
For example, the much-derided “reverse a binary tree” question (I don’t ask it but I think it’s reasonable) doesn’t really require a ton of algorithm trivia. The data structure can be explained in a few minutes and the problem definition can be made super clear and crisp.
Algorithmic interviews, at least the way they are formatted now (with interviewers silently watching over the candidate at all times, typing notes) also test for anxiety:
https://news.ycombinator.com/item?id=23848039
Maybe the content, despite it being able to be gamed and drilled via Leetcode et al, is sound, but the format in which it is tested may not be.
https://news.ycombinator.com/item?id=23848039
Maybe the content, despite it being able to be gamed and drilled via Leetcode et al, is sound, but the format in which it is tested may not be.
I've interviewed for Facebook and Google in the past, and at both, my algorithm questions were textbook leetcode/competitive programming questions. You either know the algorithm and/or data structure and know the answer,or you don't and you fail the interview.
Sorry, but introverts can do oral exams just fine.
So instead of selecting people who can do algorithmic problem solving well (under pressure), you’re selecting people who can bullshit well (under pressure).
I’m not convinced it’s really an improvement. I’ve worked with too many people who are great bullshitters but terrible engineers, who sailed through on their charisma.
But that’s why most companies have a mix of technical and behavioral interviews.
I’m not convinced it’s really an improvement. I’ve worked with too many people who are great bullshitters but terrible engineers, who sailed through on their charisma.
But that’s why most companies have a mix of technical and behavioral interviews.
[deleted]
Lots of people can bullshit their way out of this. They just repeat stuff they heard their teammates say.
Talking about higher level stuff doesn't mean you can comprehend certain development problems.
Talking about higher level stuff doesn't mean you can comprehend certain development problems.
> Lots of people can bullshit their way out of this. They just repeat stuff they heard their teammates say.
Indeed. The only way such questions are useful, is as a starting point, maybe they make the candidate more comfortable (less nervous), and they also ensure that you're not asking something the candidate is completely unfamiliar with.
But in any case, any such "experience" or generally "subjective" question must be accompanied by further detail-oriented questions - why did you do this and not that, tradeoffs of different solutions, "how does this framework/library work underneath", questions about understanding performance/security etc characteristics and why, etc.
Indeed. The only way such questions are useful, is as a starting point, maybe they make the candidate more comfortable (less nervous), and they also ensure that you're not asking something the candidate is completely unfamiliar with.
But in any case, any such "experience" or generally "subjective" question must be accompanied by further detail-oriented questions - why did you do this and not that, tradeoffs of different solutions, "how does this framework/library work underneath", questions about understanding performance/security etc characteristics and why, etc.
Lots of people can talk about code but not write it. Way more in fact!
The dirty secret is that it’s never about the coding skills.
It’s about the person’s behavior, the way he/she interacts with you. It’s about the person’s culture, tabs vs space, and that kind of things. And it’s mostly about whether you like the person or not.
And I think it’s fine. You mainly need to be sure that the person knows what a for loop is, other than that, most people are ok at programming. What really matters is that you can work with that person.
Unfortunately interviewers who don’t realize this will unconsciously tweak the interview to make it harder for people they don’t like and easier for people they do like. And then they have an “objective” reason to not hire the person: “oh my god, he didn’t know merge sort”.
I give all kinds of random questions. And many times I recommend hiring people who bombed it, and many times I recommended bailing on ones who nailed it (overconfidence, poor communication, etc).
It’s about the person’s behavior, the way he/she interacts with you. It’s about the person’s culture, tabs vs space, and that kind of things. And it’s mostly about whether you like the person or not.
And I think it’s fine. You mainly need to be sure that the person knows what a for loop is, other than that, most people are ok at programming. What really matters is that you can work with that person.
Unfortunately interviewers who don’t realize this will unconsciously tweak the interview to make it harder for people they don’t like and easier for people they do like. And then they have an “objective” reason to not hire the person: “oh my god, he didn’t know merge sort”.
I give all kinds of random questions. And many times I recommend hiring people who bombed it, and many times I recommended bailing on ones who nailed it (overconfidence, poor communication, etc).
> It’s about the person’s behavior, the way he/she interacts with you. It’s about the person’s culture, tabs vs space, and that kind of things. And it’s mostly about whether you like the person or not
Not in FAANG or famous startups it's not. You're either gonna solve 3-6 medium hard Leetcode questions perfectly or you're out. Sure, liking you will help you better not come off as a douche - but you're not gonna get an offer without being able to solve (unless it's some kind of a diversity hire).
Not in FAANG or famous startups it's not. You're either gonna solve 3-6 medium hard Leetcode questions perfectly or you're out. Sure, liking you will help you better not come off as a douche - but you're not gonna get an offer without being able to solve (unless it's some kind of a diversity hire).
Interview technical folks at a FAANG and I don't asking leetcode questions. It isn't a SWE role but people should be SWE material. I do ask them to
What folks should do is practice coding in front of someone. Writing little exploratory snippets and understand the base library of the language they claim to know. I don't even ask people to vocalize while they are working on the problem. And I solve it at the same time they are and if they get stuck, I show them some of my code.
I had one person start crying they put so much pressure on themselves, I had to take a two week break from interviewing after that.
* read code
* extend some working code with a new feature
* parse some semi-structured text into a data structure
What folks should do is practice coding in front of someone. Writing little exploratory snippets and understand the base library of the language they claim to know. I don't even ask people to vocalize while they are working on the problem. And I solve it at the same time they are and if they get stuck, I show them some of my code.
I had one person start crying they put so much pressure on themselves, I had to take a two week break from interviewing after that.
> I had one person start crying they put so much pressure on themselves, I had to take a two week break from interviewing after that.
This is a sign that the hiring process is broken. It's not testing skill level, it's testing anxiety level.
This is a sign that the hiring process is broken. It's not testing skill level, it's testing anxiety level.
The second sentence may be true but the former isn't.
People who break down and cry in interviews are rare but if they do it, they are best avoided and it was good that the interview revealed this. You cannot have that happening on the job but if they can't handle being asked to program something whilst up against a time limit or being watched, they're going to have breakdowns in other situations too.
People who break down and cry in interviews are rare but if they do it, they are best avoided and it was good that the interview revealed this. You cannot have that happening on the job but if they can't handle being asked to program something whilst up against a time limit or being watched, they're going to have breakdowns in other situations too.
I've written at length about this before, so I'll just give a link instead of repeating the whole screed, but it's a misconception to equate interview anxiety with job anxiety. They're distinct and not necessarily correlated. https://news.ycombinator.com/item?id=31377459
Thanks for taking the time to write that. I agree. Interviewing is a horrible crap shoot and would say even cruel by construction.
It's obviously unpopular, but the brutal reality is that the vast majority of candidates don't panic during interviews and can in fact demonstrate their skills when asked. I've done lots of interviewing and you aren't going to convince me or anyone else actually that breakdowns in an interview are completely irrelevant, especially when all you've been asked to do is demonstrate that you're actually qualified. It's also frankly a bit insulting to people who learned to control their emotions and work under pressure to expect that this behaviour be ignored in others who never learned to do it.
> the vast majority of candidates don't panic during interviews and can in fact demonstrate their skills when asked
Is this actually true? It's been widely and frequently claimed over the years that "199 out of 200 applicants for every programming job can't write code at all. I repeat: they can't write any code whatsoever." https://blog.codinghorror.com/why-cant-programmers-program/
> frankly a bit insulting to people who learned to control their emotions
It's frankly a bit ignorant of human psychology to think that people can just "learn" to control their emotions. We're humans, not robots. Yes, you could go to a professional therapist, if you have both the time and money; it could take a lot of both. But if many people have to go to a therapist for the sole purpose of dealing with audition-style coding interviews, then maybe there's something seriously wrong with audition-style coding interviews. It's already bizarre that candidates with many years of experience in the field have to study intensely for job interviews.
> and work under pressure
I explained at length in the link from my previous comment that working under pressure is entirely different from interview pressure.
Is this actually true? It's been widely and frequently claimed over the years that "199 out of 200 applicants for every programming job can't write code at all. I repeat: they can't write any code whatsoever." https://blog.codinghorror.com/why-cant-programmers-program/
> frankly a bit insulting to people who learned to control their emotions
It's frankly a bit ignorant of human psychology to think that people can just "learn" to control their emotions. We're humans, not robots. Yes, you could go to a professional therapist, if you have both the time and money; it could take a lot of both. But if many people have to go to a therapist for the sole purpose of dealing with audition-style coding interviews, then maybe there's something seriously wrong with audition-style coding interviews. It's already bizarre that candidates with many years of experience in the field have to study intensely for job interviews.
> and work under pressure
I explained at length in the link from my previous comment that working under pressure is entirely different from interview pressure.
By "demonstrating their skills" I meant demonstrate to what extent they have skills, not that they were all very skilled or could do the work. They were nonetheless able to prove what they did know.
I've done hundreds of interviews as the interviewer. I've had people seriously panic maybe once or twice in that entire time. I've heard occasionally that the rates for other interviewers were higher (but still low), if that's the case then they just sucked at interviewing.
Where did therapists come into it? I've never heard of anyone going to a therapist to pass coding interviews, that would be ridiculous. Learning to control your emotions is something people are meant to learn as children. It's a standard skill you're supposed to have to interact with others, like basic politeness. If you're having breakdowns as adults then it's expected that the situation is really intense, like the death of a loved one, breakdown in a relationship etc. Job interviewing is a normal part of life and shouldn't be as stressful as those things.
"It's already bizarre that candidates with many years of experience in the field have to study intensely for job interviews."
The whole "cramming leetcode" thing is way overblown, or possibly a modern feedback loop in which crammers are making it harder to detect genuine skill and experience. Again I've done a lot of interviews and nobody ever mentioned having to prepare for them. That idea seems to be a relatively recent idea (last ~10 years or so) and is probably a result of so many people competing for very highly paid jobs at a small number of firms. Normal software jobs should ask people to demonstrate their skills, but it shouldn't be something that requires exam-like prep for any working programmer.
I've done hundreds of interviews as the interviewer. I've had people seriously panic maybe once or twice in that entire time. I've heard occasionally that the rates for other interviewers were higher (but still low), if that's the case then they just sucked at interviewing.
Where did therapists come into it? I've never heard of anyone going to a therapist to pass coding interviews, that would be ridiculous. Learning to control your emotions is something people are meant to learn as children. It's a standard skill you're supposed to have to interact with others, like basic politeness. If you're having breakdowns as adults then it's expected that the situation is really intense, like the death of a loved one, breakdown in a relationship etc. Job interviewing is a normal part of life and shouldn't be as stressful as those things.
"It's already bizarre that candidates with many years of experience in the field have to study intensely for job interviews."
The whole "cramming leetcode" thing is way overblown, or possibly a modern feedback loop in which crammers are making it harder to detect genuine skill and experience. Again I've done a lot of interviews and nobody ever mentioned having to prepare for them. That idea seems to be a relatively recent idea (last ~10 years or so) and is probably a result of so many people competing for very highly paid jobs at a small number of firms. Normal software jobs should ask people to demonstrate their skills, but it shouldn't be something that requires exam-like prep for any working programmer.
> I've had people seriously panic maybe once or twice in that entire time.
I think you're making too much of the example of the candidate crying. That is indeed an outlier case, not the norm. But the crying case is an undeniable sign that job candidates can get extremely anxious during an interview, in a way that affects their performance. The issue isn't "breakdowns" per se, the issue is that coding tests are testing anxiety level rather than skill level. Ironically, most job candidates are in fact "controlling their emotions", as you say: they may show no outward sign of breakdown or panic, but on the inside, their mind and stomach are churning heavily, and this makes it difficult to perform at their normal level. Everyone is trying to act calm and collected during an interview, but often it's merely an act, and they're actually not calm and collected at all. What you perceive from the outside as incompetence may be a whole different story on the inside for the candidate. https://www.engr.ncsu.edu/news/2020/11/11/tech-sector-job-in...
Anxiety isn't all or nothing. There's a wide spectrum between totally calm and crying breakdown. When there's a crying breakdown, it's easy to recognize, but anxiety is not so easy to recognize if there's not a breakdown.
> I've done a lot of interviews and nobody ever mentioned having to prepare for them.
Why would they mention it, even if they did prepare?
> That idea seems to be a relatively recent idea (last ~10 years or so) and is probably a result of so many people competing for very highly paid jobs at a small number of firms.
Regardless, we're living in the present now, under current conditions, not living in the past, so for the purposes of getting a job, it doesn't really matter when the idea originated.
I think you're making too much of the example of the candidate crying. That is indeed an outlier case, not the norm. But the crying case is an undeniable sign that job candidates can get extremely anxious during an interview, in a way that affects their performance. The issue isn't "breakdowns" per se, the issue is that coding tests are testing anxiety level rather than skill level. Ironically, most job candidates are in fact "controlling their emotions", as you say: they may show no outward sign of breakdown or panic, but on the inside, their mind and stomach are churning heavily, and this makes it difficult to perform at their normal level. Everyone is trying to act calm and collected during an interview, but often it's merely an act, and they're actually not calm and collected at all. What you perceive from the outside as incompetence may be a whole different story on the inside for the candidate. https://www.engr.ncsu.edu/news/2020/11/11/tech-sector-job-in...
Anxiety isn't all or nothing. There's a wide spectrum between totally calm and crying breakdown. When there's a crying breakdown, it's easy to recognize, but anxiety is not so easy to recognize if there's not a breakdown.
> I've done a lot of interviews and nobody ever mentioned having to prepare for them.
Why would they mention it, even if they did prepare?
> That idea seems to be a relatively recent idea (last ~10 years or so) and is probably a result of so many people competing for very highly paid jobs at a small number of firms.
Regardless, we're living in the present now, under current conditions, not living in the past, so for the purposes of getting a job, it doesn't really matter when the idea originated.
After they were hired, and we were talking about interview design, and they were proposing questions harder than mine? Seems reasonable to mention if they had to prep then. Later I also pushed back on proposals for things like homework, but others seemed to find that idea acceptable. I guess they'd have mentioned it then.
You seem to be suggesting something totally unfalsifiable - that candidates who do a bad job aren't actually bad programmers, but are just nervous inside in undetectable ways. There's no way I or anyone else can argue against this notion because there's no way to prove a negative like that, but suffice it to say that the sort of issues I'd associate with nervousness - minor slips that they then notice and correct, stumbles, brief moments of forgetfulness etc - are not why people tend to fail interviews. They fail interviews because their coding skills suck.
You seem to be suggesting something totally unfalsifiable - that candidates who do a bad job aren't actually bad programmers, but are just nervous inside in undetectable ways. There's no way I or anyone else can argue against this notion because there's no way to prove a negative like that, but suffice it to say that the sort of issues I'd associate with nervousness - minor slips that they then notice and correct, stumbles, brief moments of forgetfulness etc - are not why people tend to fail interviews. They fail interviews because their coding skills suck.
> You seem to be suggesting something totally unfalsifiable - that candidates who do a bad job aren't actually bad programmers, but are just nervous inside in undetectable ways.
Heh. It's only unfalsifiable if you ignore a person's previous experience and accomplishments, refusing to acknowledge anything other than your own interview as an accurate gauge of the person's skills.
Also, I'm literally telling you that it happens to me. (And I've heard many people say it happens to them.) So either I'm a liar, or you have to admit that it happens. Go ahead and call me a liar if that's what you want to believe, but I'm not even trying to get a job anymore. I'm happily self-employed now, and my customers also seem very happy with my coding skills.
The term "control your emotions" is somewhat vague. You can hide your emotions, but that doesn't mean you're not experiencing the emotions. An extreme example of that is if you "put on a brave face" after a personal tragedy. You may not cry and break down, but that doesn't mean the sadness doesn't affect you severely. The reason I mentioned therapy is that you'd probably need therapy in order to somehow train yourself to not even feel the emotions in the first place in situations where they arise automatically for you, i.e., to not have certain phobias at all.
> the sort of issues I'd associate with nervousness
Have you considered that you might not have a full understanding and appreciation for human psychology and the wide variance among humans?
Heh. It's only unfalsifiable if you ignore a person's previous experience and accomplishments, refusing to acknowledge anything other than your own interview as an accurate gauge of the person's skills.
Also, I'm literally telling you that it happens to me. (And I've heard many people say it happens to them.) So either I'm a liar, or you have to admit that it happens. Go ahead and call me a liar if that's what you want to believe, but I'm not even trying to get a job anymore. I'm happily self-employed now, and my customers also seem very happy with my coding skills.
The term "control your emotions" is somewhat vague. You can hide your emotions, but that doesn't mean you're not experiencing the emotions. An extreme example of that is if you "put on a brave face" after a personal tragedy. You may not cry and break down, but that doesn't mean the sadness doesn't affect you severely. The reason I mentioned therapy is that you'd probably need therapy in order to somehow train yourself to not even feel the emotions in the first place in situations where they arise automatically for you, i.e., to not have certain phobias at all.
> the sort of issues I'd associate with nervousness
Have you considered that you might not have a full understanding and appreciation for human psychology and the wide variance among humans?
> the brutal reality is that the vast majority of candidates don't panic during interviews and can in fact demonstrate their skills
For a given job most candidates actually don't get hired, often because they are not a good fit but many times because they are too nervous to think straight.
For a given job most candidates actually don't get hired, often because they are not a good fit but many times because they are too nervous to think straight.
I've done hundreds of interviews and that is only very rarely a problem. Normally candidates aren't nervous, but just can't program. They don't have the knowledge, they're flummoxed by basic errors from their compiler/interpreter even when they got to pick what language they used, they're asked to write something (anything) in a language they advertise knowing on their CV and then admit they don't actually know it, etc.
> Normally candidates aren't nervous, but just can't program.
How do you know their level of nervousness? You can't read minds.
How do you know their level of nervousness? You can't read minds.
> I had one person start crying they put so much pressure on themselves, I had to take a two week break from interviewing after that.
Because you asked them question out of syllabus. Was the person informed that the interview wouldn't involve Leetcode?
The person spent months prepping Leetcode just to get into BigTech and then you come out of woodwork with your "special" interviewing material. Imagine prepping for a calculus test for months and then the teacher comes out and takes a "simple" test on trignometry. Would you be able to do it? Not only that, all his/her friends got a normal calculus test which they passed with flying colors.
Because you asked them question out of syllabus. Was the person informed that the interview wouldn't involve Leetcode?
The person spent months prepping Leetcode just to get into BigTech and then you come out of woodwork with your "special" interviewing material. Imagine prepping for a calculus test for months and then the teacher comes out and takes a "simple" test on trignometry. Would you be able to do it? Not only that, all his/her friends got a normal calculus test which they passed with flying colors.
This wasn't a Leetcode question. I understand where you are coming from, but it isn't that.
This had to do with the pressure one puts on themselves for a job a BigTech company.
This had to do with the pressure one puts on themselves for a job a BigTech company.
I was referring to SWE roles, as is this article I believe.
They aren't technically SWEs but they would be at any other company. I have been a SWE at three other FAANGs
> I have been a SWE at three other FAANGs
Good for you.
Now seriously I'm not sure what you're arguing for - you're saying FAANGS don't do Leetcode style questions? I get it that YOU don't do it to candidates in those roles, but the reality is for SWE roles this is how it is. People who aren't very good in Leetcode will not succeed. There's hundreds, maybe thousands of candidates for these roles and some of them can do the Leetcode thing - they are the ones who get hired. I'm not advocating for this method because I'm not particularly good at it or like solving these things on my spare time, nor do I think this method is particularly useful - but that's reality.
Good for you.
Now seriously I'm not sure what you're arguing for - you're saying FAANGS don't do Leetcode style questions? I get it that YOU don't do it to candidates in those roles, but the reality is for SWE roles this is how it is. People who aren't very good in Leetcode will not succeed. There's hundreds, maybe thousands of candidates for these roles and some of them can do the Leetcode thing - they are the ones who get hired. I'm not advocating for this method because I'm not particularly good at it or like solving these things on my spare time, nor do I think this method is particularly useful - but that's reality.
I know Google has some tech support engineering roles where they ask such simple questions to candidates. They aren't software engineers and generally don' get to contribute to the main codebase, but they help customers fix problems with how they use Google API's etc so they need to know a bit about how to code.
It is definitely about the coding skills. It is too easy to bullshit and there is a financial incentive to do so. Also many people with genunine 10 years of experience are not too good at coding.
Many people with genuine 10 years of experience do not care for stupid leetcode question they could obviously solve given some quiet and time.
The idea that everything boils down to answering some intro to programming questions quickly is absurd.
The idea that everything boils down to answering some intro to programming questions quickly is absurd.
Years of experience are not created equal. There are far fewer who can actually solve them but don’t want to than those who just can’t. If you spent the last 10 years just ossifying then you might be great in your current role but realistically you’re terrible at building new things.
While providing a syntactically correct, optimal solution to a given problem is important (this was reiterated many times during my recent interview process at a FAANG), it is also important to be able to demonstrate the ability to synthesize an approach to solving a problem to which you don't already know the answer. That is, how to solve a problem.
These questions, while certainly aimed at identifying core competency areas, are also an opportunity for the candidate to demonstrate an aptitude for problem solving. Nobody really cares "how many piano tuners operate in Seattle". What they care about is, given that kind of question, how a candidate responds. You might be surprised by the number of people totally unable to even generate a reasonable estimate:
- How many people live in Seattle? - What percentage have pianos? - How often does a piano need tuning? - How long does it take to tune a piano? - How many pianos can be tuned per work day (given commuting time)? - etc.. driving towards a number.
Did the candidate attack the problem and even attempt to solve it? Did they use metrics? The list goes on... Most LC-style questions (save the "aha! moment" ones) have a similar impetus, but are also much more practical and efficient because they test for many things at once.
And if I'm being honest... most FAANGs aren't looking for engineers that require quiet and time to solve problems. They are looking for elite engineers who can write a solution nearly as fast as the problem is presented (I'm obviously exaggerating a little bit here). The difficulty of the questions presented to me (6 technical interviews) were what I consider semi-trivial. I was able to describe the solution-space within seconds of reading the prompt and code an optimal solution nearly as fast as I could type in all cases. It honestly made me wonder about how poorly the average programmer must perform such that these were the questions they needed to evaluate my potential.
Another commenter posted this article [0] about the interviewing philosophy back in the early '00s. I found it very enlightening, and reassuringly, while reading through it I knew the answer to nearly every question (and could expand upon the topics) just off the top of my head. I'm not trying to sound arrogant here but the idea that a FAANG, who pays top-dollar for engineers, would settle for "10 years experience" as a suitable replacement for "demonstrates ability" is what's absurd. A false positive is way more costly than a false negative.
[0] https://sites.google.com/site/steveyegge2/five-essential-pho...
These questions, while certainly aimed at identifying core competency areas, are also an opportunity for the candidate to demonstrate an aptitude for problem solving. Nobody really cares "how many piano tuners operate in Seattle". What they care about is, given that kind of question, how a candidate responds. You might be surprised by the number of people totally unable to even generate a reasonable estimate:
- How many people live in Seattle? - What percentage have pianos? - How often does a piano need tuning? - How long does it take to tune a piano? - How many pianos can be tuned per work day (given commuting time)? - etc.. driving towards a number.
Did the candidate attack the problem and even attempt to solve it? Did they use metrics? The list goes on... Most LC-style questions (save the "aha! moment" ones) have a similar impetus, but are also much more practical and efficient because they test for many things at once.
And if I'm being honest... most FAANGs aren't looking for engineers that require quiet and time to solve problems. They are looking for elite engineers who can write a solution nearly as fast as the problem is presented (I'm obviously exaggerating a little bit here). The difficulty of the questions presented to me (6 technical interviews) were what I consider semi-trivial. I was able to describe the solution-space within seconds of reading the prompt and code an optimal solution nearly as fast as I could type in all cases. It honestly made me wonder about how poorly the average programmer must perform such that these were the questions they needed to evaluate my potential.
Another commenter posted this article [0] about the interviewing philosophy back in the early '00s. I found it very enlightening, and reassuringly, while reading through it I knew the answer to nearly every question (and could expand upon the topics) just off the top of my head. I'm not trying to sound arrogant here but the idea that a FAANG, who pays top-dollar for engineers, would settle for "10 years experience" as a suitable replacement for "demonstrates ability" is what's absurd. A false positive is way more costly than a false negative.
[0] https://sites.google.com/site/steveyegge2/five-essential-pho...
It's is also easy to bullshit a coding interview. The problems solved at interviews are just too small to verify code organization skills.
I suppose you never did any interviews yourself. You can't imagine how many people can't even do or know the basics.
> You can't imagine how many people can't even do or know the basics.
I can imagine it, because I've been that person.
It's simple interview anxiety, which causes temporary but severe performance degradation. Then when the interview is over, the anxiety suddenly disappears, and you can even go back and do the same problems much more easily.
But I suppose interviewers will just assume that my resume is a lie or that I managed to incompetently hold onto previous jobs.
I can imagine it, because I've been that person.
It's simple interview anxiety, which causes temporary but severe performance degradation. Then when the interview is over, the anxiety suddenly disappears, and you can even go back and do the same problems much more easily.
But I suppose interviewers will just assume that my resume is a lie or that I managed to incompetently hold onto previous jobs.
Yet they are able to do the job. Something doesn't lineup with your testing methods vs outcome.
If these people are finding work elsewhere and doing productive work for years what does it say about your process?
If these people are finding work elsewhere and doing productive work for years what does it say about your process?
Do you think they are looking for a job because they were fired somewhere else?
Or do you think companies never fire people?
We already fired plenty of people that we hired. So yes, our hiring process is not perfect. But if you're a JavaScript developer and can't explain the difference between var, let and const, I'm not going to hire you. If you claim these people are great developers, you are free to hire them.
Let me know the details of your company. Whenever we find a candidate that doesn't satisfy our requirements (which is around 90% of them), I'll send them to you so you can hire them.
Or do you think companies never fire people?
We already fired plenty of people that we hired. So yes, our hiring process is not perfect. But if you're a JavaScript developer and can't explain the difference between var, let and const, I'm not going to hire you. If you claim these people are great developers, you are free to hire them.
Let me know the details of your company. Whenever we find a candidate that doesn't satisfy our requirements (which is around 90% of them), I'll send them to you so you can hire them.
I've never said you should hire people who can't tell var from const.
I've said that even if you hire people who do know the basics of the language, and some algorithms, and even could solve a toy problem on paper, you might still hire someone who can't code at scale. Coding interviews typically don't test design skills, documentation skills, ability to not get lost on a larger code base, debugging skills, tendency to avoid hacks and increase tech debt etc.
I've said that even if you hire people who do know the basics of the language, and some algorithms, and even could solve a toy problem on paper, you might still hire someone who can't code at scale. Coding interviews typically don't test design skills, documentation skills, ability to not get lost on a larger code base, debugging skills, tendency to avoid hacks and increase tech debt etc.
Those are different skill sets. Someone may know and know how to properly use var, let and const according to standards and current trends. They may understand it deeply. But that doesn't mean that they can explain it to you.
The question changes to: is what you are testing the actual skill you need? Are you hiring a teacher who is going to explain var,let,const as part of their job or are you hiring someone who needs to know how properly to use var,let,const?
Send me over any developer who fails your teaching requirements for var,let,const but has code samples with proper usage.
I've never seen a senior developer fired for poor coding. I've seen them get fired for being part of a department cut or because they got into a fight with the owner/boss. I've seen them fired because they make too much compared to outsourced resources.
The question changes to: is what you are testing the actual skill you need? Are you hiring a teacher who is going to explain var,let,const as part of their job or are you hiring someone who needs to know how properly to use var,let,const?
Send me over any developer who fails your teaching requirements for var,let,const but has code samples with proper usage.
I've never seen a senior developer fired for poor coding. I've seen them get fired for being part of a department cut or because they got into a fight with the owner/boss. I've seen them fired because they make too much compared to outsourced resources.
> that doesn't mean that they can explain it to you.
I'm sorry what? I don't expect them to teach me, I just want them to tell the difference. By the way, the ones that can't say: "they told me to always use let". I'll send them to you :D. Best of luck.
> I've never seen a senior developer fired for poor coding.
I've seen plenty. If a junior dev needs to help you out of trouble every time, you know there is something wrong.
Some people just reach their limit very soon, and can hide away in some comppanies. Probaly companies where you seem to work. Other companies probably just have a higher bar, no disrespect meant.
I'm sorry what? I don't expect them to teach me, I just want them to tell the difference. By the way, the ones that can't say: "they told me to always use let". I'll send them to you :D. Best of luck.
> I've never seen a senior developer fired for poor coding.
I've seen plenty. If a junior dev needs to help you out of trouble every time, you know there is something wrong.
Some people just reach their limit very soon, and can hide away in some comppanies. Probaly companies where you seem to work. Other companies probably just have a higher bar, no disrespect meant.
I've worked in a number of places and people don't have the limitations you believe.
A junior helping a senior? Shame on them? Fire the senior? That sounds really foolish. You realize that junior/senior/etc are labels and someone you hire a junior can know more syntax than a senior in a framework. That's okay that the senior seeks helo.. it's natural and what a senior should do. p You are presenting your company in a bad light and making yourself sound inexperienced.
Could you have reached your limit? Would your methods fly elsewhere? Apply the len to yourself and see where you end up.
A junior helping a senior? Shame on them? Fire the senior? That sounds really foolish. You realize that junior/senior/etc are labels and someone you hire a junior can know more syntax than a senior in a framework. That's okay that the senior seeks helo.. it's natural and what a senior should do. p You are presenting your company in a bad light and making yourself sound inexperienced.
Could you have reached your limit? Would your methods fly elsewhere? Apply the len to yourself and see where you end up.
When other devs get stuck on something, they come to me or one of the other seniors. And then I or they solve it. Sometimes it's difficult, and sometimes it's a silly oversight. So yeah, I have no problem applying it to myself.
I can't really believe that people here think when you have more years under your belt, you somehow must be good. People get fired in companies because they don't reach the expected level. If you keep them, other colleagues will get frustrated and leave on their own. And then you're stuck with the underperformers.
And yes, everyone has limitations. Some sooner than others. Welcome to reality.
I find the discussions here very silly that somehow you must hire everyone without a technical interview, and never fire anyone because people don't have limitations. Not going to respond anymore.
I can't really believe that people here think when you have more years under your belt, you somehow must be good. People get fired in companies because they don't reach the expected level. If you keep them, other colleagues will get frustrated and leave on their own. And then you're stuck with the underperformers.
And yes, everyone has limitations. Some sooner than others. Welcome to reality.
I find the discussions here very silly that somehow you must hire everyone without a technical interview, and never fire anyone because people don't have limitations. Not going to respond anymore.
Look. I've seen a lot of your comments in this thread, and I don't think you're listening. You don't seem open to hearing.
What I read is a knee jerk justification for your hiring methodology without being open to any feedback.
You should take a breath. Clearly people disagree with you. Try to empathize with where they're coming from and understand the point they're trying to make.
What I read is a knee jerk justification for your hiring methodology without being open to any feedback.
You should take a breath. Clearly people disagree with you. Try to empathize with where they're coming from and understand the point they're trying to make.
Ok, so what's the ideal hiring practice then? According to some here, it's just a little talk about previous experiences, not going into technical details at all. Sorry, but then you end up with people who are technically very weak. Been there done that.
Look, I have a lot of experience with the hiring process. It's very difficult. I will listen to someone with a lot of hiring experience. But just saying "it's broken" isn't going to cut it.
And remark that a big part of our hiring practice is to put the candidate at ease. I even say that they won't get everything right, and that's no problem. I help them out when they get stuck, etc.
Look, I have a lot of experience with the hiring process. It's very difficult. I will listen to someone with a lot of hiring experience. But just saying "it's broken" isn't going to cut it.
And remark that a big part of our hiring practice is to put the candidate at ease. I even say that they won't get everything right, and that's no problem. I help them out when they get stuck, etc.
i do sort of pair programming with candidates specifically to evaluate the interaction. i let them drive and navigate, acting more like a passenger with the occasional input. that way i can see coding skills and ability to communicate.
of course i interview for my specific situation. the candidate will be working with me directly, so they should at least be able to interact with me. if i don't like them then that's almost a showstopper, but if the candidate otherwise qualifies i should try to figure out what my problem is, and see if i can get over it.
of course i interview for my specific situation. the candidate will be working with me directly, so they should at least be able to interact with me. if i don't like them then that's almost a showstopper, but if the candidate otherwise qualifies i should try to figure out what my problem is, and see if i can get over it.
SWE to lawyer here.
The analogs of the traditional coding interview in the law school world are the LSAT and classroom timed memory tests. Both have absolutely nothing whatsoever to do with being a lawyer or the kinds of activities that lawyers actually do.
What they are are intelligence tests. They are easy ways for judges and top tier law firms to tell if you are "one of them." You can get your foot in the door if you can perform, at a minimum, at a certain level. There are plenty of things you can do without 97+ percentile LSAT and A's in law school, but you are screened from the top echelon of work.
People object that this isn't fair and screens good people out -- false negatives. But experience has shown that it is effective and easy. Perhaps there is a better way, but this way seems to work well enough. There is little incentive, beyond some people complaining about it, to change a system that is working well.
I see the traditional leetcode interview in the same light. The point is to see if you can perform at a certain minimal level. Yes, it's arbitrary, and often unconnected to real life. But it is an effective way to gauge a minimum level of intelligence and competence. Not perfect, but easy and effective.
Finally, there is ego element to this. People who can perform at this level see these skills as a minimum qualification. They are uncomfortable seeing themselves in relation to others who cannot perform at this level. Knowing that their peers have similar minimum qualifications makes them feel safe and secure about their status. This is not a criticism, just an observation about human nature. People in a club need a shared identity.
The analogs of the traditional coding interview in the law school world are the LSAT and classroom timed memory tests. Both have absolutely nothing whatsoever to do with being a lawyer or the kinds of activities that lawyers actually do.
What they are are intelligence tests. They are easy ways for judges and top tier law firms to tell if you are "one of them." You can get your foot in the door if you can perform, at a minimum, at a certain level. There are plenty of things you can do without 97+ percentile LSAT and A's in law school, but you are screened from the top echelon of work.
People object that this isn't fair and screens good people out -- false negatives. But experience has shown that it is effective and easy. Perhaps there is a better way, but this way seems to work well enough. There is little incentive, beyond some people complaining about it, to change a system that is working well.
I see the traditional leetcode interview in the same light. The point is to see if you can perform at a certain minimal level. Yes, it's arbitrary, and often unconnected to real life. But it is an effective way to gauge a minimum level of intelligence and competence. Not perfect, but easy and effective.
Finally, there is ego element to this. People who can perform at this level see these skills as a minimum qualification. They are uncomfortable seeing themselves in relation to others who cannot perform at this level. Knowing that their peers have similar minimum qualifications makes them feel safe and secure about their status. This is not a criticism, just an observation about human nature. People in a club need a shared identity.
Great analogy. I think the rub is I wouldn't want to study for and take the LSAT before every interview, and especially not after working in the field for 10 or more years.
Especially if the interviewers make comments like "um how much do you actually practice law?" if you stumble on one of their pet quiz questions.
Especially if the interviewers make comments like "um how much do you actually practice law?" if you stumble on one of their pet quiz questions.
> The analogs of the traditional coding interview in the law school world are the LSAT and classroom timed memory tests.
How are these analogous? LSAT is the Law School Admission Test. It gets you into law school. It doesn't get you a job as a lawyer. (The bar exam doesn't get you a job as a lawyer either.) And you only have to take it once, not for every job interview.
How are these analogous? LSAT is the Law School Admission Test. It gets you into law school. It doesn't get you a job as a lawyer. (The bar exam doesn't get you a job as a lawyer either.) And you only have to take it once, not for every job interview.
Fair enough; it's not a perfect analogy. The main point is the nature of the metrics.
Getting into law school is the first part of the "interview." If you don't have the LSAT score, you won't get into a prestigious law school, full stop. If you aren't in a prestigious school, then a huge swath of elite positions are permanently closed to you. (In general -- yes there are a sprinkling of exceptions.) The bar exam has nothing to do with it -- everyone has to pass that, no matter what you do.
The second part of the interview is law school grades and certain extra curricular signals. You're right, there is only one "interview." But the analogy still holds up, with respect to the nature of the intelligence and memory tests.
Getting into law school is the first part of the "interview." If you don't have the LSAT score, you won't get into a prestigious law school, full stop. If you aren't in a prestigious school, then a huge swath of elite positions are permanently closed to you. (In general -- yes there are a sprinkling of exceptions.) The bar exam has nothing to do with it -- everyone has to pass that, no matter what you do.
The second part of the interview is law school grades and certain extra curricular signals. You're right, there is only one "interview." But the analogy still holds up, with respect to the nature of the intelligence and memory tests.
Aren't those one time things that define the future of your career? Sort of a pedigree being established?
You can go to a top tier university, land a great job for several years, have positive references, and still be required to do leet code for a recruiter... that seems like ongoing gatekeeping, rather than a one-time effort/pass.
You can go to a top tier university, land a great job for several years, have positive references, and still be required to do leet code for a recruiter... that seems like ongoing gatekeeping, rather than a one-time effort/pass.
Sure but I can get a much better gauge of intelligence from OSS. I’ve seen some of the worst developers I know beat Leetcode because it’s really just memorization not actual intelligence
What was your journey from engineer to lawyer like?
> You mainly need to be sure that the person knows what a for loop is, other than that, most people are ok at programming.
When I'm interviewing a senior developer that will guide other juniors, they will need to prove they know more than a for loop. They will need to show they understand the underlying concepts and technologies. Else it's a no hire, tough luck.
We also fired plenty of people that weren't up to the quality we expect.
Were are you working? At some government?
When I'm interviewing a senior developer that will guide other juniors, they will need to prove they know more than a for loop. They will need to show they understand the underlying concepts and technologies. Else it's a no hire, tough luck.
We also fired plenty of people that weren't up to the quality we expect.
Were are you working? At some government?
Statements like this make me distinctly angry because I do everything I can to remove myself from the interview, of sorts.
If they come up with, or start walking with an interesting solution, I listen to them.
Tabs and spaces don't matter.
I actively work against the biases of "whether I like the person or not" because I'm aware of that as a factor.
We can absolutely minimize all of those peripherals when we explicitly train against it and a good programming interview doesn't have to be about that.
If they come up with, or start walking with an interesting solution, I listen to them.
Tabs and spaces don't matter.
I actively work against the biases of "whether I like the person or not" because I'm aware of that as a factor.
We can absolutely minimize all of those peripherals when we explicitly train against it and a good programming interview doesn't have to be about that.
Well I didn’t mean to offend anyone. I guess that just makes you a better technical interviewer than most.
But to be perfectly unbiased, the process would have to double blind. And I don’t know anyone who likes to hire people (or anyone who wants to be hired) following a double blind process.
So at least there is _some_ non-technical bias that gets in the process.
But to be perfectly unbiased, the process would have to double blind. And I don’t know anyone who likes to hire people (or anyone who wants to be hired) following a double blind process.
So at least there is _some_ non-technical bias that gets in the process.
Yup this is true for the most part, although someone bad will be harder to justify.
It’s hilarious listening to people review candidates on third coding interviews. Usually it’s “this person didn’t know this one thing I think makes me really smart”.
This is why I lean so heavy in OSS to review candidates. I can see how they interact with people, and whether they are actually able to build software that _people want_
I find with this approach it’s a lot harder to justify nonsense.
I hear people say maybe they don’t have time for OSS and then hand them a Leetcode interview, which as we all know is just about memorizing problems. OSS is such a better metric and has the added benefit of improving the world.
It’s hilarious listening to people review candidates on third coding interviews. Usually it’s “this person didn’t know this one thing I think makes me really smart”.
This is why I lean so heavy in OSS to review candidates. I can see how they interact with people, and whether they are actually able to build software that _people want_
I find with this approach it’s a lot harder to justify nonsense.
I hear people say maybe they don’t have time for OSS and then hand them a Leetcode interview, which as we all know is just about memorizing problems. OSS is such a better metric and has the added benefit of improving the world.
> It’s hilarious listening to people review candidates on third coding interviews. Usually it’s “this person didn’t know this one thing I think makes me really smart”.
Yup. I just went through a round of interviews at multiple companies. In the past I successfully interviewed at a FAANG.
I agree completely.
Yup. I just went through a round of interviews at multiple companies. In the past I successfully interviewed at a FAANG.
I agree completely.
> This is why I lean so heavy in OSS to review candidates. I can see how they interact with people, and whether they are actually able to build software that _people want_
This.
There is absolutely no need to ask such ridiculous algorithms and advanced equations questions that you will never use or implement from scratch yourself, or asking the candidate to create a formal proof by rote for a entry-level software engineering position or even a senior level position.
Such interviewers who ask these questions have both googled the answers to these questions before the interview and also have never used them practically, but only do this to appear smarter than they actually are. Worse part is not even FAANG asks any of these formal proof questions, but only some certain startups that the last time I have checked, they have failed to make money and have shutdown. Why? Because they seriously thought they were a FAANG company.
Open-source cuts through the algorithms nonsense, saves time and gets a realistic idea of what the candidate has actually worked on in the open. I can ignore the typical hello world, or demo projects but can ask for if they have created open-source libraries or contributions to any serious projects like compilers, OS projects, or any related project to the job description etc.
A take home project is also a fine metric, but asking questions about algorithms and finding out that the interviewer (or the company) doesn't even use them tells you that the interviewer doesn't know what they are looking for.
This.
There is absolutely no need to ask such ridiculous algorithms and advanced equations questions that you will never use or implement from scratch yourself, or asking the candidate to create a formal proof by rote for a entry-level software engineering position or even a senior level position.
Such interviewers who ask these questions have both googled the answers to these questions before the interview and also have never used them practically, but only do this to appear smarter than they actually are. Worse part is not even FAANG asks any of these formal proof questions, but only some certain startups that the last time I have checked, they have failed to make money and have shutdown. Why? Because they seriously thought they were a FAANG company.
Open-source cuts through the algorithms nonsense, saves time and gets a realistic idea of what the candidate has actually worked on in the open. I can ignore the typical hello world, or demo projects but can ask for if they have created open-source libraries or contributions to any serious projects like compilers, OS projects, or any related project to the job description etc.
A take home project is also a fine metric, but asking questions about algorithms and finding out that the interviewer (or the company) doesn't even use them tells you that the interviewer doesn't know what they are looking for.
> The dirty secret is that it’s never about the coding skills.
If you don't answer the 2 leetcode mediums in optimal complexity within 45 minutes then you don't pass. It's about coding skills.
If you don't answer the 2 leetcode mediums in optimal complexity within 45 minutes then you don't pass. It's about coding skills.
The best coding interviews for 90% of tech jobs are ones that are heavy on coding and light on theory. Very few jobs are particularly well served by somebody with strong theoretical foundations, while most are well served by somebody who can pump out high quality code quickly.
Yet most companies interview as if they are inventing novel storage/processing mechanisms. Theory is important to understand which tools to leverage when, of course, but not to the extent that it's typically prioritized in the typical interview.
I've had plenty of people crush the theory portion of the interview and be poor performers on the job (slow coding, low volume output), but I've never had somebody crush the coding portion (implement very fast and proficiently), and perform poorly on the job
Yet most companies interview as if they are inventing novel storage/processing mechanisms. Theory is important to understand which tools to leverage when, of course, but not to the extent that it's typically prioritized in the typical interview.
I've had plenty of people crush the theory portion of the interview and be poor performers on the job (slow coding, low volume output), but I've never had somebody crush the coding portion (implement very fast and proficiently), and perform poorly on the job
Not for nothing but we have plenty of slow coders that are “low volume” but some of the most diligent and edge case counting engineer’s I’ve ever met. Even then, if the business doesn’t depend on everyone working like a banshee, my team may as well take their time.
Even as their manager it’s not my job to lead a death march.
Even as their manager it’s not my job to lead a death march.
Most growth companies have an endless bucket of work.
Somebody who produces 2x at same quality of work is worth 2x the other person, fairly objectively.
It's common to find people who produce up to 10x the average. Anyone in a startup or growth oriented company is smart to optimize for these people.
The most proficient and fast coders often write higher quality code too, in my experience.
People who over index on comp sci theory often are lacking in theory underlying good code. But it's not mutually exclusive. Just that if you have to optimize for one, coding proficiency matters much more for startup or growth type companies.
Exception might be like an AI/ML based startup with a lot of theoretical challenges. Most companies much more business/rule oriented though
Somebody who produces 2x at same quality of work is worth 2x the other person, fairly objectively.
It's common to find people who produce up to 10x the average. Anyone in a startup or growth oriented company is smart to optimize for these people.
The most proficient and fast coders often write higher quality code too, in my experience.
People who over index on comp sci theory often are lacking in theory underlying good code. But it's not mutually exclusive. Just that if you have to optimize for one, coding proficiency matters much more for startup or growth type companies.
Exception might be like an AI/ML based startup with a lot of theoretical challenges. Most companies much more business/rule oriented though
"have an endless bucket of work"
A long time ago I managed a developer who was a genius at finding and fixing bugs - so I thought I would reward him by writing a new module we needed from scratch. He could not do it - when faced with a complex mess of existing code he was perfectly happy and could dive in and fix stuff, faced with a blank sheet of paper he froze and didn't know where to start.
Developers are people and all people are different - we all have our strengths and weaknesses.
A long time ago I managed a developer who was a genius at finding and fixing bugs - so I thought I would reward him by writing a new module we needed from scratch. He could not do it - when faced with a complex mess of existing code he was perfectly happy and could dive in and fix stuff, faced with a blank sheet of paper he froze and didn't know where to start.
Developers are people and all people are different - we all have our strengths and weaknesses.
How do you actually measure 2x or 10x workload while keeping quality the same?
It's _very_ easy to measure how fast it takes a ticket to go from "in progress" to "done" but measuring whether code has bugs, or leads to downtime or is just impossible to parse is a different problem.
If you never need code to be readable or documented I can write it quite quickly. If you want something other humans can have a hope of maintaining, that might take a bit.
It's _very_ easy to measure how fast it takes a ticket to go from "in progress" to "done" but measuring whether code has bugs, or leads to downtime or is just impossible to parse is a different problem.
If you never need code to be readable or documented I can write it quite quickly. If you want something other humans can have a hope of maintaining, that might take a bit.
I think everyone who has been in tech for 7 years has a short list of people they’d want on their 4-person tech team and a longer list of people they wouldn’t.
That I can’t tell you to two decimal places exactly how much better the first list is than the second doesn’t mean I can’t tell who’s on which list with high fidelity/repeatability.
That I can’t tell you to two decimal places exactly how much better the first list is than the second doesn’t mean I can’t tell who’s on which list with high fidelity/repeatability.
But those are people you've actually worked with day to day, right? They've had the long form of the interview with you and passed.
I know a bunch of people like that, but I've only worked with so many people. A startup would exhaust their friends pretty fast and be back to interviewing strangers.
I know a bunch of people like that, but I've only worked with so many people. A startup would exhaust their friends pretty fast and be back to interviewing strangers.
I love creating these lists in my head. Sometimes when I do a good job, I imagine myself on someone else's good list. Often when I am struggling with something I worry someone is visualizing me on their bad list.
Over time it has made me question if my short list is as good as I think it is. Maybe I am biased and only observed good/bad work for people on either list. Also people change over time and take on new responsibilities that change their work.
Over time it has made me question if my short list is as good as I think it is. Maybe I am biased and only observed good/bad work for people on either list. Also people change over time and take on new responsibilities that change their work.
I am 100% certain that I am a member of the “naughty” list for some ex-colleagues and the “nice” list for other ex-colleagues. ;)
It's pretty easy to tell when you're managing a team and familiar with the code. You can't quantify it explicitly, but it can be quite clear.
In my experience the top 20% of employees do 80% of the work
In my experience the top 20% of employees do 80% of the work
> In my experience the top 20% of employees do 80% of the work
That would make them 16x as productive compared to the bottom 80% who does 20% of the work. So 10x seems to be roughly the same thing as the 80/20 rule.
That would make them 16x as productive compared to the bottom 80% who does 20% of the work. So 10x seems to be roughly the same thing as the 80/20 rule.
I think the ideal is to have both in some proportion: like 10% slow coders dedicated to maintaining the structure of the application. And 90% people who can churn out features quickly
I agree but I would say: The ratio varies depending on the kind of product and especially on where you want to be on the spectrum of speed, security, safety, time to client...
It's 10x the worst not average developer so it becomes 2x the average.
Besides a 10x developer cannot be uncovered through a leetcode test or a coding test. A 10x developer pushes back on requirements, bridges gaps.. identifies issues early.. it's about making things happen not typing faster or knowing the stack well
Besides a 10x developer cannot be uncovered through a leetcode test or a coding test. A 10x developer pushes back on requirements, bridges gaps.. identifies issues early.. it's about making things happen not typing faster or knowing the stack well
It's definitely 10x the average. But exact numbers are going to be subject to debate.
The worst are net negative productivity
The worst are net negative productivity
I've found that coding interviews just prove that you can write rushed, crappy code to solve a problem.
> Crappy code
Being able to come up with an algorithm in a reasonable time doesn’t make it crappy. No wonder you can’t pass those.
Being able to come up with an algorithm in a reasonable time doesn’t make it crappy. No wonder you can’t pass those.
I mean that's what we do, right?
some people can't even do that.
> I've had plenty of people crush the theory portion of the interview and be poor performers on the job (slow coding, low volume output)
I think the reason for that observation relates to why these interview tests are generally a waste of time.
Based on the tests that I've seen, they are 'recall lots of useless information' tests. That means it's actually possible to study for these interview tests (if you have the desire) and if you can ace the test by just spending time memorizing answers.
However, 'slow coding' has very little to do with coding and everything to do with problem solving. You only learn how to problem solve through lived experience and many people never learn that skill.
These interview tests generally fail only because they're not actually testing for the skills needed to do the job.
I think the reason for that observation relates to why these interview tests are generally a waste of time.
Based on the tests that I've seen, they are 'recall lots of useless information' tests. That means it's actually possible to study for these interview tests (if you have the desire) and if you can ace the test by just spending time memorizing answers.
However, 'slow coding' has very little to do with coding and everything to do with problem solving. You only learn how to problem solve through lived experience and many people never learn that skill.
These interview tests generally fail only because they're not actually testing for the skills needed to do the job.
Anything that anyone can find in a couple of seconds on a search engine is probably not worth an interview question. And that's valid for the interviewer as well don't ask something that you could find by looking at the resume.
This.
I don't get why it's so hard to create this type of interviews...
> slow coding
What does that even mean?
What does that even mean?
You have a task to implement a new product feature and it takes you 1 week instead of 1 day.
In an interview context, I've seen cases where somebody fully understands the logic underlying the solution, yet it takes them 30m to write it, while another person can do it in 5m or less.
Essentially how long it takes to translate from mental understanding to working code
In an interview context, I've seen cases where somebody fully understands the logic underlying the solution, yet it takes them 30m to write it, while another person can do it in 5m or less.
Essentially how long it takes to translate from mental understanding to working code
> Essentially how long it takes to translate from mental understanding to working cod
Congrats, you just invented whiteboard interview.
Congrats, you just invented whiteboard interview.
Whiteboard interview has nothing to do with coding proficiency. Writing pseudo code of an algorithm by hand does not tell you much at all about how proficient somebody is as a coder.
Coding is the act of writing code, not ideating an algorithmic solution. Its almost completely orthogonal to computer science problem solving.
The principles of high quality code (from a design perspective) are also completely orthogonal to runtime complexity of an algorithm.
Coding is the act of writing code, not ideating an algorithmic solution. Its almost completely orthogonal to computer science problem solving.
The principles of high quality code (from a design perspective) are also completely orthogonal to runtime complexity of an algorithm.
Before we criticize the current interview format and propose alternatives, we need to understand how we got here first. This is my understanding of what happened (I wasn't there for most of this!).
Leetcode-style interviews became popular in the mid 00s, primarily because they were used by hot tech companies of the time. The thing to understand is that at that time, the idea of asking people to write code during an interview what sort of revolutionary. Prior to that interviews had little structure. It wasn't unusual for the hiring manager to make the decision, some times based on credentials, recommendations, or trivial-like questions.
This type of interview became wildly popular because it allowed budding unicorns to hire programmers of high quality at scale. The process was less biased than the alternative, reproducible and scalable. Here you have two blog posts [1][2] that show the line of thought at the time.
The reality is that big tech has elevated leetcode type interview to an art. They have reached a near local optimal through years of experiments and refinements. It is working well for them so they don't have the need to take big risks such as completely revamping their hiring process.
I love the topic of hiring and interviewing and I'd love to truly get at the bottom of which method works best. I like this article because it explicitly calls out shortcomings with typical alternatives that are not usually mentioned. I hope in the future a new crop of unicorns can take these practices to the next level and do a fair comparison.
[1] https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... [2] https://sites.google.com/site/steveyegge2/five-essential-pho...
Leetcode-style interviews became popular in the mid 00s, primarily because they were used by hot tech companies of the time. The thing to understand is that at that time, the idea of asking people to write code during an interview what sort of revolutionary. Prior to that interviews had little structure. It wasn't unusual for the hiring manager to make the decision, some times based on credentials, recommendations, or trivial-like questions.
This type of interview became wildly popular because it allowed budding unicorns to hire programmers of high quality at scale. The process was less biased than the alternative, reproducible and scalable. Here you have two blog posts [1][2] that show the line of thought at the time.
The reality is that big tech has elevated leetcode type interview to an art. They have reached a near local optimal through years of experiments and refinements. It is working well for them so they don't have the need to take big risks such as completely revamping their hiring process.
I love the topic of hiring and interviewing and I'd love to truly get at the bottom of which method works best. I like this article because it explicitly calls out shortcomings with typical alternatives that are not usually mentioned. I hope in the future a new crop of unicorns can take these practices to the next level and do a fair comparison.
[1] https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... [2] https://sites.google.com/site/steveyegge2/five-essential-pho...
> the idea of asking people to write code during an interview what sort of revolutionary
And I think it's a great idea, personally I would never consider working anywhere that hired devs without seeing them write some code. But my problem is with the types of questions asked at many companies, specifically the types that require weeks (or months!) of prepping.
During my last job search one interview that stood out (positively) was a problem around parsing some HTTP headers (it started simple and then had layers of complexity added as I solved each one). It was honestly one of the best questions I've seen in an interview as it requires the candidate to be able to write code and solve a problem but without requiring/expecting the candidate to have prepped beforehand to learn (or brush up on) theoretical stuff far removed from the types of problems we actually solve on a daily bases (evidence of this disconnect is demonstrated by the fact people need to prep).
And I think it's a great idea, personally I would never consider working anywhere that hired devs without seeing them write some code. But my problem is with the types of questions asked at many companies, specifically the types that require weeks (or months!) of prepping.
During my last job search one interview that stood out (positively) was a problem around parsing some HTTP headers (it started simple and then had layers of complexity added as I solved each one). It was honestly one of the best questions I've seen in an interview as it requires the candidate to be able to write code and solve a problem but without requiring/expecting the candidate to have prepped beforehand to learn (or brush up on) theoretical stuff far removed from the types of problems we actually solve on a daily bases (evidence of this disconnect is demonstrated by the fact people need to prep).
I think this is a great example of the kind of implicit expectations that always pervade these discussions. I had a similar interview in my last job search, and for me it stood out negatively. At the time I’d never parsed any part of an HTTP response in my professional career, so I made some mistakes that look kinda silly if you know what you’re doing, and it was clear that the interviewer didn’t believe me when I explained that this isn’t a problem that comes up often in my field. I don’t want it to sound like I’m complaining, it’s not a big deal, but the interviews a web developer sees as neat and low-prep are exactly the ones I need to prepare extra for.
Was it really the 00-s?
I had two jobs at the turn of the decade and they were conversations about topics such as “where would I go to find authoritative information on x” (google is not the answer) and approaches to troubleshooting.
Granted, I’m a script monkey (sysadmin) but some level of automation has always been part of the job.
I’ve had precisely 1 leet-code style interview (2hours for 16 problems) and I couldn’t tolerate the pressure.
The interview I had at google was much more humane than what people are attempting to cargo cult.
I had two jobs at the turn of the decade and they were conversations about topics such as “where would I go to find authoritative information on x” (google is not the answer) and approaches to troubleshooting.
Granted, I’m a script monkey (sysadmin) but some level of automation has always been part of the job.
I’ve had precisely 1 leet-code style interview (2hours for 16 problems) and I couldn’t tolerate the pressure.
The interview I had at google was much more humane than what people are attempting to cargo cult.
This is probably the best take in the entire comment section.
I see whole bunch of people complaining about Leetcode (LC) style coding questions, but I don't really see any alternative that is as good as LC. LC interviews solve all the following criteria better than other interview formats in aggregate.
- Objectivity: can you evaluate candidates as objectively as possible as opposed to subjectivity?
- Scalability: can you evaluate many candidates fast with high throughput?
- Accuracy: can you filter out bad candidates? Yes, the filter is often restrictive to the point where good candidates also get filtered out unfortunately.
- Meritocratic: can you make sure the process is assessed based on the merit of solving LC questions well as opposed to having connection to someone you know at the company or having gone to some prestigious school or other companies?
Leetcode format does good in all criteria above, or at least better than other formats below in aggregate.
1. Take home assignments.
- Objectivity: bad
- Scalability: bad. Once the question leaks, it's harder to come up with another question easily. Also doesn't scale for candidate side either. As a rule of thumb, I always decline interviews that require take home because it's useless to try hard as this is not applicable to interview at other companies. ROI is extremely low for candidates.
- Accuracy: good.
- Meritocratic: good
2. Coding questions with heavy emphasis on practicality (Stripe & Square)
I was pretty impressed with questions from Stripe and Square. They created problems that are heavily related to string, array, hashmap manipulation, searching, replacing etc using regex and such. If you are not into LC questions, you might prefer this, but I actually found some of these questions harder than dynamic programming questions I got from Google.
- Objectivity: good
- Scalability: bad. Once the question leaks, it's harder to come up with another question easily.
- Accuracy: good.
- Meritocratic: good
3. Knowledge / Trivia questions about programming language or framework
This might make sense a bit if you are specifically looking for iOS, Android or web developer but it still will need to be paired with LC questions.
- Objectivity: good
- Scalability: good
- Accuracy: bad. Candidates can memorize and recite these answers to the questions. Also too many good candidates actually do not memorize the details about programming language or framework. Also it can easily obtained from just Googling so people can cheat easily.
- Meritocratic: good
4. Pure behavioral
- Objectivity: bad
- Scalability: good
- Accuracy: bad
- Meritocratic: bad
5. Ex-coworker
- Objectivity: bad
- Scalability: bad
- Accuracy: good. If you've worked with the candidate before, you'll have a pretty good grasp of whether or not he/she can do the job well.
- Meritocratic: bad
I see whole bunch of people complaining about Leetcode (LC) style coding questions, but I don't really see any alternative that is as good as LC. LC interviews solve all the following criteria better than other interview formats in aggregate.
- Objectivity: can you evaluate candidates as objectively as possible as opposed to subjectivity?
- Scalability: can you evaluate many candidates fast with high throughput?
- Accuracy: can you filter out bad candidates? Yes, the filter is often restrictive to the point where good candidates also get filtered out unfortunately.
- Meritocratic: can you make sure the process is assessed based on the merit of solving LC questions well as opposed to having connection to someone you know at the company or having gone to some prestigious school or other companies?
Leetcode format does good in all criteria above, or at least better than other formats below in aggregate.
1. Take home assignments.
- Objectivity: bad
- Scalability: bad. Once the question leaks, it's harder to come up with another question easily. Also doesn't scale for candidate side either. As a rule of thumb, I always decline interviews that require take home because it's useless to try hard as this is not applicable to interview at other companies. ROI is extremely low for candidates.
- Accuracy: good.
- Meritocratic: good
2. Coding questions with heavy emphasis on practicality (Stripe & Square)
I was pretty impressed with questions from Stripe and Square. They created problems that are heavily related to string, array, hashmap manipulation, searching, replacing etc using regex and such. If you are not into LC questions, you might prefer this, but I actually found some of these questions harder than dynamic programming questions I got from Google.
- Objectivity: good
- Scalability: bad. Once the question leaks, it's harder to come up with another question easily.
- Accuracy: good.
- Meritocratic: good
3. Knowledge / Trivia questions about programming language or framework
This might make sense a bit if you are specifically looking for iOS, Android or web developer but it still will need to be paired with LC questions.
- Objectivity: good
- Scalability: good
- Accuracy: bad. Candidates can memorize and recite these answers to the questions. Also too many good candidates actually do not memorize the details about programming language or framework. Also it can easily obtained from just Googling so people can cheat easily.
- Meritocratic: good
4. Pure behavioral
- Objectivity: bad
- Scalability: good
- Accuracy: bad
- Meritocratic: bad
5. Ex-coworker
- Objectivity: bad
- Scalability: bad
- Accuracy: good. If you've worked with the candidate before, you'll have a pretty good grasp of whether or not he/she can do the job well.
- Meritocratic: bad
You miss an important criterion though which is “does my interview has anything to do with the actual work?”, and lc fails miserably. And this one criterion probably classifies lc even lower than take home test or coding questions.
> 1. Take home assignments.
> Meritocratic: good
I'd go with "iffy": because of the "take home" part there's no guarantee it's the candidate that did the work. This can probably be determined with a follow-up conversation during the in-person part, but without such an addition I don't see this format as "good" on this metric.
> Meritocratic: good
I'd go with "iffy": because of the "take home" part there's no guarantee it's the candidate that did the work. This can probably be determined with a follow-up conversation during the in-person part, but without such an addition I don't see this format as "good" on this metric.
If I have my Jr. Dev a hard problem to work on and they said "breathe over my shoulder for 30 minutes and I'll have the answer for you" I would be having a serious corrective chat with them at our next meeting. No one works that way. Coding interviews are 10 percent about coding and 90 anxiety management... Which to be clear is important, but let's be honest about what we're filtering for in these things.
I don't even think it's as clever as testing for anxiety. I think they have to be there watching your back to make sure you're not cheating - the problems they give are easily searchable online.
As a by product they will also test for candidates who are super good under stress (or drilled Leetcode so hard there's no reason to be in stress).
If interviews make you so anxious you can't think then you just have to practice more (I mean do more actual interviews). What's your alternative?
Best I can think of is take home tests but I wouldn't want to rely entirely on that, and they have their own problems too.
Best I can think of is take home tests but I wouldn't want to rely entirely on that, and they have their own problems too.
I used to defend coding interviews, until I had an interview about a month ago, and the leetcode questions came off as kind of insulting.
I have a decade of experience, working as a senior and staff engineer at megacorporations, have experience as a research scientist, am in a PhD program for computer science research, but lets just double check that I know how to use a hash table.
I have a decade of experience, working as a senior and staff engineer at megacorporations, have experience as a research scientist, am in a PhD program for computer science research, but lets just double check that I know how to use a hash table.
I can tell you right now I have seen 3-4 principal “architects” who don’t know how to write simple tests and all of them were external hires. I absolutely respect any company that requires experienced hires to at a minimum understand how a hash table works. It’s embarrassing when a principal engineer can’t evaluate a simple design because they’ve been “leading” for the past few years. Finally I know at least one person who gets hired on at a principal level, immediately seeks out forward looking projects with no defined success criteria and leaves a year or two in, rinse and repeat at various companies. It’s a waste of a money (400k-500k/year) and a waste of everyone’s time who has to onboard him.
Not that this would apply in your case (I don't know you), but in general there are plenty of well-credentialed people that are poor engineers (imo), or who are a good engineer but a poor fit for that company/team/position. And there are people that simply lie on their resume.
If it is standard at the company to give all candidates that same question, this is actually a good way to reduce bias in hiring decisions. Otherwise, you might bias toward hiring, say, people with college degrees or people who worked at large corporations, for example. And this might unintentionally bias against certain demographics that obtain college degrees at lower rates (for example).
If it is standard at the company to give all candidates that same question, this is actually a good way to reduce bias in hiring decisions. Otherwise, you might bias toward hiring, say, people with college degrees or people who worked at large corporations, for example. And this might unintentionally bias against certain demographics that obtain college degrees at lower rates (for example).
But many senior candidates actually can’t code anymore.
Also, typically the point of these questions is to see if the candidate can solve a novel problem not if they can use a hash table.
Also, typically the point of these questions is to see if the candidate can solve a novel problem not if they can use a hash table.
Ya know, I've heard this repeatedly over the past 15 years, but I've never actually encountered this mythical "senior programmer who can't code." Never worked with one. Never interviewed one. Never met one at a meetup. I think it's just a boogeyman used to frighten hiring managers.
I used to be an infra engineer. Can’t code (just never made the time to buckle down and grind leetcode or Project Euler out, bit of personal development regret on my part), can glue components together with code (have open source projects I maintain that reflect this), but not an SWE, and the expectation seemed to be you’re either a SWE or an SRE (which is just an SWE in an on call rotation). It taught me to leave for infosec/risk/compliance (more comp, no code or on call expectation).
People out there who can’t code interviewing for SWE roles, I can’t speak to, but there are definitely folks being asked to be fluent in code for non SWE roles who aren’t SWEs, so it makes sense everyone gets put through the leetcode grinder rubric to calibrate their level. I don’t think this is about misrepresentation, but expectation misalignment and a crude tool being necessary to evaluate human skills against a baseline.
(nearing 40, 22 years in tech, getting out asap, ymmv)
People out there who can’t code interviewing for SWE roles, I can’t speak to, but there are definitely folks being asked to be fluent in code for non SWE roles who aren’t SWEs, so it makes sense everyone gets put through the leetcode grinder rubric to calibrate their level. I don’t think this is about misrepresentation, but expectation misalignment and a crude tool being necessary to evaluate human skills against a baseline.
(nearing 40, 22 years in tech, getting out asap, ymmv)
I interviewed a while ago at a well rated (but not MANGA) company. I was livid when the interviewer decided to move to an easier question and was skeptical when I “solved” in-order traversal of a binary tree without thinking. “Oh, you’ve seen this problem before?”
“Yeah; a few weeks into college…”
I’m a staff engineer and tech lead at a MANGA company. I code in up to 5 languages a day. I’ve written Network stacks, compression software, virtualization software, and the backbone of multiple PaaS platforms. And they were focused on whether I knew niche algorithms (apparently based on EXT3) and other stupid stuff.
I’m a staff engineer and tech lead at a MANGA company. I code in up to 5 languages a day. I’ve written Network stacks, compression software, virtualization software, and the backbone of multiple PaaS platforms. And they were focused on whether I knew niche algorithms (apparently based on EXT3) and other stupid stuff.
Literally had the same thing. I've lead large scale software projects for a critical service at a FAANG, and the interviewer will just nit pick on some random detail. Very odd feeling failing an interview like that and then going into work on Monday and banging out software on a highly competitive team
> a MANGA company
That's a new one to me, what's it made of?
(All that's coming to mind is I presume totally unrelated: https://en.wikipedia.org/wiki/Manga )
That's a new one to me, what's it made of?
(All that's coming to mind is I presume totally unrelated: https://en.wikipedia.org/wiki/Manga )
It's FAANG but rearranged and using Meta's new name instead of Facebook.
It's not a boolean can/cannot code. It's a cannot code at the level expected of the seniority.
I interview tons of software engineering candidates (pre pandemic it was at least one or two a week). At least half of my senior candidates fail basic phone screener questions.
These are questions that reflect something they'd be doing day to day in the role, but also are incredibly standard in our field.
The issue in most of them is that they've taken their support network for granted. The tons of developers before them who laid groundwork, and the others currently who take care of many tasks they consider junior.
Well that leaves them in a weird spot where they've not actually done a lot of the foundational work their predecessors set up for them, and they've let a lot of their skills atrophy by delegating to juniors. They'll even say as much and say "oh so and so does that, or they had it set up and it's great"
But then what am I hiring them for? Certainly they're not bringing a wealth of programming ability over, so maybe soft skills? That's hard to hedge bets on for a SWE, maybe as an product manager or something.
I interview tons of software engineering candidates (pre pandemic it was at least one or two a week). At least half of my senior candidates fail basic phone screener questions.
These are questions that reflect something they'd be doing day to day in the role, but also are incredibly standard in our field.
The issue in most of them is that they've taken their support network for granted. The tons of developers before them who laid groundwork, and the others currently who take care of many tasks they consider junior.
Well that leaves them in a weird spot where they've not actually done a lot of the foundational work their predecessors set up for them, and they've let a lot of their skills atrophy by delegating to juniors. They'll even say as much and say "oh so and so does that, or they had it set up and it's great"
But then what am I hiring them for? Certainly they're not bringing a wealth of programming ability over, so maybe soft skills? That's hard to hedge bets on for a SWE, maybe as an product manager or something.
I think actually whats happening is some people are just not good at interviewing on the spot. And people are looking for gotchas in these interviews, "He didnt know how to use breadth first search?!?!?!", when the senior engineers daily life is probably more like a surgeon operating on a highly convoluted large scale code base full of business logic.
That may be true for some other interviews, but like I said, I ask questions based on what we'd be doing day to day. I don't care about CS theory jargon or language gotchas.
Again, at least half of software engineers I talk to for the screening round are not strong developers for the reasons I mentioned above.
Again, at least half of software engineers I talk to for the screening round are not strong developers for the reasons I mentioned above.
Be grateful! They definitely exist.
At my last company, I joined as their lead developer and discovered they already had another developer in the pipeline they were thinking of hiring. They thought the guy was super senior, and when I read his CV sure enough he came across that way. He emphasized on his CV that he was not only a Java expert but an expert on the internals of HotSpot itself.
Probably he'd been saying that for years, and given how many firms use the JVM it's probably a great line. Unlucky for him he got an interviewer (me) who actually was an expert in the HotSpot internals and liked to relax by reading its source code. Guess what? He knew nothing about the JVM, not even stuff you'd learn by reading the user guide. I got the sense he'd read one or two popular press articles about it 20 years ago, put it on his CV and never updated it.
So we progress to some coding. He can't code. He struggles with even the basics of starting a new program in an editor and compiling it.
The guy was demanding a huuuge salary and great perks. After I wrote up what I'd found from the coding interview he was quietly dropped from the pipeline. They'd been about to accept because they were so dazzled by his claims but weren't checking them.
The "knew it once maybe sorta" problem is common, especially with C++ where lots of people were struggling with it in the 90s and then jumped to Java/C# as soon as they came out. But they still list C++ on their CV. Ask them to write a C++ program (that does anything at all) and they'll just refuse or state point blank they'd feel really uncomfortable doing that.
Lots of people lie about their experience unfortunately. They probably don't think of it as lying. They just write down every possible thing they've ever thought or read about and exaggerate, gambling that they'll never be called on it. And, they're probably right. That's why coding interviews took off.
At my last company, I joined as their lead developer and discovered they already had another developer in the pipeline they were thinking of hiring. They thought the guy was super senior, and when I read his CV sure enough he came across that way. He emphasized on his CV that he was not only a Java expert but an expert on the internals of HotSpot itself.
Probably he'd been saying that for years, and given how many firms use the JVM it's probably a great line. Unlucky for him he got an interviewer (me) who actually was an expert in the HotSpot internals and liked to relax by reading its source code. Guess what? He knew nothing about the JVM, not even stuff you'd learn by reading the user guide. I got the sense he'd read one or two popular press articles about it 20 years ago, put it on his CV and never updated it.
So we progress to some coding. He can't code. He struggles with even the basics of starting a new program in an editor and compiling it.
The guy was demanding a huuuge salary and great perks. After I wrote up what I'd found from the coding interview he was quietly dropped from the pipeline. They'd been about to accept because they were so dazzled by his claims but weren't checking them.
The "knew it once maybe sorta" problem is common, especially with C++ where lots of people were struggling with it in the 90s and then jumped to Java/C# as soon as they came out. But they still list C++ on their CV. Ask them to write a C++ program (that does anything at all) and they'll just refuse or state point blank they'd feel really uncomfortable doing that.
Lots of people lie about their experience unfortunately. They probably don't think of it as lying. They just write down every possible thing they've ever thought or read about and exaggerate, gambling that they'll never be called on it. And, they're probably right. That's why coding interviews took off.
> They probably don't think of it as lying. They just write down every possible thing they've ever thought or read about and exaggerate, gambling that they'll never be called on it.
Well, job postings are infamous for listing a ridiculous number of "requirements" that aren't really requirements. Those who are familiar with hiring know that job postings themselves are lies and encourage people to ignore the requirements and apply anyway. Even the people who work for the hiring company say this! So it's a two-way street. There are already perverse incentives created by the hiring companies. And we also know that resumes are screened for specific buzzwords. You may not be recruited or interviewed unless unless your resume has the right buzzwords.
Well, job postings are infamous for listing a ridiculous number of "requirements" that aren't really requirements. Those who are familiar with hiring know that job postings themselves are lies and encourage people to ignore the requirements and apply anyway. Even the people who work for the hiring company say this! So it's a two-way street. There are already perverse incentives created by the hiring companies. And we also know that resumes are screened for specific buzzwords. You may not be recruited or interviewed unless unless your resume has the right buzzwords.
They are out there, but usually in a 3~4 step process they get filtered out before the coding interview.
I did a bunch of really quick 15~20 min pre-interviews to check if the person was entering the right process before we both wasted hours of our time. Some who were supposed to have coded and deployed whole applications didn’t know how their versioning system worked. Or they reviewed PR/MRs a lot and tweaked a few lines here and there but had little practical knowledge of issues you’d regularly hit when developing, and only heard of popular tools and resources by name.
We could still had them onboard and give them time to get up to speed, but it’s harder to do when they weren’t upfront about it and you’re there trying to guess what else they’re bulshitting on their resume.
I did a bunch of really quick 15~20 min pre-interviews to check if the person was entering the right process before we both wasted hours of our time. Some who were supposed to have coded and deployed whole applications didn’t know how their versioning system worked. Or they reviewed PR/MRs a lot and tweaked a few lines here and there but had little practical knowledge of issues you’d regularly hit when developing, and only heard of popular tools and resources by name.
We could still had them onboard and give them time to get up to speed, but it’s harder to do when they weren’t upfront about it and you’re there trying to guess what else they’re bulshitting on their resume.
I've interviewed a few. The most memorable being someone who struggled with what a linked list is, and thought two nested loops over the same number of times was a linear-time algorithm.
I have. Supposedly an experienced fellow but put out code that would embarrass an intern (and a bad intern at that).
So, they could code, just not up to your standard.
The resulting code didn't build either so, no.
Does the industry even need engineers who can code?
This is a hyperbolic rhetorical question but consider how much of modern work really is CRUD API gluing, and how much of that work is not rigorous engineering, but tedious implementation as per documentation and wading through hastily-thrown together legacy cruft written during previous crunch time.
How much of that work involves novel programming and memorized understanding? Most of it requires study of existing codebases, not given a blank REPL to hash out something new.
This is a hyperbolic rhetorical question but consider how much of modern work really is CRUD API gluing, and how much of that work is not rigorous engineering, but tedious implementation as per documentation and wading through hastily-thrown together legacy cruft written during previous crunch time.
How much of that work involves novel programming and memorized understanding? Most of it requires study of existing codebases, not given a blank REPL to hash out something new.
OK. What novel problem is the senior engineer solving using a hash table in a coding interview?
Then tell your company to not to reach out to recruit senior & experienced candidates if you aren't sure if they can code from looking at their experience, github, etc. I'm personally very tired of answering recruiters & hiring managers for specialized roles, then the interviews end up being some unrelated CS quiz, mostly from people with less experience than I have.
I once had an interview that told me to “design Twitter”. I drew some diagrams on the whiteboard, and the topic of the database came up.
I asked “do we want to have consistency or availability for this?”, to which the interviewer said “I want both”, to which I said “umm, I’m pretty sure you can’t have both, CAP theorem and whatnot”. We spent two minutes arguing which led to me taking out my phone and pulling up the CAP Theorem Wikipedia page, and eventually the interviewer begrudgingly admitted I was right.
I surprisingly did get an offer after that interview but I found it a little annoying to be interviewed by a person with less experience than me who I have to argue with because his understanding of computer science is wrong.
I asked “do we want to have consistency or availability for this?”, to which the interviewer said “I want both”, to which I said “umm, I’m pretty sure you can’t have both, CAP theorem and whatnot”. We spent two minutes arguing which led to me taking out my phone and pulling up the CAP Theorem Wikipedia page, and eventually the interviewer begrudgingly admitted I was right.
I surprisingly did get an offer after that interview but I found it a little annoying to be interviewed by a person with less experience than me who I have to argue with because his understanding of computer science is wrong.
Well, that's a really great sign you don't want to work there. If I don't have a good interview experience I don't want the job.
Yeah I can see why it seems insulting but have you ever interviewed a lot of people.
When I started interviewing I started off by jumping into what I considered to be not insulting questions, but I very quickly learnt how awkward that can be if the candidate just had no clue.
Now I start with for loops and preface it with "apologies if this seems too simple" or something like that.
You would be surprised how many people can't do a simple for loop. I mean really simple.
When I started interviewing I started off by jumping into what I considered to be not insulting questions, but I very quickly learnt how awkward that can be if the candidate just had no clue.
Now I start with for loops and preface it with "apologies if this seems too simple" or something like that.
You would be surprised how many people can't do a simple for loop. I mean really simple.
Nowadays when I interview people, I try and come up with an example that feels like a "real" problem you'd realistically solve on the job, but isn't too hard, usually something involving building an index or some basic concurrency. It's still not perfect, but I do work pretty hard to try and make the problems feel fair and actually something you'd do on the job. I also generally don't concern myself with compiler/syntax errors unless they're extremely egregious, since I think those errors can typically be explained by nerves and realistically won't be much of a blocker in a real job.
I really hate the contrived problems that interviewers pull from HackerRank and its ilk. Even if you could glean useful information from watching someone program a knight hopping around a phone dial pad (a real problem I've gotten, which I'm highly skeptical of its utility), it comes off as pretty irritating to the interviewee. You're not solving little programming riddles on day to day work, you're solving engineering problems, and I think that these leetcode questions don't reflect that.
I think there's evidence of this too; you can study for these leetcode questions and get better at them, so what exactly are we testing? You're not testing the person's experience, you're testing how long they spent on hackerrank the night before, or how many interviews they had recently.
I really hate the contrived problems that interviewers pull from HackerRank and its ilk. Even if you could glean useful information from watching someone program a knight hopping around a phone dial pad (a real problem I've gotten, which I'm highly skeptical of its utility), it comes off as pretty irritating to the interviewee. You're not solving little programming riddles on day to day work, you're solving engineering problems, and I think that these leetcode questions don't reflect that.
I think there's evidence of this too; you can study for these leetcode questions and get better at them, so what exactly are we testing? You're not testing the person's experience, you're testing how long they spent on hackerrank the night before, or how many interviews they had recently.
Sorry you have to go through peasant ropes, your highness.
I jumped through the hoops plenty of times. Just because I have to do it doesn’t mean I have to like it, or think it’s a good use of anyones time.
I think one can fairly accurately determine my level of skill from my resume, and thus I don’t think that the leetcode interviews are all that useful. I keep hearing about senior engineers who can’t code, and maybe I’m just lucky, but that really has not been my experience when interviewing, and I don’t think “spent the previous night practicing on hackerrank” really gives you any useful information.
I think one can fairly accurately determine my level of skill from my resume, and thus I don’t think that the leetcode interviews are all that useful. I keep hearing about senior engineers who can’t code, and maybe I’m just lucky, but that really has not been my experience when interviewing, and I don’t think “spent the previous night practicing on hackerrank” really gives you any useful information.
Instead of endlessly rehashing the controversy of coding interviews, has anyone considered just how engineers are expected to be able to understand the material on the interviews?
Should it be from university CS programs? Which are supposed not to be vocational? And would discriminate against those who are not degree holders?
Should it be from internships, which are nowadays subject to the same Leetcode examinations as actual work?
Should it be from junior developer roles, which are an endangered species, as every business in desperate need for seniors, and lack the patience to train?
Maybe open source can be a sort of free apprenticeship to teach developers these good practices?
Or maybe grinding away at Leetcode, Project Euler, and arbitrary coding puzzles is really the only pedagogical solution.
Should it be from university CS programs? Which are supposed not to be vocational? And would discriminate against those who are not degree holders?
Should it be from internships, which are nowadays subject to the same Leetcode examinations as actual work?
Should it be from junior developer roles, which are an endangered species, as every business in desperate need for seniors, and lack the patience to train?
Maybe open source can be a sort of free apprenticeship to teach developers these good practices?
Or maybe grinding away at Leetcode, Project Euler, and arbitrary coding puzzles is really the only pedagogical solution.
> Or maybe grinding away at Leetcode, Project Euler, and arbitrary coding puzzles is really the only pedagogical solution.
It is how people pass these tests, but it’s pretty poor pedagogy.
It is how people pass these tests, but it’s pretty poor pedagogy.
I don't get your comment on junior developers.
We have no issue whatsoever hiring juniors (we hire about as many as mid-levels), but we don't have to settle for someone who doesn't have internships or projects and the openings can be closed in 2 weeks so it seems like nobody is hiring, but actually the bar has risen (and there are plenty of candidates who clear the bar).
We have no issue whatsoever hiring juniors (we hire about as many as mid-levels), but we don't have to settle for someone who doesn't have internships or projects and the openings can be closed in 2 weeks so it seems like nobody is hiring, but actually the bar has risen (and there are plenty of candidates who clear the bar).
That incredible dearth of junior positions is stranding a lot of new grads and juniors, as seen in discussions like this:
https://news.ycombinator.com/item?id=31537711
https://news.ycombinator.com/item?id=31537711
> has anyone considered just how engineers are expected to be able to understand the material on the interviews?
I am confused - how is this question materially different from asking how engineers are expected to know what they will need in order to properly do their jobs? Just how are engineers expected to know programming languages, or domain-specific languages, or standard libraries, or platform apis? Is this what you are asking?
I am confused - how is this question materially different from asking how engineers are expected to know what they will need in order to properly do their jobs? Just how are engineers expected to know programming languages, or domain-specific languages, or standard libraries, or platform apis? Is this what you are asking?
You are unintentionally correct, most engineers don't really know their programming language, standard libraries or basic APIs. They find _some_ solution on the internet that's good enough and run with it. Sometimes that is fine, sometimes it is not.
I am indeed asking that as well, given the completely unorganized and nonstandardized state of the industry.
I used to hate leetcode-style coding interviews, but now I don't. The reason for this is a comment from a Google engineer. That comment changed my mind. In that comment they explained the motivation behind asking these type of questions and it all finally clicked for me.
Corporations are not testing your knowledge of algorithms per se. While that knowledge is obviously important, what's more important is how dedicated you are when it comes to reaching a goal. Essentially, they are testing your will power. You know the rules. Can you achieve success and not give up in the middle of the road by skipping, say, Dynamic Programming? They could've tested your level of dedication by asking to bake some fancy cake, but throwing CS-related questions just makes much more sense.
That reasoning completely changed my perspective about algo-heavy coding interviews.
Corporations are not testing your knowledge of algorithms per se. While that knowledge is obviously important, what's more important is how dedicated you are when it comes to reaching a goal. Essentially, they are testing your will power. You know the rules. Can you achieve success and not give up in the middle of the road by skipping, say, Dynamic Programming? They could've tested your level of dedication by asking to bake some fancy cake, but throwing CS-related questions just makes much more sense.
That reasoning completely changed my perspective about algo-heavy coding interviews.
> what's more important is how dedicated you are when it comes to reaching a goal. Essentially, they are testing your will power. You know the rules. Can you achieve success and not give up in the middle of the road by skipping, say, Dynamic Programming?
That's an interesting theory. And they pick the goal that happens to be best suited to fresh/current undergrads from Stanford who weren't working a draining job to put themselves through school, so can afford the additional person-months to achieve the gatekeeping that was set up by people like them, for people like them.
I and many other experienced software engineers have demonstrated perseverance and rising to the occasion of extremely challenging goals. But that theory suggests they're not interested in that. Maybe they're interested in the class shibboleth that was established when computers became less of a meritocracy for people with a passion for it, and more of a go-to status job for the upper middle class?
That's an interesting theory. And they pick the goal that happens to be best suited to fresh/current undergrads from Stanford who weren't working a draining job to put themselves through school, so can afford the additional person-months to achieve the gatekeeping that was set up by people like them, for people like them.
I and many other experienced software engineers have demonstrated perseverance and rising to the occasion of extremely challenging goals. But that theory suggests they're not interested in that. Maybe they're interested in the class shibboleth that was established when computers became less of a meritocracy for people with a passion for it, and more of a go-to status job for the upper middle class?
The thing is, the practical field is littered with countless different technologies & endless subjective opinions. We'd need 3,000 different tests to test everyone's practical knowledge, to let them pick between. Oh, you use Go and SolidJS, here's a test for you... and who is going to be able to properly give you a score other than works/doesn't work? This person likes functional style but this person is more OO... the base truth is: code is artisinal & subjective 98% of the time. Most code is soft. There's poor metrics to assess & judge it by. Many many many things work. What we do is cultural, is social, is subjective.
I'm not sure how to make this come through but I'm rather sympathetic. I think practical experience & can-do coder know-how count for most of the marbles. But I also think it's impressively hard to assess by, that most of what we do is just taste & path dependence on the specific techs we've seen that happened to leave good impressions on us: not science, not knowledge, not truth: most coding, most practice is almost entirely happenstance.
But again I want to empathize. Because I think it's cruel to disregard engineers that have seen so much. But I also think you're hyperbolizing overmuch & discrediting the discourse when you make your argument so sharply pointedly. Because there's some truth, absolutely, but it's also no-where near as lopsided. For example, you talk about fresh/current undergrads at Stanford. But this is a pretty regular, basic CS path that any university student should be familiar with. I only had two undergrad classes that really talked to algorithms (Algorithms and then Data structures; OS kind of somewhat), albeit there was some computational thinking already in play at that point. Understanding complexity & algorithms is understanding the basics, it really is. If you can't see time flowing, don't know the tally & impact of the work you're asking for, you're lacking knowledge to avoid a lot of bad decisions. This isn't made up fake shit taught only to the elite.
What's alluring about this system is that it has some hard facts & theory underneath it. It's not just opinion & engineering pop-culture. The answers don't change, the rules don't change, the material stays the same, and it's all rooted in being able to analyze and understand problems, rooted in comprehension. Being able to see & understand & analyze, being able to apply some basic principles: this is a pretty sure way to find people who will be able to Not Mess It Up, who are capable of looking, assessing, & navigating through scenarios computationally.
And last and perhaps most key to me: it's also material that someone with even a modest bit of aptitude can cram for. Pick up a book like "Cracking the Coding Interview" and you can semi-accurately reproduce the output of four years of undergrad in two or three months of occasional light practice & trying. Projects like Leetcode can get you the experience. You may feel bad that what you want yourself to be measured on doesn't count here, but to people trying to go get hired, it's enormously helpful to them to have well defined expectations, to have specific kinds of tests to expect & be able to prepare & study for. I see so many protests that it's not fair, that it favors only those with the luxury of time, but those views to me massively undermine how wonderfully vastly accessible it is that there's a known, well-described, well-supported subject-area we can study. The industry overflows with things to learn, study, & know, but here's something concrete & specific, which undergirds it all, which alone might not determine your coding skill, but which does indicate you at least have some raw basic intellectual capacity to go understand problems put before you & apply some sensible computational thinking to tackling them.
There's tons of things not tested for, but by having a well defined, technology-neutral set of computational thinking tests, based on actual science with objective, factual answers, rather a much wider corpus of future/current/legacy pop-engineering-of-the-day built on nothing but wishy-washy subjective opinion, I think we achieve one of the best possible wins we can against an enemy we both hate: class-based discrimination, cruel tests that reject outgroups.
I'm not sure how to make this come through but I'm rather sympathetic. I think practical experience & can-do coder know-how count for most of the marbles. But I also think it's impressively hard to assess by, that most of what we do is just taste & path dependence on the specific techs we've seen that happened to leave good impressions on us: not science, not knowledge, not truth: most coding, most practice is almost entirely happenstance.
But again I want to empathize. Because I think it's cruel to disregard engineers that have seen so much. But I also think you're hyperbolizing overmuch & discrediting the discourse when you make your argument so sharply pointedly. Because there's some truth, absolutely, but it's also no-where near as lopsided. For example, you talk about fresh/current undergrads at Stanford. But this is a pretty regular, basic CS path that any university student should be familiar with. I only had two undergrad classes that really talked to algorithms (Algorithms and then Data structures; OS kind of somewhat), albeit there was some computational thinking already in play at that point. Understanding complexity & algorithms is understanding the basics, it really is. If you can't see time flowing, don't know the tally & impact of the work you're asking for, you're lacking knowledge to avoid a lot of bad decisions. This isn't made up fake shit taught only to the elite.
What's alluring about this system is that it has some hard facts & theory underneath it. It's not just opinion & engineering pop-culture. The answers don't change, the rules don't change, the material stays the same, and it's all rooted in being able to analyze and understand problems, rooted in comprehension. Being able to see & understand & analyze, being able to apply some basic principles: this is a pretty sure way to find people who will be able to Not Mess It Up, who are capable of looking, assessing, & navigating through scenarios computationally.
And last and perhaps most key to me: it's also material that someone with even a modest bit of aptitude can cram for. Pick up a book like "Cracking the Coding Interview" and you can semi-accurately reproduce the output of four years of undergrad in two or three months of occasional light practice & trying. Projects like Leetcode can get you the experience. You may feel bad that what you want yourself to be measured on doesn't count here, but to people trying to go get hired, it's enormously helpful to them to have well defined expectations, to have specific kinds of tests to expect & be able to prepare & study for. I see so many protests that it's not fair, that it favors only those with the luxury of time, but those views to me massively undermine how wonderfully vastly accessible it is that there's a known, well-described, well-supported subject-area we can study. The industry overflows with things to learn, study, & know, but here's something concrete & specific, which undergirds it all, which alone might not determine your coding skill, but which does indicate you at least have some raw basic intellectual capacity to go understand problems put before you & apply some sensible computational thinking to tackling them.
There's tons of things not tested for, but by having a well defined, technology-neutral set of computational thinking tests, based on actual science with objective, factual answers, rather a much wider corpus of future/current/legacy pop-engineering-of-the-day built on nothing but wishy-washy subjective opinion, I think we achieve one of the best possible wins we can against an enemy we both hate: class-based discrimination, cruel tests that reject outgroups.
The earlier comment suggested that Leetcode was a litmus test for whether someone has will power beyond CS curriculum, and that's what I was responding to. Are you arguing a different motivation?
That was my first post in this thread.
I think computational thinking is a good objective measure, and is learnable. That comes first to me. I also think acquiring & practicing computational thinking demonstrates will power, a willingness & capacity to tackle something, and that that something is valuable. And it's an objective, common, topical, & agreed upon/expectable test, one that doesn't rely on socially convincing someone in 5 minutes you indeed have willpower.
If you only think of this as trying to get good at Leetcode, I think you risk ignoring the actual lessons. Leetcode is an ok practicing grounds, but what's being tested relies on a mix of computer science and problem solving, and just going to leetcode & trying to get good there will probably eventually work out, but you'll miss the computer science that would actually be helping you.
I think computational thinking is a good objective measure, and is learnable. That comes first to me. I also think acquiring & practicing computational thinking demonstrates will power, a willingness & capacity to tackle something, and that that something is valuable. And it's an objective, common, topical, & agreed upon/expectable test, one that doesn't rely on socially convincing someone in 5 minutes you indeed have willpower.
If you only think of this as trying to get good at Leetcode, I think you risk ignoring the actual lessons. Leetcode is an ok practicing grounds, but what's being tested relies on a mix of computer science and problem solving, and just going to leetcode & trying to get good there will probably eventually work out, but you'll miss the computer science that would actually be helping you.
How dedicated you are to do a task with no intrinsic value besides getting hired at companies with leetcode-style interviews. I don't think google is better off when everyone they hire has wasted hundreds of hours on a fundamentally useless skill rather than having used that time to learn different skills or just enjoy time with family and minimize the chance they burn out on the job at a later date.
There's other ways to tease out if someone is "dedicated to reaching a goal" like actually asking them questions about their life, daily routine, accomplishments etc. I think these are much better signals than "this person can implement many sorting algorithms and solve towers of hanoi or the egg drop puzzle without a google search to refresh their memory" with an N=1 sample size used to gauge how reliably they can do that. How many people if asked to do this on a job rather than an interview a year later would then go and implement it from memory without double checking they didn't forget some edge case on stackoverflow?
I know grinding leetcode is fundamentally useless because you will immediately start losing your ability to interview once you get hired. If you don't change jobs within a year you'll need to start studying all that crap again for the next interview.
There's other ways to tease out if someone is "dedicated to reaching a goal" like actually asking them questions about their life, daily routine, accomplishments etc. I think these are much better signals than "this person can implement many sorting algorithms and solve towers of hanoi or the egg drop puzzle without a google search to refresh their memory" with an N=1 sample size used to gauge how reliably they can do that. How many people if asked to do this on a job rather than an interview a year later would then go and implement it from memory without double checking they didn't forget some edge case on stackoverflow?
I know grinding leetcode is fundamentally useless because you will immediately start losing your ability to interview once you get hired. If you don't change jobs within a year you'll need to start studying all that crap again for the next interview.
> How many people if asked to do this on a job rather than an interview a year later would then go and implement it from memory without double checking they didn't forget some edge case on stackoverflow?
An excellent question.
If the interviewer is to ask such algorithms questions looked up on leetcode, etc, then they must be prepared for when the candidate asks if they also use it themselves on the job. If the interviewer admits they don't use it, then they also admitted that they don't know what they are looking for and really are wasting the candidate's time.
If it were me, I would look at open-source contributions (no hello-world or demo projects) where that is enough proof for me to evaluate an entry-level candidate and cut through the algorithms nonsense and ask questions to the candidate based on that which will save everyone time.
An excellent question.
If the interviewer is to ask such algorithms questions looked up on leetcode, etc, then they must be prepared for when the candidate asks if they also use it themselves on the job. If the interviewer admits they don't use it, then they also admitted that they don't know what they are looking for and really are wasting the candidate's time.
If it were me, I would look at open-source contributions (no hello-world or demo projects) where that is enough proof for me to evaluate an entry-level candidate and cut through the algorithms nonsense and ask questions to the candidate based on that which will save everyone time.
Yes, leetcode skills don't help every day. But I definitely enjoy & respect more fellow coders who have computational thinking skills, who can break down problems, who understand costs.
That this isn't the bulk of what we do doesn't change the fact that these challenges do test computational thinking acuity. Having the ability to see & speak computationally is a good skill, one that connects our day-to-day abstract practice with actual real processes. Being able to break down problems & analyze how to tackle them shows an objective ability to assess & work through problems. I want to work with people who can be clear, who can model & explain & step through situations.
And these skills are, generally, learnable, and relatively quickly. I disagree that these abilities fade, but yes, some re-familiarizing & re-training is probably important, especially because, as you point out, the sample size is indeed often N=1, and that's pretty wild.
That this isn't the bulk of what we do doesn't change the fact that these challenges do test computational thinking acuity. Having the ability to see & speak computationally is a good skill, one that connects our day-to-day abstract practice with actual real processes. Being able to break down problems & analyze how to tackle them shows an objective ability to assess & work through problems. I want to work with people who can be clear, who can model & explain & step through situations.
And these skills are, generally, learnable, and relatively quickly. I disagree that these abilities fade, but yes, some re-familiarizing & re-training is probably important, especially because, as you point out, the sample size is indeed often N=1, and that's pretty wild.
Sure, but just make sure you are Google before you try this.
If you are a no-name company paying whatever your HR department defines as "market rate" and you don't get hundreds of very qualified candidates for every role then interviewing like Google might not be a great idea.
If you are a no-name company paying whatever your HR department defines as "market rate" and you don't get hundreds of very qualified candidates for every role then interviewing like Google might not be a great idea.
They are testing also how docile you are. If you are ready to do useless tasks and go through hundreds of pages of the X-corp way of life then you have a chance to stay long and prosper.
Maybe I’d prefer to not dance like a monkey for employment.
So, they're actually testing to see if you have commitments outside of work.
"Great" filtering for parents or people caring for ailing family or any other non-work commitment.
I guess it's not ageism at all, right?
"Great" filtering for parents or people caring for ailing family or any other non-work commitment.
I guess it's not ageism at all, right?
> what's more important is how dedicated you are when it comes to reaching a goal.
That may sound good in theory, but in practice I've been rejected quite a few times for not delivering the correct or most optimal solution despite trying my best.
That may sound good in theory, but in practice I've been rejected quite a few times for not delivering the correct or most optimal solution despite trying my best.
I'll multiply that reasoning by a few orders of magnitude to show what's wrong with it. Would they hire Olympics winners for these positions?
One reason I'm self-employed now is to avoid this whole flaming hoop audition game. Thankfully my customers don't want to test me. At this point I'm tired of arguing against coding interviews, because the same arguments have already been repeated ad nauseam, and the linked article doesn't seem to me to add anything new. I just don't want to hear about any "talent shortage", because too many programmers are being systematically excluded out of fear of the dreaded "false positive".
I've been asked to interview. I'm not a great interviewer, but maybe OK. I try to figure out if they can work the tools. I'm pretty lax - interviews are stressful and missing a comma can take a minute, I don't like people twisting in the wind over a syntax error and will point it out if I'm pretty sure they'd get there. I try to decide if I can work with this person. I try to figure out if they'd be ok with how things work at my current employer.
I have two very memorable no's. They were for pretty senior/advanced IC roles. Juniors would get far more slack. I usually ask an expression evaluator question.
One person, used eval(string). Brilliant, absolutely brilliant. full points for answering, but how do we make sure the string is safe to eval? Long frustrating discussion around regexes that that maybe a junior dev is responsible for maintaining. Man, just write the parser. I know you can do it. Give me something that's something that's not _crazy_ to put into prod. Give me something that won't be a bug tarpit. Sorry man, I loved your answer. I hated your response to how this can be operationalized. Clearly super smart. Maybe I should have given the green light, but the response to criticism was so bad. There are lots of people I disagree with, that I'm happy to work with. I think I made the right choice, but I wonder from time to time.
Another person I absolutely would have loved to work with, and would have learned a ton. Very into formal methods, or slightly relaxed versions of verifying reliability. My organization at the time was pretty fast and loose. I believed they would be miserable. In the interview I told them I'd give them the green light, but they'd have a huge uphill battle.
it's so hard to tell. People are adaptable, small mismatches can be papered over, but big mismatches - it's so tough to say. Relaxing and being vulnerable on both sides is so hard to do well. Maybe I was just a jerk. I think I made the right calls, but it haunts me.
I have two very memorable no's. They were for pretty senior/advanced IC roles. Juniors would get far more slack. I usually ask an expression evaluator question.
One person, used eval(string). Brilliant, absolutely brilliant. full points for answering, but how do we make sure the string is safe to eval? Long frustrating discussion around regexes that that maybe a junior dev is responsible for maintaining. Man, just write the parser. I know you can do it. Give me something that's something that's not _crazy_ to put into prod. Give me something that won't be a bug tarpit. Sorry man, I loved your answer. I hated your response to how this can be operationalized. Clearly super smart. Maybe I should have given the green light, but the response to criticism was so bad. There are lots of people I disagree with, that I'm happy to work with. I think I made the right choice, but I wonder from time to time.
Another person I absolutely would have loved to work with, and would have learned a ton. Very into formal methods, or slightly relaxed versions of verifying reliability. My organization at the time was pretty fast and loose. I believed they would be miserable. In the interview I told them I'd give them the green light, but they'd have a huge uphill battle.
it's so hard to tell. People are adaptable, small mismatches can be papered over, but big mismatches - it's so tough to say. Relaxing and being vulnerable on both sides is so hard to do well. Maybe I was just a jerk. I think I made the right calls, but it haunts me.
It is nice to hear a defense of this style since there’s often so much complaining about it, and the tips to mitigate the downsides are useful.
Personally I don’t mind a whiteboard session or two during a loop but what’s wild to me is how, especially at big companies, you’re expected to do four or five of these to get an offer.
How often do these companies decide “well they understand when to use DFS and they can merge-sort a linked-list and they knew to use dynamic programming but we can’t hire them because they couldn’t remember how to implement a heap”?
Personally I don’t mind a whiteboard session or two during a loop but what’s wild to me is how, especially at big companies, you’re expected to do four or five of these to get an offer.
How often do these companies decide “well they understand when to use DFS and they can merge-sort a linked-list and they knew to use dynamic programming but we can’t hire them because they couldn’t remember how to implement a heap”?
The problem is that these problems are not even whiteboard problems anymore. They’re often strictly timed coding exercises on Coderpad or Karat with actual mock IDEs and REPLs expecting running code under strict expectations.
The more casual, flexible, subjective coding interview described by the OP is not the prevalent form of whiteboarding interviews, which has been superseded by the weeder version.
The more casual, flexible, subjective coding interview described by the OP is not the prevalent form of whiteboarding interviews, which has been superseded by the weeder version.
Do you have any suggestions for a simple online coding portal? I know some companies have used a simple Google Doc for this reason, but that seems far from ideal.
Coderpad and Hackerrank are pretty de rigeur
> you’re expected to do four or five of these to get an offer.
I agree that this is unreasonable but not for the reason that you mention.
Once you can pass this kind of interview, you can throw me 10 more and I will still pass them.
I agree that this is unreasonable but not for the reason that you mention.
Once you can pass this kind of interview, you can throw me 10 more and I will still pass them.
The idea of observing someone writing code is a freaking joke. It puts an enormous amount of pressure on the candidate, and it forces the candidate to think and talk deeply at the same time, which in and of itself is a skill that is not never for the job.
Coding interviews are a joke and the defense of them is even worse.
Edit: on further thought. I think I’d be ok with coding interviews if the interviewee got to bring in their own coding question and the interviewer had to do it as well. That way the they also knows the people they are working with can live up to their ideal of a good coder.
Edit: on further thought. I think I’d be ok with coding interviews if the interviewee got to bring in their own coding question and the interviewer had to do it as well. That way the they also knows the people they are working with can live up to their ideal of a good coder.
This is not a defense.
It's saying coding interviews are not spectacularly awful if you (thoughtfully) change them.
Great! That's what everyone complaining about them has been saying in one way or another all along, change them.
It's saying coding interviews are not spectacularly awful if you (thoughtfully) change them.
Great! That's what everyone complaining about them has been saying in one way or another all along, change them.
I upvoted this because it's well written, even though I disagree with the conclusions. I'm not sure the suggested alternative coding challenges are any better. The text preview one starts ok, but introducing HTML raises the complexity through the roof!
I wrote about my own experiences with coding challenges during my recent job search on my blog at https://blog.urth.org/2022/04/19/software-job-search-2022-re...
I'll summarize my conclusion about live coding challenges, which is that I'm not convinced that my performance on these challenges reflected any abstract skill I have. Instead, they mostly reflected the fact that the problems I was given were either nearly identical to work I've done in the past, or were similar enough to things I'd done recently that they felt pretty easy.
I guess there's _some_ signal in that, but I don't know if it really says anything about how good I am at coding in general.
I wrote about my own experiences with coding challenges during my recent job search on my blog at https://blog.urth.org/2022/04/19/software-job-search-2022-re...
I'll summarize my conclusion about live coding challenges, which is that I'm not convinced that my performance on these challenges reflected any abstract skill I have. Instead, they mostly reflected the fact that the problems I was given were either nearly identical to work I've done in the past, or were similar enough to things I'd done recently that they felt pretty easy.
I guess there's _some_ signal in that, but I don't know if it really says anything about how good I am at coding in general.
If you're augering for a $250,000 job, and the gate is a leetcode interview, just spend a couple weeks prepping for it. It'll be the biggest bang for the buck effort you'll ever make.
It takes months of practice to go through all the material for a FAANG interview. The recruiters even tell you as such.
If it takes months of practice, probably one didn't learn very much in programming classes. But you'll learn a heck of a lot with those months of practice, and that'll pay off handsomely with that $250,000 offer.
I usually give a data structure coding interview:
Here's some data models, and here's how you can reference eachother by properties. Cool. Let's do 3 levels of nested loops to extract the relationships and map them. That's like 99% of what they'll be doing.
The signals I'd look for are "do they start the entire thing by looking at structures or not", etc.
But unfortunately this breaks down when you're interviewing 10-year+ candidates. It doesn't yield anything useful. So sticking for those positions with a really really basic problem and pseudocoding it, and using the remaining time to instead have them map out solutions they did in the past seems to be useful.
Most things leet code interviews screen is "does this person know how to cram" because I tell you, everyone I know who aced leet code interviews crammed for 3 months and literally knew any one you can think of, and every variation. Juniors tend to do better in those, ironically enough.
Here's some data models, and here's how you can reference eachother by properties. Cool. Let's do 3 levels of nested loops to extract the relationships and map them. That's like 99% of what they'll be doing.
The signals I'd look for are "do they start the entire thing by looking at structures or not", etc.
But unfortunately this breaks down when you're interviewing 10-year+ candidates. It doesn't yield anything useful. So sticking for those positions with a really really basic problem and pseudocoding it, and using the remaining time to instead have them map out solutions they did in the past seems to be useful.
Most things leet code interviews screen is "does this person know how to cram" because I tell you, everyone I know who aced leet code interviews crammed for 3 months and literally knew any one you can think of, and every variation. Juniors tend to do better in those, ironically enough.
> Juniors tend to do better in those, ironically enough.
Is it ironic or intentional? Seems like a great way to hide age discrimination under the guise of "fairness" (everyone gets the same test).
Is it ironic or intentional? Seems like a great way to hide age discrimination under the guise of "fairness" (everyone gets the same test).
That's an interesting observation: if you are hiring someone whose role is largely high level decision making, like a principal engineer, that skillset is orthogonal to anything you could get them to show you on a keyboard. Almost by definition: the role is simply more than what a single person can do at the keyboard in X amount of time, so testing for that would be an incomplete test.
> Most things leet code interviews screen is "does this person know how to cram" because I tell you, everyone I know who aced leet code interviews crammed for 3 months and literally knew any one you can think of, and every variation.
As I commented above, I successfully passed multiple leetcode interviews in the last four years, including FAANG. I did not spend a single minute on prep.
As I commented above, I successfully passed multiple leetcode interviews in the last four years, including FAANG. I did not spend a single minute on prep.
The real reason there are these coding things:
A) SWEs are shit behavioral interviewers in any other way. You are not about to chnage that even if you are google.
B) It is crucial that newcomers be hired by their peers and by their direct manager. This has solid research. People hate it when they get an hire and they hadn't a meaningful part in the decision... The new hire is more likely to fail.
A+B means it is better to have a terrible process lead by SWEs (e.g. coding interviews), than a good one lead by professional recruiters (which will fail, no matter their capabilities and methodology) (and yes, I do believe that without considering A, some non technical recuiters trained at workplace psychology will have a much better success rate without needing any technical interview. They will need someone technical to have a talk with the candidate, but definitely not to ask leetcode etc).
A) SWEs are shit behavioral interviewers in any other way. You are not about to chnage that even if you are google.
B) It is crucial that newcomers be hired by their peers and by their direct manager. This has solid research. People hate it when they get an hire and they hadn't a meaningful part in the decision... The new hire is more likely to fail.
A+B means it is better to have a terrible process lead by SWEs (e.g. coding interviews), than a good one lead by professional recruiters (which will fail, no matter their capabilities and methodology) (and yes, I do believe that without considering A, some non technical recuiters trained at workplace psychology will have a much better success rate without needing any technical interview. They will need someone technical to have a talk with the candidate, but definitely not to ask leetcode etc).
Mind sharing the solid researching on that hiring by the team comment? My company is switching to pooled hiring and it irks me, so I’d love to read up…
Software development is really weirdly unique in the way we interview candidates. In no other profession does a job candidate with work experience have to “study” or “practice” for an interview other than reading up about the company. Unless they’re interviewing for their first job, talk to candidates about their experience and dig in to what they actually accomplished.
My view is that a coding interview is not an interview of whether you can code.
It is an interview about your communication, whether the code you write can be maintained, whether you can inspire/build trust (can you explain what you're doing? does that feel reasonable to the other person?).
Where people fail is to think that it's a test of experience, or a "heads down I must make every test case pass", that the outcome matters more than the process.
It's all about communication.
It is an interview about your communication, whether the code you write can be maintained, whether you can inspire/build trust (can you explain what you're doing? does that feel reasonable to the other person?).
Where people fail is to think that it's a test of experience, or a "heads down I must make every test case pass", that the outcome matters more than the process.
It's all about communication.
Very well written article despite the fact that I completely disagree with the conclusions logic of the author. I'm on the "reading code" and having a more personal connection side of the road. Partially on the debugging aspect for some cases. Pretty much the process explained here: https://talktotheduck.dev/debugging-the-technical-interview-...
The "write code" types of interviews fit for juniors who might not know how to even write code properly. If a person doesn't know how to do that and is a junior they usually have the right energy and can put 100% into the job to adapt/learn. I hired such juniors who had no experience in the languages/platforms and were productive within a month. They had good character and discipline.
Distilling a question like that for an advanced coder is impossible. No wonder the article didn't provide any sample of a "good interview". We'd all burn him at the stake because we'd all hate it. There's no such piece of code that would provide a valuable data point about a candidate. So this is a waste of your time when dealing with a junior and with an advanced coder.
Coding interviews are difficult. You will make mistakes, this is unavoidable. I think the best mistakes to make are the ones where you hire people who are part of your team and can grow with it. Even if they aren't the best coders around, being part of the team will help them hone those skills and evolve. If I had to pick a 10/10 coder or a team player I'd pick the latter every time.
The "write code" types of interviews fit for juniors who might not know how to even write code properly. If a person doesn't know how to do that and is a junior they usually have the right energy and can put 100% into the job to adapt/learn. I hired such juniors who had no experience in the languages/platforms and were productive within a month. They had good character and discipline.
Distilling a question like that for an advanced coder is impossible. No wonder the article didn't provide any sample of a "good interview". We'd all burn him at the stake because we'd all hate it. There's no such piece of code that would provide a valuable data point about a candidate. So this is a waste of your time when dealing with a junior and with an advanced coder.
Coding interviews are difficult. You will make mistakes, this is unavoidable. I think the best mistakes to make are the ones where you hire people who are part of your team and can grow with it. Even if they aren't the best coders around, being part of the team will help them hone those skills and evolve. If I had to pick a 10/10 coder or a team player I'd pick the latter every time.
Coding questions are a waste of time. A better interview technique is asking about a problem they solved and drill down on it. This actually is better and allows for a conversation and discussion. I know you can program so don't need to trick you up on it. If you can't then you'll quickly be found out on the job.
On the last type, I had one where I had to write code for a checkout system with the developer sitting next to me. I hated the pressure and felt very stressed, halfway through the problem I hit a bug and my mind went literally blank, all the map in my mind of variables, flow etc literally went black. I remember it happening and I totally lost myself, ended up looking out the window for a while at the sun shining on a building. Somehow got it back and ended up finishing the task and I remember basically shouting I'd finished and that that was it, I wouldn't do anymore as I was so relieved it was over.
Got the job.
Got the job.
There is also one thing about coding interviews that no one ever talks about: how there are so many bad interviewers out there.
Nobody trains for it and everyone thinks it's easy.
[deleted]
I don’t know how to feel about coding interviews. I never had a serious coding interview in my 24 year job history between 7 companies from 1996-2020. They were all your standard “dark matter” [1] development enterprise dev jobs.
Then I did a slight pivot to get into BigTech via cloud app dev consulting - “application modernization” [2] - still with no coding interview although I still do the same type of enterprise/cloud app dev I did in the latter part of my career in the “real world” - and a shit ton of yaml.
Even in 2020, if I couldn’t have gotten a job at either Azure or AWS, I probably would have done what it takes and practiced to increase my income by an easy six figures or more.
Coding interviews are a “gravity problem”. You can not like either. But they still exist.
That being said, if I were starting out today and could break the can’t get a job <-> don’t have experience cycle by “grinding leetcode” for few months and and end up in the top 10% of income earners in the US right out of college, I would have definitely done it and I encourage my younger relatives graduating in CS to do it.
However, if you’re just hiring for your standard yet another senior CRUD developer and paying as such ($90K-$160K) - don’t expect me to do the monkey dance.
I think algorithm style coding interviews are the great equalizer. Even though I at 48 probably couldn’t pass one without some serious practice.
[1] https://www.hanselman.com/blog/dark-matter-developers-the-un...
[2] yes it’s a full time position with the same compensation structure.
Then I did a slight pivot to get into BigTech via cloud app dev consulting - “application modernization” [2] - still with no coding interview although I still do the same type of enterprise/cloud app dev I did in the latter part of my career in the “real world” - and a shit ton of yaml.
Even in 2020, if I couldn’t have gotten a job at either Azure or AWS, I probably would have done what it takes and practiced to increase my income by an easy six figures or more.
Coding interviews are a “gravity problem”. You can not like either. But they still exist.
That being said, if I were starting out today and could break the can’t get a job <-> don’t have experience cycle by “grinding leetcode” for few months and and end up in the top 10% of income earners in the US right out of college, I would have definitely done it and I encourage my younger relatives graduating in CS to do it.
However, if you’re just hiring for your standard yet another senior CRUD developer and paying as such ($90K-$160K) - don’t expect me to do the monkey dance.
I think algorithm style coding interviews are the great equalizer. Even though I at 48 probably couldn’t pass one without some serious practice.
[1] https://www.hanselman.com/blog/dark-matter-developers-the-un...
[2] yes it’s a full time position with the same compensation structure.
I don't really have an opinion about whether or not programming interviews are useful. I think they probably are useful in most situations. Nevertheless, I suck at them. I don't think I've ever passed one. I'm not even a good programmer. When I ship code, I'm slow. If I get hired it's because someone has seen code I've written in the wild, and it works, and it is clean and readable and well tested -- and because the person hiring can see that I love building stuff.
I'm finishing up a personal open-source project and am looking for a new position focused on ruby or rails, so if you're hiring and are open to evaluating me based on something cool I've written, hit me up.
I'm finishing up a personal open-source project and am looking for a new position focused on ruby or rails, so if you're hiring and are open to evaluating me based on something cool I've written, hit me up.
Other professions handle this via professional qualifications. Architects - and please do contradict me actual architects! - are not tested on their knowledge of basic architectural concepts as part of interviewing at a firm, because that's what the professional qualification is for. You have done the test already.
And here's where our industry isn't a "real" profession yet. There are plenty of qualifications one can get, but none that gates the ability to call oneself a software engineer. The result: we have to prove it in interviews.
And I don't know whether this is better or not. It is different though. The price of not having to sit through exams is to have to jump through hoops.
And here's where our industry isn't a "real" profession yet. There are plenty of qualifications one can get, but none that gates the ability to call oneself a software engineer. The result: we have to prove it in interviews.
And I don't know whether this is better or not. It is different though. The price of not having to sit through exams is to have to jump through hoops.
That system is basically "sit through one exam when you're young". This can work fine in something that's not as detailed as coding.
The thing with software is people tend to think of it as something that needs maintenance to stay fresh, perhaps a bit like piloting an airplane.
Airplanes have logs and pilots recertify on an ongoing basis, but software doesn't work like that. Someone might have stopped coding but still have a title.
I'm not saying software actually does work like that, but I'm sure it's a view some people subscribe to.
The thing with software is people tend to think of it as something that needs maintenance to stay fresh, perhaps a bit like piloting an airplane.
Airplanes have logs and pilots recertify on an ongoing basis, but software doesn't work like that. Someone might have stopped coding but still have a title.
I'm not saying software actually does work like that, but I'm sure it's a view some people subscribe to.
Is there anyone who has never had a bad experience with a licensed qualified professional? Bad doctors, dentists and lawyers are easy to find. I wish I had a way of gauging knowledge before letting someone drill my teeth
Programming interviews are really important but the questions should not be those about tricky data structure or based on obsecure mathematical trick. IMO the question should be presented in terms of real world concepts so that the candidate can be tested on what data structure they use to model the data related to the problem statement and the problem statement should be simple enough that you can easily come up with a very high level approach yet the implementation would require understadning concepts of conditionals/iteration/recursion. And you should also check how the candidate break the problem into smaller problems (instead of writing a single big function)
[deleted]
I agree with this articles ending, what you offer though is a hybrid of pair programming, what we ought to say is that effectively we need to have a conversation, trust me i wont do a coding game with you, ill just ignore you, but a conversation like a human being, that i can do and i'm more than willing, and yes ask me anything because i will ask you too, the interview is a two way process, many of us learned that the hard way, therefore we wont repeat the same mistakes.
Perhaps I’m biased as someone with a better than average sense of algorithms and discrete math, but I think such interview questions have a lot of utility, provided the job will require some analysis of algorithms, eg you’re working on the platform, or a solver/optimizer.
Obviously it’s pants on head stupid to do that if you want a web dev to make a pretty app, tie frameworks together and liaise with 3rd party integrators. However, I think many employers want the crème of such devs who are also good at algos.
Obviously it’s pants on head stupid to do that if you want a web dev to make a pretty app, tie frameworks together and liaise with 3rd party integrators. However, I think many employers want the crème of such devs who are also good at algos.
> However, I think many employers want the crème of such devs who are also good at algos.
Which is not going to work for the vast majority of devs and jobs.
Most devs either don’t care or need to care about algorithms, and the few devs who do, spend most days not using them.
This would be akin to asking a civil engineer to build a wooden road bridge. Having the skills is great, but how often is the company going to need them?
Which is not going to work for the vast majority of devs and jobs.
Most devs either don’t care or need to care about algorithms, and the few devs who do, spend most days not using them.
This would be akin to asking a civil engineer to build a wooden road bridge. Having the skills is great, but how often is the company going to need them?
I am not a civil engineer, but I can imagine a situation where a wooden bridge could be used as a stopgap to pull off a more tricky construction.
Maybe not in the 21st century however, and I don’t think your analogy is perfect.
It might be more like wanting to hire an architect who is also great at civil eng. The skills definitely synergize and can help prune the architect’s search space of good designs, but such a candidate is probably just going to go for an engi job, and lean on their architecture skills there: for example an algo dev that can do mock-ups.
Maybe not in the 21st century however, and I don’t think your analogy is perfect.
It might be more like wanting to hire an architect who is also great at civil eng. The skills definitely synergize and can help prune the architect’s search space of good designs, but such a candidate is probably just going to go for an engi job, and lean on their architecture skills there: for example an algo dev that can do mock-ups.
> I think many employers want the crème of such devs who are also good at algos
I think that many employers don’t have a clue what makes a good dev, in general, or in the positions they’re trying to fill.
I think that many employers don’t have a clue what makes a good dev, in general, or in the positions they’re trying to fill.
I think in-person coding interviews (so both sides have skin in the game) are good but the problems should actually be applicable to the task at hand. In practice, this is rarely the case - I'm not sure if it's because there's a shortage of non-algorithm-focused questions or if because every bullshit startup wants to come across as some super advanced software R&D operation while what they ultimately need is a variant of CRUD.
In my 25 years as a developer I have been sitting on both ends of a coding interview a good number of times. One thing I find coding interviews helpful for is to weed out people that apply for a specific job description (e.g. "looking for experienced python developer", who's resume shows 7 years of recent python experience, but that during a simple coding test have problems getting the syntax right.
I wonder how well the following exam would correlate to job performance:
A completely fictitious language is developed. Simple enough that you can become fluent within an hour. From this you read a bunch of problems in said language and must solve them.
A set of assertions about the problems are written down and you must verify them. Finally a full program, created from a dialect of the given language is given and you try to understand
A completely fictitious language is developed. Simple enough that you can become fluent within an hour. From this you read a bunch of problems in said language and must solve them.
A set of assertions about the problems are written down and you must verify them. Finally a full program, created from a dialect of the given language is given and you try to understand
Please provide proof that a coding interview makes a statistically significant effect on predicting success in a position, all other things being equal.
My experience is obviously just anecdotal, but I can say that in the severely limited candidate pool I worked with in the past decade, there has been no correlation between their proficiency at code interviews and their performance at their day to day job.
I’m also not familiar with other fields, but are doctors or lawyers asked to do anything analogous to code assessments during interviews?
I’m also not familiar with other fields, but are doctors or lawyers asked to do anything analogous to code assessments during interviews?
How could you reasonably prove this? “All other things being equal” seems harder to evaluate than a coding interview.
Drive, experience, intelligence, communication skills, teamwork, emotional intelligence, grit, curiosity, physical health, mental stamina, family/home situation, and dozens of other factors play into job performance, many of which you can’t even ask about to generate your test dataset. Many of these traits are what you’re trying to measure the effects of in the interview.
Drive, experience, intelligence, communication skills, teamwork, emotional intelligence, grit, curiosity, physical health, mental stamina, family/home situation, and dozens of other factors play into job performance, many of which you can’t even ask about to generate your test dataset. Many of these traits are what you’re trying to measure the effects of in the interview.
If I'm not mistaken, Google has a paper on this, in which there is a statistical confirmation of coding interviews being a predictor of success.
3 problems I can see with this
a) Its like saying a Tobacco company has a paper saying smoking is healthy
b) Google measures things that matter to them. The operate on a scale that 99% of companies aren't. Their predictor of success only applies to companies of their scale.
c) What is their definition of success? And how does it apply to smaller companies less than 1000 employees?
a) Its like saying a Tobacco company has a paper saying smoking is healthy
b) Google measures things that matter to them. The operate on a scale that 99% of companies aren't. Their predictor of success only applies to companies of their scale.
c) What is their definition of success? And how does it apply to smaller companies less than 1000 employees?
Precisely. Especially with (b) and (c)
What works for Google does not mean it will also work for the 99% of companies that are NOT Google or any other FAANMG company.
This only shows that we have an incessant coding interview cargo-cult around how FAANMG companies interview candidates and several startups taking that to the extreme and thinking they are working on complex Google scale problems.
To these interviewers at these startups still under this delusion and for the 900th time as many have previously said for years as a wake up call:
You are NOT Google.
What works for Google does not mean it will also work for the 99% of companies that are NOT Google or any other FAANMG company.
This only shows that we have an incessant coding interview cargo-cult around how FAANMG companies interview candidates and several startups taking that to the extreme and thinking they are working on complex Google scale problems.
To these interviewers at these startups still under this delusion and for the 900th time as many have previously said for years as a wake up call:
You are NOT Google.
A wake up call is coming to theses startups because the money has stopped flowing. I suspect a lot of these startups replicating Google in both interview style and architecture are going together fail fast.
Read up on HN post on Fast. Someone was commenting one how they failed the interview because the HM wanted someone that wasn't a cowboy. They wanted someone that build microservices like Google. Being a cowboy is what you need to find PM fit. Focusing on microservices when you don't even have enough customers is a huge waste of money and detrimental to your company because of operational complexity. Any changes that should take 5 mins, now takes 3 hours. Fast had so little customers that an engineer suggested that they run the system under a single AWS monolith and was quickly shot down.
Read up on HN post on Fast. Someone was commenting one how they failed the interview because the HM wanted someone that wasn't a cowboy. They wanted someone that build microservices like Google. Being a cowboy is what you need to find PM fit. Focusing on microservices when you don't even have enough customers is a huge waste of money and detrimental to your company because of operational complexity. Any changes that should take 5 mins, now takes 3 hours. Fast had so little customers that an engineer suggested that they run the system under a single AWS monolith and was quickly shot down.
Google isn't selling its interview process. The one who is hurt if it doesn't work is Google itself, they have no reason to lie to themselves here. And I really doubt that someone tricked Larry Page into adopting this with faulty statistics, he cared so much about hiring that he personally reviewed every single hire until the company grew to over ten thousand employees. The hiring process changed many times over the years, and what Google runs now is what they ended up with.
Reread a) and b) from another perspective because never did I say they were lying to themselves.
I agree with b) and c). A is a bit forced though as tobacco companies faked studies to show that what they sell is healthy even though they knew it was horribly bad for you, I don't see how Google is anything like that with respect to hiring processes. If they talked about ads or search then the likeness would make sense, but I don't see it for hiring.
But I agree that Google is different than most companies. There the main thing is ads and search and infrastructure to run both of those. They can always use more developers to help optimize the profits of those algorithms or reduce runtime costs, the ROI on that is extremely high so that is what their hiring process is optimized for. Some of their other projects might have done better with a different process, but they don't have much weight compared to the revenue generating projects.
But I agree that Google is different than most companies. There the main thing is ads and search and infrastructure to run both of those. They can always use more developers to help optimize the profits of those algorithms or reduce runtime costs, the ROI on that is extremely high so that is what their hiring process is optimized for. Some of their other projects might have done better with a different process, but they don't have much weight compared to the revenue generating projects.
> But I agree that Google is different than most companies. There the main thing is ads and search and infrastructure to run both of those.
One of the Marques Brownlee videos mentions that when he met Eric Schmidt, Schmidt mentions that every problem Google have were scaling. If you look at it from his perspective, it would make a lot of sense why algo is so important for Google interviews. He applied it not only in tech, but with hiring as well.
One of the Marques Brownlee videos mentions that when he met Eric Schmidt, Schmidt mentions that every problem Google have were scaling. If you look at it from his perspective, it would make a lot of sense why algo is so important for Google interviews. He applied it not only in tech, but with hiring as well.
"CEO Eric Eric Schmidt stood out because he 'was the only candidate who had been to Burning Man.'" https://en.wikipedia.org/wiki/Eric_Schmidt#cite_note-31
Funny how the "objective tests" that leaders apply to grunts never get applied to the leaders themselves.
Funny how the "objective tests" that leaders apply to grunts never get applied to the leaders themselves.
It's the whole "Do as I say, not as I do" motto. In their eyes, you are workers, and they are employers, so these objective tests don't apply to them. They hire based on their need and not your wants. They don't need another Eric Schmidt, they need more lemmings to do lemmings work.
Ultimately, that's how I think these companies eventually mature. The people that know how to steer the cargo ship retires and you are left with workers that only know how to do the work assigned. Bureaucracy ensues and no one dares to steer for fear it would sink it or get fired for causing unneeded risk. And plus, they don't know how to.
Ultimately, that's how I think these companies eventually mature. The people that know how to steer the cargo ship retires and you are left with workers that only know how to do the work assigned. Bureaucracy ensues and no one dares to steer for fear it would sink it or get fired for causing unneeded risk. And plus, they don't know how to.
Did they hire people who failed their interviews in order to see how well they did?
That would be an amazing ab testing for hiring. And they are among the 100 companies who could easily do it.
Compared to what alternative?
[deleted]
[deleted]
Ad companies disguised as tech companies seem to love this type of interview.
It's interesting to see this occupy the same front page as:
https://news.ycombinator.com/item?id=31544634
How do you test knowledge indeed?
https://news.ycombinator.com/item?id=31544634
How do you test knowledge indeed?
Coding interviews are because of cheaters and frauds:
https://news.ycombinator.com/item?id=31544634
https://news.ycombinator.com/item?id=31544634
The academic qualification system is pretty broken in the computer science world. One core problem is a lot of teachers use automated tests to judge submitted code because they're either lazy or overworked, and it's very easy to game that setup.
I'd summarise the article as such. Fizz buzz like coding coding interviews are best. Looks at how the interviewee codes not just now, but in the future as we.
> Be careful of anything requiring recursion, it’s not super common to use it in practice
Lol be careful not to ask for a loop, nobody uses loops in real programming problems
Lol be careful not to ask for a loop, nobody uses loops in real programming problems
nice if i could just get a job on my experience alone. don't enjoy coding, idea of a test or leetcode is tiring to think about.
dont mind talking over code casually but dont put me on the spot or give me homework. not doing that.
dont mind talking over code casually but dont put me on the spot or give me homework. not doing that.
The real issue is "continuous recruitment" imo.
> Don’t ask “trick” questions, like anything where you have to use an obscure data structure or algorithm in a clever way (or dynamic programming).
I just want to put this out there to anyone who gets involved in interviews that this is a very important rule -- if your question can basically be redefined as some really clever solution you found on a problem you put > 30 minutes into, it's probably not a good question.
A lot of the seniors I work with would come up with questions like this, and I get their logic:
1. I'm a senior and this was a challenge for me 2. The logic is clear when you see it and look at it 3. I want people who can do work similar to me 4. I know how to get to the conclusion, so I can judge based on the path the candidate takes Conclusion: It's a good question
While I understand how we can get to such a conclusion, I think that in most cases the only takeaway you can get is "can this person think like I did in this situation?", which maybe has some value for judging workplace compatibility, but I don't think it's a fair assessment of someone's technical aptitude. If you yourself required a lot of time and brainpower to sort through a tricky issue like this with the benefit of the problem scope and system needs that led to this issue, it's very unlikely you'll be able to judge any assessment
Point 4 seems good at first because we can say "the goal is to just see them think and use their experience"; there is truth to it but without the rest of the context it's not really a good assessment because ultimately, you're looking to see "did you get the same conclusion I did? Did you avoid the same pitfalls I hit?"
Instead, focus on a general scenario that tests their understanding of basic principles. How do they use the fundamental knowledge for the subject matter to work through a new problem that you yourself haven't really dealt with. It removes your bias a bit and opens you up to those "wow why didn't I think of that?" moments that you'd have working on tough projects together.
Try your best to give a good faith interpretation to any path the candidate takes and help take it to natural conclusions, and see how the candidate reacts; if you can see they've made a workable but problematic solution, nudge them towards the problems you see and see how they discuss it.
For me a candidate that can really process their own thoughts and do a good analysis of their own solution (its benefits and shortcomings) is extremely valuable; I usually lead off with "I know how I'd do it, but that doesn't mean it's the right answer, so focus on your strategy and walk me through it." If you want tot test compatibility or you feel you have a stronger solution, accept their solution (if it's valid) then go ahead and discuss your thoughts a bit and see how they respond and how they discuss it. The more you make it a technical conversation against peers and less a university exam, the more you have to see how they think and operate and the better you can understand them.
I just want to put this out there to anyone who gets involved in interviews that this is a very important rule -- if your question can basically be redefined as some really clever solution you found on a problem you put > 30 minutes into, it's probably not a good question.
A lot of the seniors I work with would come up with questions like this, and I get their logic:
1. I'm a senior and this was a challenge for me 2. The logic is clear when you see it and look at it 3. I want people who can do work similar to me 4. I know how to get to the conclusion, so I can judge based on the path the candidate takes Conclusion: It's a good question
While I understand how we can get to such a conclusion, I think that in most cases the only takeaway you can get is "can this person think like I did in this situation?", which maybe has some value for judging workplace compatibility, but I don't think it's a fair assessment of someone's technical aptitude. If you yourself required a lot of time and brainpower to sort through a tricky issue like this with the benefit of the problem scope and system needs that led to this issue, it's very unlikely you'll be able to judge any assessment
Point 4 seems good at first because we can say "the goal is to just see them think and use their experience"; there is truth to it but without the rest of the context it's not really a good assessment because ultimately, you're looking to see "did you get the same conclusion I did? Did you avoid the same pitfalls I hit?"
Instead, focus on a general scenario that tests their understanding of basic principles. How do they use the fundamental knowledge for the subject matter to work through a new problem that you yourself haven't really dealt with. It removes your bias a bit and opens you up to those "wow why didn't I think of that?" moments that you'd have working on tough projects together.
Try your best to give a good faith interpretation to any path the candidate takes and help take it to natural conclusions, and see how the candidate reacts; if you can see they've made a workable but problematic solution, nudge them towards the problems you see and see how they discuss it.
For me a candidate that can really process their own thoughts and do a good analysis of their own solution (its benefits and shortcomings) is extremely valuable; I usually lead off with "I know how I'd do it, but that doesn't mean it's the right answer, so focus on your strategy and walk me through it." If you want tot test compatibility or you feel you have a stronger solution, accept their solution (if it's valid) then go ahead and discuss your thoughts a bit and see how they respond and how they discuss it. The more you make it a technical conversation against peers and less a university exam, the more you have to see how they think and operate and the better you can understand them.
Stop defending whiteboard interviews. We know [1] that they are inherently biased against certain groups and have a very large false negative rate.
It amazes me how many people will defend the current interviewing process and then openly admit that the majority of their engineers could not pass the same process without spending dozens of hours preparing.
[1] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
It amazes me how many people will defend the current interviewing process and then openly admit that the majority of their engineers could not pass the same process without spending dozens of hours preparing.
[1] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
That study lacks data. The entire result disappears if you exclude the women. And womens results looks strange, every woman does perfectly in the private version and every woman does horribly in the observed version. It is possible the interviewer either was a hot man who distracted the women, or he was a sexist creep who distracted them in other ways, or it was just random which is possible since there were so few women.
But other than that the study didn't find any evidence that men have any issues with being observed while coding. So unless a better study comes around we can dismiss complaints about observed interviews being stressful unless it comes from a woman, it is in the study you linked after all. (I do believe there is a stress effect, but that study isn't showing it, need a bit more samples and probably several different observers to get better data)
But other than that the study didn't find any evidence that men have any issues with being observed while coding. So unless a better study comes around we can dismiss complaints about observed interviews being stressful unless it comes from a woman, it is in the study you linked after all. (I do believe there is a stress effect, but that study isn't showing it, need a bit more samples and probably several different observers to get better data)
"Do coding interviews work in the actual world in which we live in?" No, fundamentally not. Almost nobody is sufficiently capable to actually analyze code, and writing code with an audience is absolutely a stupid anxiety inducing high stress situation that makes candidates into babbling children who can't define an array - EVEN IF THE CANDIDATE CAN IN FACT PROGRAM JUST FINE. Add in one idiot on the interview panel who can't stop "tsk"-ing or asking obnoxious questions ("Why did you use a foreach and not a for loop?") and you have an hour long anxiety sandwich where the only thing you are getting from the candidate is whether or not they can dance in front of a sufficiently hostile crowd on demand.