Does stress impact technical interview performance? (2020) [pdf](chrisparnin.me)
chrisparnin.me
Does stress impact technical interview performance? (2020) [pdf]
http://www.chrisparnin.me/pdf/stress_FSE_20.pdf
146 comments
I interviewed at a company last month that reminded me of your experience. The interviewer was extremely rude and it was more of an interrogation than anything.
After berating one of my responses, I told him this wasn’t going to work out, that this was easily the most off putting interview I’d ever experienced, but thanks for the consideration.
The dude was stunned, apologized profusely, and we somehow salvaged the last 20 minutes which we spent nerding out on tech and hobbies.
I ended up taking my name out of consideration, but it was a cool experience realizing I can (respectfully but assertively) tell people to fuck off if I need to.
After berating one of my responses, I told him this wasn’t going to work out, that this was easily the most off putting interview I’d ever experienced, but thanks for the consideration.
The dude was stunned, apologized profusely, and we somehow salvaged the last 20 minutes which we spent nerding out on tech and hobbies.
I ended up taking my name out of consideration, but it was a cool experience realizing I can (respectfully but assertively) tell people to fuck off if I need to.
Well done for doing that. It's empowering to bear in mind that you are interviewing them too. If your interviewer is highly unpleasant then why on earth would you want to work for them / a company that thinks its OK to be represented by them. No thanks!
The paper is easy to read, and a bunch of questions that came to mind were addressed, or at least acknowledged, in one place or another of the paper.
I appreciated the following tidbit, which I didn't know (since I figured prep for this particular test was now widespread there, by convention, but not officially stated):
> students at Stanford take courses for passing technical interviews, CS9: Problem-Solving for the CS Technical Interview.
And because of things like that, the current interview convention seem even less useful as a predictor, for purposes other than "Did you go to Stanford, or are of similar socioeconomic status, to have been coached on beating the current metric, specifically?"
I appreciated the following tidbit, which I didn't know (since I figured prep for this particular test was now widespread there, by convention, but not officially stated):
> students at Stanford take courses for passing technical interviews, CS9: Problem-Solving for the CS Technical Interview.
And because of things like that, the current interview convention seem even less useful as a predictor, for purposes other than "Did you go to Stanford, or are of similar socioeconomic status, to have been coached on beating the current metric, specifically?"
If you are expected to study to pass an interview, is the result really indicative of how you will succeed in the job, or a measure of your skill is studying to pass an interview?
Related from last year:
Does Stress Impact Technical Interview Performance? - https://news.ycombinator.com/item?id=23844367 - July 2020 (26 comments)
Does Stress Impact Technical Interview Performance? - https://news.ycombinator.com/item?id=23844367 - July 2020 (26 comments)
Literally just reading that headline gave me cold sweats and a wave of anxiety.
Tech wouldn't be where it is today without antiquated interviews and an industry of human resource management built on faith and tradition rather than science. Talented people go elsewhere, create new innovation, and prosper while the large tech companies copy or acquire the innovation. You are successful not only because of what you accomplish in life but because of what you don't accomplish. You choose not to spend hundreds of hours training to answer coding problems, prioritizing other activities to advance yourself.
I am surprised that there are not more "traditional tests". Give problem, give person 45 minutes, leave the room, come back and discuss it. I have strong social anxiety. It is extremely difficult to solve a problem with someone watching me. I get panic attacks in at least 1/3 of my interviews. I have never had to code with someone watching me during my day to day job. I am not not interviewing for the job of coding performance artist. Why make me audition like I am an actor or a musician?
I can't say I enjoy interviews, but they're really not that bad. Reading the comments here, you'd think everyone gets paralyzing anxiety during interviews. They're a chance to show off how smart you are.
I've probably done at least a few hundred tech interviews in my career, and the vast majority of them are simply not an opportunity to show anything off. You're putting up with various interviewers in various moods of varying levels of competence, some of them actively hostile or working on some sort of weird agenda. You're bouncing between interviewers and getting asked the same question multiple times and having to explain the same thing over and over.
Even if they don't make you anxious (that started to subside after the first hundred times or so) they're still not a good time. I've had senior staff running interviews at big companies do things like harass me over data structure choices, insist that I was lying when I described a textbook algorithm, accuse me of industrial espionage, and all other sorts of absurd or unpleasant things. In some cases even after that I still got an offer, because the interviewers weren't out to give someone a hard time... they were just bad at running interviews. One of my all-time worst interviews involved the design director (?) at a billion-dollar game studio calling me (unscheduled!) at 7am and demanding that I come up with 50 unique uses for a paperclip. Really.
The gap between an average or bad interview and a truly good one is enormous, and the number of well-conducted interviews I've had is incredibly small. I can only imagine how much worse it is for someone with additional obstacles, like how is someone in a wheelchair supposed to stand up and write C++ on a whiteboard for 30 minutes? What does the whiteboard interview look like for a blind woman?
Even if they don't make you anxious (that started to subside after the first hundred times or so) they're still not a good time. I've had senior staff running interviews at big companies do things like harass me over data structure choices, insist that I was lying when I described a textbook algorithm, accuse me of industrial espionage, and all other sorts of absurd or unpleasant things. In some cases even after that I still got an offer, because the interviewers weren't out to give someone a hard time... they were just bad at running interviews. One of my all-time worst interviews involved the design director (?) at a billion-dollar game studio calling me (unscheduled!) at 7am and demanding that I come up with 50 unique uses for a paperclip. Really.
The gap between an average or bad interview and a truly good one is enormous, and the number of well-conducted interviews I've had is incredibly small. I can only imagine how much worse it is for someone with additional obstacles, like how is someone in a wheelchair supposed to stand up and write C++ on a whiteboard for 30 minutes? What does the whiteboard interview look like for a blind woman?
Didn't "stereotype threat" fail replication? Why do researchers cite it?
It'd be interesting to see the impact wrt age.
Yes.
It seems that technical interview is a great way to do psychological evaluation. How does this person manage stress and deal with it?
Why do you want all of your employees to be the ones most resilient to stress?
Are you hiring for a fire department? The Navy Seals? Engineering section on a nuclear submarine?
Maybe, just maybe, things would improve if you had a few people who said 'no way' when someone tries to crank up the drama on your team instead of just grinning and bearing it.
Capitalists don't reward the quiet workers. They exploit them. Billionaires and wannabees do it remorselessly.
Are you hiring for a fire department? The Navy Seals? Engineering section on a nuclear submarine?
Maybe, just maybe, things would improve if you had a few people who said 'no way' when someone tries to crank up the drama on your team instead of just grinning and bearing it.
Capitalists don't reward the quiet workers. They exploit them. Billionaires and wannabees do it remorselessly.
These aren't even the same kinds of stress. I've been in life-threatening situations in the military and felt no stress in the moment. I've played drums in front of a crowd of 4,000 people and felt no stress. Whiteboarding interviews wreck me though.
My brain read 'waterboarding' and then didn't entirely correct itself.
If I had a Bitcoin for every time I heard a manager say that he idolizes the Navy SEALs.....
The authors address this: the induced stress is not necessarily similar to the stress experienced on the job they're applying for.
This has been my experience. Time is crunched in a rather unrealistic way. Not that time crunches don't happen, but it's never once been the case in my real job that I was approaching an unpracticed academic algorithm problem which had a 30 minute time cap.
Recent example: I was asked to do a density map with about ~20 minutes remaining in an interview. I could immediately see the performance issue of the naive solution (re-summing the same positions), but I needed some time to think through an efficient solution. But because of the time limit and the fact that I had someone watching my every step (i.e. there was more performance than problem solving going on), we agreed that I would take the naive approach and simply sum all of the positions as quickly as I could. When I wasn't asked back for a second round, it was obviously in part because I had resorted to such an ugly and inefficient solution.
I don't blame the interviewer—he was actually one of the best I've ever interacted with—and I actually think the problems selected fit the role very well. But at the end of the day I don't feel that format gave a very good snapshot of my abilities. (Case in point, I looked up solutions afterwards and an intuitive approach I had thought of was very close to the optimal solution, but I didn't feel I had time to go down that route.)
I don't think there is a perfect interview format. They all have downsides. But this is definitely a tradeoff you're making with timed algos.
Recent example: I was asked to do a density map with about ~20 minutes remaining in an interview. I could immediately see the performance issue of the naive solution (re-summing the same positions), but I needed some time to think through an efficient solution. But because of the time limit and the fact that I had someone watching my every step (i.e. there was more performance than problem solving going on), we agreed that I would take the naive approach and simply sum all of the positions as quickly as I could. When I wasn't asked back for a second round, it was obviously in part because I had resorted to such an ugly and inefficient solution.
I don't blame the interviewer—he was actually one of the best I've ever interacted with—and I actually think the problems selected fit the role very well. But at the end of the day I don't feel that format gave a very good snapshot of my abilities. (Case in point, I looked up solutions afterwards and an intuitive approach I had thought of was very close to the optimal solution, but I didn't feel I had time to go down that route.)
I don't think there is a perfect interview format. They all have downsides. But this is definitely a tradeoff you're making with timed algos.
You can be the most resilient person to real life work/emergency/deadline pressure and stress and still get stage fright when giving talks or being in an interview.
It’s a great way to pass on otherwise awesome employees. You don’t need all your employees to be the same. You don’t need them all of them to manage their stress the same way. You don’t even need all of them to be good at managing their stress.
> A technical interview has an uncanny resemblance to the trier social stress test [39], a procedure used for decades by psychologists and is the best known “gold standard” procedure [1] for the sole purpose of reliably inducing stress.
> The trier social stress test involves having a subject prepare and then deliver an interview-style presentation and perform mental arithmetic, all in front of an audience.
> … rather than avoiding unwanted stress, technical interviews may be inadvertently designed with the sole purpose of inducing it.
39: https://www.karger.com/Article/Abstract/119004
> The trier social stress test involves having a subject prepare and then deliver an interview-style presentation and perform mental arithmetic, all in front of an audience.
> … rather than avoiding unwanted stress, technical interviews may be inadvertently designed with the sole purpose of inducing it.
39: https://www.karger.com/Article/Abstract/119004
I mean, that's what the interview format was designed to do. Schockley decided that how he should hire engineers was to stress them and ask them brain teasers. He had no basis but his own opinion, but he had to hire fast because none of his colleagues from Bell Labs were willing to work with him again. Then a bunch of his engineers all quit because he was such a terrible boss and founded Fairchild Semiconductor.
We've at least abolished the brain teaser part in most cases.
We've at least abolished the brain teaser part in most cases.
We really haven't though. I've been going through the leetcode questions and I'd say > 30% are basically brain teasers and either you've seen the answer or related answer before or you haven't. If you're lucky and have some magical insight then maybe you solve it. Otherwise you look up the answer, possibly implement it yourself to try to cement it in your mind, and now if you're lucky it's added to your internal list of known solutions.
To give just one example, I don't think I'd have ever solved the "find the largest rectangle" in a bitmap. Certainly not in a 45 minute interview. The brute force solution is O(N^3 * M^3). Me, I couldn't think of any spatial structures that would make the problem all that much smaller. For any structural solution it seems to be the degenerate case still ends up using too much space and too much time. But, there's an optimal "magic" solution is O(N*M). If you happen to know it or have seen then then you bang it out. If you haven't it's arguably just luck that you'll have the insight during your interview.
I've seen several others of the same vane. Given 4-24 hours and a debugger to help my thoughts and directly my mistaking assumptions I might solve some of them but in 45 minutes on a whiteboard, it's just a lucky role of the dice that I've seen something similar before.
The Jan 2021 google foobar was similar. I didn't figure it out nor did a friend of mine. We both spent 3-4 days. I bought it up with another friend. His first solution was wrong. When I pointed out why he was like "Oh yea, this the XYZ-algo" (meaning he didn't solve it, he just had read the solution before). The wikipedia page for the arguably obscure algo basically showed the exact same example as the foobar one.
To give just one example, I don't think I'd have ever solved the "find the largest rectangle" in a bitmap. Certainly not in a 45 minute interview. The brute force solution is O(N^3 * M^3). Me, I couldn't think of any spatial structures that would make the problem all that much smaller. For any structural solution it seems to be the degenerate case still ends up using too much space and too much time. But, there's an optimal "magic" solution is O(N*M). If you happen to know it or have seen then then you bang it out. If you haven't it's arguably just luck that you'll have the insight during your interview.
I've seen several others of the same vane. Given 4-24 hours and a debugger to help my thoughts and directly my mistaking assumptions I might solve some of them but in 45 minutes on a whiteboard, it's just a lucky role of the dice that I've seen something similar before.
The Jan 2021 google foobar was similar. I didn't figure it out nor did a friend of mine. We both spent 3-4 days. I bought it up with another friend. His first solution was wrong. When I pointed out why he was like "Oh yea, this the XYZ-algo" (meaning he didn't solve it, he just had read the solution before). The wikipedia page for the arguably obscure algo basically showed the exact same example as the foobar one.
My least favourite interview technique has to be "talk me through a complex problem on a whiteboard". It's not so much the stress but the fact that you can't have a train of thought longer than 5 seconds before someone finds the silence awkward or interprets it as you're lost and breaks the silence and train of thought. It's a test for solving complex problems in 5 second blocks while being constantly interrupted.
I actually love these kinds of questions. Allows the candidate to run through the problem as a conversation instead of a binary solved/not-solved outcome.
I usually preface it by saying that I will take a minute to write down a few ideas, so the silence isn't awkward. It is such a great way to engage your interviewer, while gaining a finer ability to choose the level of abstraction when you are working through a problem.
I love it also because it helps those who aren't native English speakers. The ability to visually represent ideas is equally distributed across language skills. It doesn't even need to be suited for a visual thinker. It can be math, venn diagrams, graphs or even straight up code.
I usually preface it by saying that I will take a minute to write down a few ideas, so the silence isn't awkward. It is such a great way to engage your interviewer, while gaining a finer ability to choose the level of abstraction when you are working through a problem.
I love it also because it helps those who aren't native English speakers. The ability to visually represent ideas is equally distributed across language skills. It doesn't even need to be suited for a visual thinker. It can be math, venn diagrams, graphs or even straight up code.
Also terrifying and mentally paralyzing for a large percentage of devs. I'm ok with these interviews (and agree they can be fun when they go well) if you are ok with a very high rate of false negatives and being basically inhumane to that same group of people.
It's funny you say that, because the alternative is Coderpad/Hackerrank style quizzes which I completely despise.
Those give me a world of anxiety. I know I can practice leetcode for 3 months full time and ace them, but that feels so wrong in spirit. Also, that's a tall ask for someone with a fulltime job. I will fully admit that I have never tried to remember syntax exactly for any language I use because coding speed was never a bottleneck for my job. Thus, I can almost never finish the darned problems in 45 minutes. My palms get sweaty and my head blanks out.
Weirdly enough. I can literally write the exact code on a whiteboard no-problem. The brain is weird.
Those give me a world of anxiety. I know I can practice leetcode for 3 months full time and ace them, but that feels so wrong in spirit. Also, that's a tall ask for someone with a fulltime job. I will fully admit that I have never tried to remember syntax exactly for any language I use because coding speed was never a bottleneck for my job. Thus, I can almost never finish the darned problems in 45 minutes. My palms get sweaty and my head blanks out.
Weirdly enough. I can literally write the exact code on a whiteboard no-problem. The brain is weird.
The alternative could maybe be something actually related to the job at hand. Like “I’ve got a database here and we need a view on this page - how would you approach this problem?”
I guarantee you will find people who do the actual work. Companies don’t need massive amounts of coders that can minmax an arbitrary book algorithm. If you want someone to be “clever” in exactly that way then you can push them in that direction while solving a closer to real looking problem.
I guarantee you will find people who do the actual work. Companies don’t need massive amounts of coders that can minmax an arbitrary book algorithm. If you want someone to be “clever” in exactly that way then you can push them in that direction while solving a closer to real looking problem.
Is not the inability to think on your feet in a productive way in front of others - especially stakeholders - a gigantic red flag you want to catch?
Having reasonable social skills and the ability to not look like an idiot when you're on the spot are both super important to being a productive member of a business
Edit: someone has just downvoted every one of my comments in this thread and not left a response, why?
Having reasonable social skills and the ability to not look like an idiot when you're on the spot are both super important to being a productive member of a business
Edit: someone has just downvoted every one of my comments in this thread and not left a response, why?
> Is not the inability to think on your feet in a productive way in front of others - especially stakeholders - a gigantic red flag you want to catch?
In my engineers? No. In my sales people? Yes.
I want my newly hired engineers (including of the software variety) to be focused on engineering, not doing half-assed "problem solving" presentations without any preparation or analysis. That's not how problem solving works, and I want my engineers to take their time, model problems, explore different approaches, and find solutions that are well explored, documented, and proved.
Conversely, being able to think/bullshit your way through live presentations without being given a chance to prepare anything is EXACTLY what I want in sales people; that skill is part of the skillset of being charismatic, and that is the skillset of people who do sales.
I wouldn't turn down an engineer who turns out to be good with charisma, and the ability to come up with decent answers on the spot is something I expect senior engineers to grow into after a few years of familiarity with our problem spaces. But I certainly wouldn't select for it, much as I wouldn't select a car mechanic based on how nice their haircut looks.
In my engineers? No. In my sales people? Yes.
I want my newly hired engineers (including of the software variety) to be focused on engineering, not doing half-assed "problem solving" presentations without any preparation or analysis. That's not how problem solving works, and I want my engineers to take their time, model problems, explore different approaches, and find solutions that are well explored, documented, and proved.
Conversely, being able to think/bullshit your way through live presentations without being given a chance to prepare anything is EXACTLY what I want in sales people; that skill is part of the skillset of being charismatic, and that is the skillset of people who do sales.
I wouldn't turn down an engineer who turns out to be good with charisma, and the ability to come up with decent answers on the spot is something I expect senior engineers to grow into after a few years of familiarity with our problem spaces. But I certainly wouldn't select for it, much as I wouldn't select a car mechanic based on how nice their haircut looks.
> not doing half-assed "problem solving" presentations without any preparation or analysis.
> I want my engineers to take their time, model problems, explore different approaches, and find solutions
So, you represent "whiteboard interviews" in bad faith as 'half-assed faux problem solving', and then you state all the things that I personally believe a whiteboard interview shows about an engineer's approach.
Listen, I am sure some roles don't require someone to ever problem solve, and that's fine. But if you think a white-board interview where you get to see someone approach a new problem, vocalize their common approaches, speak about the trade-offs of their approaches, and discuss how they'd implement it is closer to "half-assed" than "modeling problems and exploring different approaches", that's a you problem and not a me problem.
Your haircut strawman is par for the course in this thread.
> I want my engineers to take their time, model problems, explore different approaches, and find solutions
So, you represent "whiteboard interviews" in bad faith as 'half-assed faux problem solving', and then you state all the things that I personally believe a whiteboard interview shows about an engineer's approach.
Listen, I am sure some roles don't require someone to ever problem solve, and that's fine. But if you think a white-board interview where you get to see someone approach a new problem, vocalize their common approaches, speak about the trade-offs of their approaches, and discuss how they'd implement it is closer to "half-assed" than "modeling problems and exploring different approaches", that's a you problem and not a me problem.
Your haircut strawman is par for the course in this thread.
> In my engineers? No. In my sales people? Yes.
Thank you for this framing, really help me clarify my thoughts around this issue.
Thank you for this framing, really help me clarify my thoughts around this issue.
Thinking on your feet on a team is wildly different than thinking on your feet up against an opponent, especially when that opponent is your economic future.
As others have mentioned, live coding an unfamiliar problem on a whiteboard in front of strangers with a job on the line is very different from the typical day to day interactions in most dev jobs. The social skills needed for a sales engineer are different than those needed for a developer.
I would say that being able to understand other people's perspective is more important than comfort in high stress situations with strangers. So if someone didn't have anxiety himself, I would expect him to be able to understand how it affects others when it is explained to him. If he refuses to listen and insists it just lack of social skills on the part of those with anxiety, I would conclude he lacks the social skills necessary to work with a diverse group of people.
I would say that being able to understand other people's perspective is more important than comfort in high stress situations with strangers. So if someone didn't have anxiety himself, I would expect him to be able to understand how it affects others when it is explained to him. If he refuses to listen and insists it just lack of social skills on the part of those with anxiety, I would conclude he lacks the social skills necessary to work with a diverse group of people.
So, because some people have anxiety, we should abandon all types of interviewing that look for signal wrt someone's social & communication skills?
I expect anxiety to impact some people, and I expect stupidity to impact a whole lot more. Should I drop the technical interviews, because the stupid people find them extremely stressful? Should I take the assertion of every person with social anxiety that in a job interview they can't perform, but in every other situation a job presents them with they'll be just fine?
I'm sorry, but candidates being able to stand up, think thru a problem, discuss the merits of their solution, and explore the tradespace of other solutions is not a skill just for "sales" people. It's basic collaboration skills.
PS: I used to have heavy social anxiety and impostor syndrome. Those things were terribly unfun, but I openly recognize I frequently caused awkward situations, often failed to speak up when I should've, and was less open to sharing of ideas and code for fear of emotional harm. I spent extensive time to work on my social skills & technical ability, and now I don't feel those things anymore. I am also much more socially capable, open about sharing, willing to speak up in any given situation, and obviously technically stronger from studying.
Please fuck off with speaking down to me. "live coding is very different from the typical day to day interactions". So is the whole fucking interview, which is why I'm waiting for someone to explain to me how whiteboarding is somehow so much more unfairly stressful than all the other parts.
I expect anxiety to impact some people, and I expect stupidity to impact a whole lot more. Should I drop the technical interviews, because the stupid people find them extremely stressful? Should I take the assertion of every person with social anxiety that in a job interview they can't perform, but in every other situation a job presents them with they'll be just fine?
I'm sorry, but candidates being able to stand up, think thru a problem, discuss the merits of their solution, and explore the tradespace of other solutions is not a skill just for "sales" people. It's basic collaboration skills.
PS: I used to have heavy social anxiety and impostor syndrome. Those things were terribly unfun, but I openly recognize I frequently caused awkward situations, often failed to speak up when I should've, and was less open to sharing of ideas and code for fear of emotional harm. I spent extensive time to work on my social skills & technical ability, and now I don't feel those things anymore. I am also much more socially capable, open about sharing, willing to speak up in any given situation, and obviously technically stronger from studying.
Please fuck off with speaking down to me. "live coding is very different from the typical day to day interactions". So is the whole fucking interview, which is why I'm waiting for someone to explain to me how whiteboarding is somehow so much more unfairly stressful than all the other parts.
>So, because some people have anxiety, we should abandon all types of interviewing that look for signal wrt someone's social & communication skills?
No we should recognize that some parts of the interview give us false signals. We should improve the process to get better signals.
> Should I take the assertion of every person with social anxiety that in a job interview they can't perform, but in every other situation a job presents them with they'll be just fine?
No. Give me the problem, leave the room. Come back in 45 minutes and let me tell you about how I solved it. Don't make me solve it with you staring over me.
> Please fuck off with speaking down to me.
My first instinct was to say something rude in return. Instead I will ask why you think I am speaking down to you? I didn't downvote you. I did say I don't think you are good at understanding other people's perspectives.
>PS...
I am glad for you. That is awesome. I have worked on my anxiety for a long time. I used to get panic attacks when I had to say "here" during attendance roll call. I am much better than that now. But I still have anxiety and will likely always have it.
>I'm waiting for someone to explain to me how whiteboarding is somehow so much more unfairly stressful than all the other parts.
I can't tell you why it is more stressful, only that it is. So give me the problem, let me solve it in peace, and then let's discuss. You will get a stronger signal that way.
PS: I had very good SAT scores. That is about as an objective technical skill test as you can get. I also tutored teens in Math and Comp Sci. I have worked in teams with fellow engineers, designers, managers, and non technical business people. I know how to communicate with a wide variety of different people. But make me do a whiteboard interview and I will get a panic attack about a third of the time. You will then conclude I am an idiot who can't communicate.
No we should recognize that some parts of the interview give us false signals. We should improve the process to get better signals.
> Should I take the assertion of every person with social anxiety that in a job interview they can't perform, but in every other situation a job presents them with they'll be just fine?
No. Give me the problem, leave the room. Come back in 45 minutes and let me tell you about how I solved it. Don't make me solve it with you staring over me.
> Please fuck off with speaking down to me.
My first instinct was to say something rude in return. Instead I will ask why you think I am speaking down to you? I didn't downvote you. I did say I don't think you are good at understanding other people's perspectives.
>PS...
I am glad for you. That is awesome. I have worked on my anxiety for a long time. I used to get panic attacks when I had to say "here" during attendance roll call. I am much better than that now. But I still have anxiety and will likely always have it.
>I'm waiting for someone to explain to me how whiteboarding is somehow so much more unfairly stressful than all the other parts.
I can't tell you why it is more stressful, only that it is. So give me the problem, let me solve it in peace, and then let's discuss. You will get a stronger signal that way.
PS: I had very good SAT scores. That is about as an objective technical skill test as you can get. I also tutored teens in Math and Comp Sci. I have worked in teams with fellow engineers, designers, managers, and non technical business people. I know how to communicate with a wide variety of different people. But make me do a whiteboard interview and I will get a panic attack about a third of the time. You will then conclude I am an idiot who can't communicate.
No. It's pretty rare outside of interviews that you have to code in front of an audience. For most jobs having performance anxiety is no red flag at all.
If we had to categorise interviewing as a science I think it would land squarely in the social sciences.
Difficult to impossible to reproduce results; practitioners with wildly different skills and biases; institutions with systemic biases; individual candidates who can vary wildly from day to day; cohorts that change over time and distance; a lay population with strong opinions.
We'd be better off selecting candidates randomly, but even this only kicks the can down the he road.
Difficult to impossible to reproduce results; practitioners with wildly different skills and biases; institutions with systemic biases; individual candidates who can vary wildly from day to day; cohorts that change over time and distance; a lay population with strong opinions.
We'd be better off selecting candidates randomly, but even this only kicks the can down the he road.
> It's pretty rare outside of interviews that you have to code in front of an audience
But it is most certainly not rare that you will be expected to reason about possible solutions to problem domains you barely understand using techniques you have varying levels of familiarity to an audience.
Maybe I'm wrong, maybe this is just my line of work, and if you're recruiting a person who pulls tickets and there's no expectations beyond that whatever. But I would be assed to hire someone who, presented with a question my team thought fair to ask in an interview, was unable to stand up and cohesively walk me thru their approach, reasonably attack some parts of that approach, and offer some insights to what pros and cons they see in their approach and others they might have passed up. How is that person ever going to help in a brainstorm if they are so socially anxious they cant speak to an audience? How are they ever going to help their teammates in a Code Review if they are unable to find their voice in front of a whiteboard-tier problem that every single other candidate is facing?
If your social anxiety is this debilitating, that you can't do your literal profession in front of a crowd who you purport to be on the same level as by virtue of interviewing with them, I think you have a problem yes.
But it is most certainly not rare that you will be expected to reason about possible solutions to problem domains you barely understand using techniques you have varying levels of familiarity to an audience.
Maybe I'm wrong, maybe this is just my line of work, and if you're recruiting a person who pulls tickets and there's no expectations beyond that whatever. But I would be assed to hire someone who, presented with a question my team thought fair to ask in an interview, was unable to stand up and cohesively walk me thru their approach, reasonably attack some parts of that approach, and offer some insights to what pros and cons they see in their approach and others they might have passed up. How is that person ever going to help in a brainstorm if they are so socially anxious they cant speak to an audience? How are they ever going to help their teammates in a Code Review if they are unable to find their voice in front of a whiteboard-tier problem that every single other candidate is facing?
If your social anxiety is this debilitating, that you can't do your literal profession in front of a crowd who you purport to be on the same level as by virtue of interviewing with them, I think you have a problem yes.
Sounds like stress doesn't impact you, even in interview settings. Good for you! Too bad it seems like you also lack the empathy to see why other people might find interviews different from normal work-collaboration duties.
And no, it's not your job to worry about other people's issues, but if you think that 'being stressed in an interview setting' == 'unable to collaborate' (this is what I read from your ridiculous strawmen), you're going to pass on a lot of good people.
And no, it's not your job to worry about other people's issues, but if you think that 'being stressed in an interview setting' == 'unable to collaborate' (this is what I read from your ridiculous strawmen), you're going to pass on a lot of good people.
> Sounds like stress doesn't impact you, even in interview settings
No, it sounds like you didn't read the top of the thread, where we're specifically discussing Whiteboard Prompts. I agree interviews are stressful; I disagree with the argument that "Whiteboard Prompts" are so much higher stress than the rest of the interview that we can't conclude anything from them about the candidate.
I further argue that if one can't perform in a Whiteboard Prompt, wherein the entire goal is to have the type of conversation about a problem that engineers are paid to have about problems rather than coming to a code solution, than one will probably be difficult to work with in any situation which involves some pressure. Some classic examples of equally high pressure situations as talking to an interviewer you don't know but who is probably on your level include things I do each week:
* speak with stakeholders about progress
* opine to stakeholders about complexity of features, timelines and methodologies of their implementation, and possible blockers
* work with my team & extended team members to give Code Reviews, where often they are my direct report or someone else's direct report
So, I conclude that if one is able to excel technically, but completely unable to handle this socialization aspect of interviews, than in a large corporation where you frequently have rotating casts of external stakeholders, this implies you will not be fit for any of the work interfacing or collaborating with colleagues outside the team. This implies this person will be a low performer if hired into this team.
Call it a "ridiculous strawmen" if you want, but I don't see why we should completely discard testing of someone's ability to interact in new situations, otherwise surely I could argue in the ridiculously reductionist way you are interpreting my multiple, specific comments elucidating my stance that TripleByte has solved the interviewing problem.
No, it sounds like you didn't read the top of the thread, where we're specifically discussing Whiteboard Prompts. I agree interviews are stressful; I disagree with the argument that "Whiteboard Prompts" are so much higher stress than the rest of the interview that we can't conclude anything from them about the candidate.
I further argue that if one can't perform in a Whiteboard Prompt, wherein the entire goal is to have the type of conversation about a problem that engineers are paid to have about problems rather than coming to a code solution, than one will probably be difficult to work with in any situation which involves some pressure. Some classic examples of equally high pressure situations as talking to an interviewer you don't know but who is probably on your level include things I do each week:
* speak with stakeholders about progress
* opine to stakeholders about complexity of features, timelines and methodologies of their implementation, and possible blockers
* work with my team & extended team members to give Code Reviews, where often they are my direct report or someone else's direct report
So, I conclude that if one is able to excel technically, but completely unable to handle this socialization aspect of interviews, than in a large corporation where you frequently have rotating casts of external stakeholders, this implies you will not be fit for any of the work interfacing or collaborating with colleagues outside the team. This implies this person will be a low performer if hired into this team.
Call it a "ridiculous strawmen" if you want, but I don't see why we should completely discard testing of someone's ability to interact in new situations, otherwise surely I could argue in the ridiculously reductionist way you are interpreting my multiple, specific comments elucidating my stance that TripleByte has solved the interviewing problem.
I did read the thread. Someone was saying how whiteboard prompts are scary and has a high false negative rate.
You took that and turned it into "if you find interviews scary, that's a signal that you have unreasonable social skills and you are unable to think on your feet".
In other words, you believe that the false negative rate is not so high, because you believe that the the stress induced by an interview is similar enough to the stress induced in other stressful situations that are common in the work place.
The only point I am trying to make is that yes, interviews are very different than the other scenarios that you mentioned.
Here's one reason: in an whiteboard problem solving session, the other person has explictly prepared something that they know more about than you. On top of that, you are both aware of this fact. This almost never happens in a normal work-collaboration. A collabaration happens because both parties have something to offer. If not, they'd be explaining to you, and not the other way around.
The only point I am trying to make is that yes, interviews are very different than the other scenarios that you mentioned.
Here's one reason: in an whiteboard problem solving session, the other person has explictly prepared something that they know more about than you. On top of that, you are both aware of this fact. This almost never happens in a normal work-collaboration. A collabaration happens because both parties have something to offer. If not, they'd be explaining to you, and not the other way around.
> In other words, you believe that the false negative rate is not so high, because you believe that the the stress induced by an interview is similar enough to the stress induced in other stressful situations that are common in the work place.
No, not really.
I think people who have a weakness are most likely to attack, without evidence, things that expose that weakness as unfair. People in this thread are going so as to call the interview technique of whiteboarding a problem “practically inhumane”.
You are right that I think inability to talk thru ones train of thought on a new problem to them to be a signal of inability. I think whiteboarding exposes this kind of weakness. I also think it has many causes: extreme stress reaction, bad social skills, lack of domain knowledge, bad communication skills, and plain old stupidity.
So to read that “whiteboarding” should be disallowed from interviewing, because it is so much more high stress than every other part of the process, despite those parts also having equal importance in the overall stakes, really needs some elucidation. I accept anecdotes from workers who claim they are only low performers in the interview because of stress or anxiety, but I strongly doubt unmitigated social anxiety presents as a detriment to ones apparent job performance in interviews but not jobs at a high enough rate to give any credence to these claims without some sort of data driven backing.
No, not really.
I think people who have a weakness are most likely to attack, without evidence, things that expose that weakness as unfair. People in this thread are going so as to call the interview technique of whiteboarding a problem “practically inhumane”.
You are right that I think inability to talk thru ones train of thought on a new problem to them to be a signal of inability. I think whiteboarding exposes this kind of weakness. I also think it has many causes: extreme stress reaction, bad social skills, lack of domain knowledge, bad communication skills, and plain old stupidity.
So to read that “whiteboarding” should be disallowed from interviewing, because it is so much more high stress than every other part of the process, despite those parts also having equal importance in the overall stakes, really needs some elucidation. I accept anecdotes from workers who claim they are only low performers in the interview because of stress or anxiety, but I strongly doubt unmitigated social anxiety presents as a detriment to ones apparent job performance in interviews but not jobs at a high enough rate to give any credence to these claims without some sort of data driven backing.
I think you're missing the real world nuance here, anxiety isn't as on/off as you make it out to be. For me, a developer with social anxiety, there is a huge difference between an interview environment and talking out a problem or code review with a team I'm familiar with. The two main factors for me are one my anxiety just skyrockets around new people but quickly calms down after being around someone for a couple days/weeks and getting used to them. The second is one aspect of my anxiety be is an abundance of worrying about people judging me/what I'm doing/wearing/whatever and in an interview the interviewer is doing just that: judging my performance & worthiness to join their team.
Anyways that was a long winded way to say it is entirely possible for someone to be anxious and generally bad at interviewing while also being capable of being a productive team member.
Anyways that was a long winded way to say it is entirely possible for someone to be anxious and generally bad at interviewing while also being capable of being a productive team member.
Hi there. I have social anxiety. There's a fucking huge difference between performing in front of someone more important in a job interview compared to talking to peers in a casual setting.
So the 50+ percent of the population who have a problem with performance under high stress situation have a giant red flag above their head? We are doing something wrong.
Well, yes. If the job requires you to perform under stress, being unable to do so should certainly be considered for disqualification.
Job interviews aren't perfect and everybody wants something different from them. But maybe the employer actually is looking for something you don't have and wouldn't it be better to find out before joining?
Job interviews aren't perfect and everybody wants something different from them. But maybe the employer actually is looking for something you don't have and wouldn't it be better to find out before joining?
I kinda agree as a I have posted elsewhere:
"The scariest part of all this is that people think this is a good way to filter candidates. It shows you that the company is populated by stress tolerant, herd mentality thinkers. I see them as a cross between army grunts and high school nerds. Simple fast logic, drawn to rules and systems, prone to being swayed by appeals to authority. Low sensitivity nervous systems meaning high stress tolerance and low ability to perceive, synthesise and abstract. Working with these guys is like wearing a straight jacket anyway - avoid like the plague. Unless you happen to fit that model, in which case go right ahead, you do you, but please be aware that there are other kinds of human out there who can add value outside of the cookie cutter."
But we have a problem where we are recruiting all kinds of personalities into a market and then after significant investment in education telling them that they aren't suited for the role.
Fundamentally what is going on here is a battle between different kinds of personalities and it's annoying that I'm in the minority. But there is an argument here to say that this isn't a very economically productive situation. You need all kinds of people in an organisation.
"The scariest part of all this is that people think this is a good way to filter candidates. It shows you that the company is populated by stress tolerant, herd mentality thinkers. I see them as a cross between army grunts and high school nerds. Simple fast logic, drawn to rules and systems, prone to being swayed by appeals to authority. Low sensitivity nervous systems meaning high stress tolerance and low ability to perceive, synthesise and abstract. Working with these guys is like wearing a straight jacket anyway - avoid like the plague. Unless you happen to fit that model, in which case go right ahead, you do you, but please be aware that there are other kinds of human out there who can add value outside of the cookie cutter."
But we have a problem where we are recruiting all kinds of personalities into a market and then after significant investment in education telling them that they aren't suited for the role.
Fundamentally what is going on here is a battle between different kinds of personalities and it's annoying that I'm in the minority. But there is an argument here to say that this isn't a very economically productive situation. You need all kinds of people in an organisation.
I get the stress tolerant part although I don't agree that this would imply a low ability to perceive, synthesise and abstract. Why would the people that pass such an interview be herd mentality thinkers?
Because the ability to perceive is tuned by the reactivity of your nervous system. Think of it like the sensitivity on a microphone. A normal microphone might work well to record a conversation at a party. But it wont capture the beating of a fly's wings. And conversely a high sensitivity microphone will be useless at a party, it will just get blown out by the noise.
In the same way the minds of people are tuned to different levels of stimulation. People with low sensitivity are better suited to noisy information rich environments. People with high sensitivity work better in quieter less information dense ones. You might want a highly sensitive person to conduct a one on one interview, because they can better pickup the subtleties of the interaction. But you don't want that for a lawyer who has to work in a busy courtroom.
That covers perception. The synthesis comes both from the lower perception floor and from the tendency for those things that are perceived to perpetuate within the nervous system. A reactive nervous system carries waves upon its surface more readily. It reacts even to itself more strongly. The inter-neuron gain is higher. Consequently a highly sensitive nervous system (i.e. introversion) is associated with withdrawal from high stimulation environments and the perception and processing of subtlety.
The herd mentality comes from being less able to pick up the subtleties that let you know the group think is not an accurate picture of reality. If you cannot perceive the discrepancies then you cannot doubt the status quo.
So when you do public whiteboard interviews you are selecting for low sensitivity individuals who are on average less able to discern subtle patterns in perceptions coming in at low volume. Does that sound like the kind of person who excels at coding?
Although it might sound like someone who can work the least sub optimally in a noisy open office.
In the same way the minds of people are tuned to different levels of stimulation. People with low sensitivity are better suited to noisy information rich environments. People with high sensitivity work better in quieter less information dense ones. You might want a highly sensitive person to conduct a one on one interview, because they can better pickup the subtleties of the interaction. But you don't want that for a lawyer who has to work in a busy courtroom.
That covers perception. The synthesis comes both from the lower perception floor and from the tendency for those things that are perceived to perpetuate within the nervous system. A reactive nervous system carries waves upon its surface more readily. It reacts even to itself more strongly. The inter-neuron gain is higher. Consequently a highly sensitive nervous system (i.e. introversion) is associated with withdrawal from high stimulation environments and the perception and processing of subtlety.
The herd mentality comes from being less able to pick up the subtleties that let you know the group think is not an accurate picture of reality. If you cannot perceive the discrepancies then you cannot doubt the status quo.
So when you do public whiteboard interviews you are selecting for low sensitivity individuals who are on average less able to discern subtle patterns in perceptions coming in at low volume. Does that sound like the kind of person who excels at coding?
Although it might sound like someone who can work the least sub optimally in a noisy open office.
[deleted]
[deleted]
Can you provide a source that 50+ % of the population find live-coding/collaboration interviews to be a high stress situation while the rest of the interview is not? I think you are conflating your personal feelings with objectivity until I understand why you think a majority of the population is significantly impacted by specifically this technique of interviewing but not others.
Additionally, if you find a whiteboard question to be high stress to the point of not being able to answer, while other candidates are smoothly answering it, demonstrating their chops both as a technical knowledge worker but also as a collaborative and communicative engineer, you bet your ass that's a giant red flag. Why should it not be?
Additionally, if you find a whiteboard question to be high stress to the point of not being able to answer, while other candidates are smoothly answering it, demonstrating their chops both as a technical knowledge worker but also as a collaborative and communicative engineer, you bet your ass that's a giant red flag. Why should it not be?
Please see my other answer, just below, for how I think about your response.
Also, omg I cant believe you just used the word 'chops'. It's like a fighter pilot from ww2 just teleported into the chat.
Also, omg I cant believe you just used the word 'chops'. It's like a fighter pilot from ww2 just teleported into the chat.
Your argument, as best as I can tell, is that people are completely unimodal in how much stress they can handle, and therefore are completely determined in multiple dimensions such as "sensitivity", "intelligence", and "attention to detail"?
I asked for a source that people find whiteboarding significantly more stressful than other forms of interviewing people are not complaining about here, and you gave some bullshit opinion that denigrates a massive amount of people simply for having stress management techniques / more advanced self-mastery.
Here's my opinion: technical interviews catch people who are not technically qualified, and whiteboard & other social aspects of interviewing catch people who are not socially qualified. I think that teams work together better with frequent and high-quality discourse. Do you?
I asked for a source that people find whiteboarding significantly more stressful than other forms of interviewing people are not complaining about here, and you gave some bullshit opinion that denigrates a massive amount of people simply for having stress management techniques / more advanced self-mastery.
Here's my opinion: technical interviews catch people who are not technically qualified, and whiteboard & other social aspects of interviewing catch people who are not socially qualified. I think that teams work together better with frequent and high-quality discourse. Do you?
You are conflating social skills stress tolerance and self mastery. The idea that you can think your way out of something that is a basic property of your nervous system is very insulting. What is happening here is that you are unable to step outside of yourself and to understand what it might be like to have a different kind of personality. What that tells me is that you are not emotionally intelligent. Someone with high social skills has an ability to step outside of themselves and appreciate the world from another's shoes that you seem not to possess. So the statement that I am in some way socially inept because I have a more perceptive but less stress tolerant nervous system, honestly falls a little flat. You might even say that being attuned to detail can make you better attuned to social detail.
I spent, what, 35 years. Maybe. Working on myself because I bought into the line that you can change your personality, that if you are not living up to what the status quo expects of you, then you need to work on yourself. And then I had kids. And I saw that they could not change. That their nature was not some malleable thing that listens to the whims of what the world expects from them. There are parts that can and do change, but the bedrock properties do not change, they are who and what they are and that is good.
You cannot step outside of yourself and admit that others may live in a different reality that finds things you find easy, difficult. And finds things you find difficult, easy. There are high stress tolerance people and they are great and good and useful. And you have low stress tolerance people and they are great and good and useful. But they are not the same, they have different abilities and to try to deny that is wrong headed.
I spent, what, 35 years. Maybe. Working on myself because I bought into the line that you can change your personality, that if you are not living up to what the status quo expects of you, then you need to work on yourself. And then I had kids. And I saw that they could not change. That their nature was not some malleable thing that listens to the whims of what the world expects from them. There are parts that can and do change, but the bedrock properties do not change, they are who and what they are and that is good.
You cannot step outside of yourself and admit that others may live in a different reality that finds things you find easy, difficult. And finds things you find difficult, easy. There are high stress tolerance people and they are great and good and useful. And you have low stress tolerance people and they are great and good and useful. But they are not the same, they have different abilities and to try to deny that is wrong headed.
Is that expected to be a major part of their job?
If no, then no, it's not.
If yes, perhaps. Interview settings are still quite different than what they're going to face on-the-job, but you do have to infer from something, so I'm not sure there's any real alternative.
If no, then no, it's not.
If yes, perhaps. Interview settings are still quite different than what they're going to face on-the-job, but you do have to infer from something, so I'm not sure there's any real alternative.
> Is that expected to be a major part of their job?
How do you collaborate with your teammates? I do it directly by approaching a problem in a shared coding or whiteboarding session, where we try to break it down to its barest bones and then implement logic to animate these bones. Once that is done, we begin to look at what types of higher-fidelity information we lost, and iterate.
This requires a good ability to communicate, write code in real-time, understand perfect is the enemy of good, and in general be able to simultaneously think deeply and support conversation / collaboration.
Of course, there are frequent async steps in such a workplace, but if you are the type of engineer who doesn't participate in these stages and instead can only pull tickets and write them to a spec, you are the type of engineer I don't want to hire. So I consider the signal:noise ratio quite high wrt watching someone melt under the same pressure as every other interviewee faced.
How do you collaborate with your teammates? I do it directly by approaching a problem in a shared coding or whiteboarding session, where we try to break it down to its barest bones and then implement logic to animate these bones. Once that is done, we begin to look at what types of higher-fidelity information we lost, and iterate.
This requires a good ability to communicate, write code in real-time, understand perfect is the enemy of good, and in general be able to simultaneously think deeply and support conversation / collaboration.
Of course, there are frequent async steps in such a workplace, but if you are the type of engineer who doesn't participate in these stages and instead can only pull tickets and write them to a spec, you are the type of engineer I don't want to hire. So I consider the signal:noise ratio quite high wrt watching someone melt under the same pressure as every other interviewee faced.
I agree that collaborating in real-time is important, but I disagree that interview conditions allow you to assess real-time collaboration abilities accurately.
In a real life situation, both you and your partner will be generally familiar with the problem at hand; if not there's an expectation that the person with more knowledge will be explaining not just the problem, but also the solutions that are already under consideration, so that everyone is on the same page.
In an interview situation, the solution is obviously withheld because the goal is to make the candidate derive it, so any back-and-forth is artificial - the interviewee is the one who has to do analysis while being scrutinized while the interviewer has already done the work.
And of course, normally your partner doesn't have control over whether you get hired or fired. If you were dropped into a room with the CTO and told that you had to solve a new-to-you problem they had already solved or you would be fired, you would probably find yourself performing much worse than if your team-mate asked you to weigh in on the problem that had been bouncing around on Slack for the past few days.
In a real life situation, both you and your partner will be generally familiar with the problem at hand; if not there's an expectation that the person with more knowledge will be explaining not just the problem, but also the solutions that are already under consideration, so that everyone is on the same page.
In an interview situation, the solution is obviously withheld because the goal is to make the candidate derive it, so any back-and-forth is artificial - the interviewee is the one who has to do analysis while being scrutinized while the interviewer has already done the work.
And of course, normally your partner doesn't have control over whether you get hired or fired. If you were dropped into a room with the CTO and told that you had to solve a new-to-you problem they had already solved or you would be fired, you would probably find yourself performing much worse than if your team-mate asked you to weigh in on the problem that had been bouncing around on Slack for the past few days.
While you're collaborating with your teammates, if you reference anything external or get it wrong or don't come up with an answer in 45 minutes you're fired!
Still stress free?
Still stress free?
So, you walk into job interviews with the feeling you have already earned the job? And that failing the interview is equivalent to being fired?
Weak strawman aside, I don't know what your point is. You can reference external things in most interviews I've executed. "Getting it wrong" is much likelier to apply to portions of the interview outside the Whiteboarding, like for example the coding portions where you find out if they can code. The whiteboarding is a loosely structured conversation hoping to understand how the candidate would approach the problem. Even not coming up with a perfect answer seems like it'd be far more stressful in the coding part than the whiteboarding part.
Indeed, every single thing you've listed is actually probably directly worse in other portions of the interview. Yet everyone only rails against whiteboarding in this thread, with no one answering my questions like "why is whiteboarding so much more stressful than each other piece" and "why do you think social anxiety would present cleanly as a detriment to your interview performance but never your job performance?
Too many strawmans, not enough substance.
Weak strawman aside, I don't know what your point is. You can reference external things in most interviews I've executed. "Getting it wrong" is much likelier to apply to portions of the interview outside the Whiteboarding, like for example the coding portions where you find out if they can code. The whiteboarding is a loosely structured conversation hoping to understand how the candidate would approach the problem. Even not coming up with a perfect answer seems like it'd be far more stressful in the coding part than the whiteboarding part.
Indeed, every single thing you've listed is actually probably directly worse in other portions of the interview. Yet everyone only rails against whiteboarding in this thread, with no one answering my questions like "why is whiteboarding so much more stressful than each other piece" and "why do you think social anxiety would present cleanly as a detriment to your interview performance but never your job performance?
Too many strawmans, not enough substance.
I think I'm picking up that your model of how a typical technical interview is structured is different from what the other commenters have in mind.
It sounds to me like you view the technical interview as being split into a high-level whiteboarding phase followed by a separate coding test that is not done on the whiteboard.
In every interview I've had that involved coding and a whiteboard, I was expected to do everything on the whiteboard, from drawing diagrams and working through test cases to writing the actual code under evaluation, all in front of the interviewer.
Under this (very common) interview system, there is no "other piece" for whiteboarding to be more or less stressful than - it's embedded into the entire interview! (Yes, there are other interviews throughout the day that might only involve block diagrams or ask behavioral questions with no coding whatsoever, but I don't think that's what most people have in mind when they're told to prepare for an algorithmic whiteboard interview).
I think the interview process would be a lot better if it started with a no-coding-required whiteboard collaboration phase, followed by coding solo, and then ending with a mini code-review on the whiteboard. This is similar to the private technical interview procedure described in the paper.
As to your other question, I'm not sure why you seem so dead-set against the idea that performing analytical tasks while being evaluated in real-time would be harder than doing so under normal conditions. That's what this paper set out to investigate.
It sounds to me like you view the technical interview as being split into a high-level whiteboarding phase followed by a separate coding test that is not done on the whiteboard.
In every interview I've had that involved coding and a whiteboard, I was expected to do everything on the whiteboard, from drawing diagrams and working through test cases to writing the actual code under evaluation, all in front of the interviewer.
Under this (very common) interview system, there is no "other piece" for whiteboarding to be more or less stressful than - it's embedded into the entire interview! (Yes, there are other interviews throughout the day that might only involve block diagrams or ask behavioral questions with no coding whatsoever, but I don't think that's what most people have in mind when they're told to prepare for an algorithmic whiteboard interview).
I think the interview process would be a lot better if it started with a no-coding-required whiteboard collaboration phase, followed by coding solo, and then ending with a mini code-review on the whiteboard. This is similar to the private technical interview procedure described in the paper.
As to your other question, I'm not sure why you seem so dead-set against the idea that performing analytical tasks while being evaluated in real-time would be harder than doing so under normal conditions. That's what this paper set out to investigate.
> Under this (very common) interview system
I have never experienced such an interview, both when I am the interviewer and the interviewee, and I've been thru the entire process at multiple FAANGs. What you describe - literally coding on a whiteboard - is a far cry from whiteboarding a problem in my mind, this is a fair point
> I'm not sure why you seem so dead-set against the idea that performing analytical tasks while being evaluated in real-time would be harder than doing so under normal conditions
The entire interview has this elevated difficulty. You are constantly being judged and evaluated over the course of the interview. I have consistently asked for people to clarify why the Whiteboard part is elevated over any other technical portion in how stressful it is and how much that impacts performance, and I mostly have been attacked with strawmen. I appreciate your good-faith argument here, and certainly agree with you that conducting an entire interview including coding with a pen instead of a keyboard is a braindead practice. I will be more explicit of my assumptions next time as I would not have anticipated entire interviews to be conducted on whiteboards when you are trying to also find people who can code. I would expect what I've seen, which is that you use multiple different evaluation techniques across different scenarios and levels of interactivity and then you evaluate the tradespace, in which the whiteboard & collaborative time gives valuable data on how personality interop would work with your team.
I have never experienced such an interview, both when I am the interviewer and the interviewee, and I've been thru the entire process at multiple FAANGs. What you describe - literally coding on a whiteboard - is a far cry from whiteboarding a problem in my mind, this is a fair point
> I'm not sure why you seem so dead-set against the idea that performing analytical tasks while being evaluated in real-time would be harder than doing so under normal conditions
The entire interview has this elevated difficulty. You are constantly being judged and evaluated over the course of the interview. I have consistently asked for people to clarify why the Whiteboard part is elevated over any other technical portion in how stressful it is and how much that impacts performance, and I mostly have been attacked with strawmen. I appreciate your good-faith argument here, and certainly agree with you that conducting an entire interview including coding with a pen instead of a keyboard is a braindead practice. I will be more explicit of my assumptions next time as I would not have anticipated entire interviews to be conducted on whiteboards when you are trying to also find people who can code. I would expect what I've seen, which is that you use multiple different evaluation techniques across different scenarios and levels of interactivity and then you evaluate the tradespace, in which the whiteboard & collaborative time gives valuable data on how personality interop would work with your team.
> I have never experienced such an interview, both when I am the interviewer and the interviewee, and I've been thru the entire process at multiple FAANGs.
I've been through the all-day on-site interview at two FAANGs and every coding interview involved me standing at the whiteboard for the entire interview. As the saying goes, data is not the plural form of anecdote, but it's certainly common enough as a format that it has to be considered when discussing whiteboard interviews.
> The entire interview has this elevated difficulty. You are constantly being judged and evaluated over the course of the interview. I have consistently asked for people to clarify why the Whiteboard part is elevated over any other technical portion in how stressful it is and how much that impacts performance, and I mostly have been attacked with strawmen.
I agree that the entire interview has elevated difficulty, but which aspects are intrinsic / essential and which parts could we remove with minimal impact to the effectiveness of the interview?
For example, we both seem to be in agreement that writing code with a pen makes the whole interview harder without facilitating what we usually want to measure - the candidate's ability to reason about complex problems and write good code to solve those problems.
From the findings in the paper, just having a proctor in the room is correlated with the candidates performing worse on the programming task (as measured by writing code that passes the test cases). I'm sure there are benefits to being able to observe the candidate directly, but those benefits should be weighed against the drawbacks and evaluated in light of whether or not it's really a net positive, regardless of whether the interview process as a whole will still be stressful in other ways.
I've been through the all-day on-site interview at two FAANGs and every coding interview involved me standing at the whiteboard for the entire interview. As the saying goes, data is not the plural form of anecdote, but it's certainly common enough as a format that it has to be considered when discussing whiteboard interviews.
> The entire interview has this elevated difficulty. You are constantly being judged and evaluated over the course of the interview. I have consistently asked for people to clarify why the Whiteboard part is elevated over any other technical portion in how stressful it is and how much that impacts performance, and I mostly have been attacked with strawmen.
I agree that the entire interview has elevated difficulty, but which aspects are intrinsic / essential and which parts could we remove with minimal impact to the effectiveness of the interview?
For example, we both seem to be in agreement that writing code with a pen makes the whole interview harder without facilitating what we usually want to measure - the candidate's ability to reason about complex problems and write good code to solve those problems.
From the findings in the paper, just having a proctor in the room is correlated with the candidates performing worse on the programming task (as measured by writing code that passes the test cases). I'm sure there are benefits to being able to observe the candidate directly, but those benefits should be weighed against the drawbacks and evaluated in light of whether or not it's really a net positive, regardless of whether the interview process as a whole will still be stressful in other ways.
Only if we consider technical work a performance art.
Only earlier today I told one of the new starters: can you please not stand behind me while I work. They asked "why?", my response: "because it fucking irritates me".
My work isn't a stage show. If we're actively involved in a skill sharing session together, or conveying meaningful information, that's different entirely.
But I think it's perfectly normal to not want to be the centre of anyone's attention while your doing knowledge / technical work.
Only earlier today I told one of the new starters: can you please not stand behind me while I work. They asked "why?", my response: "because it fucking irritates me".
My work isn't a stage show. If we're actively involved in a skill sharing session together, or conveying meaningful information, that's different entirely.
But I think it's perfectly normal to not want to be the centre of anyone's attention while your doing knowledge / technical work.
> But I think it's perfectly normal to not want to be the centre of anyone's attention while your doing knowledge / technical work
You are indeed involved in a skill-sharing - well, demonstrating - session where you are meant to convey meaningful information about your abilities as a colleague to the audience. Your analogy has therefore directly fallen apart.
Also, I am not really sure what you are getting at with this language, but in general, if I was to have an interviewee who was to state the interview "fucking irritates" them, it would most certainly not be a successful interview.
Taking a good faith chomp at what you've said, representing an interview as "performance art" when it is probably trying to suss out your skills again seems like the kind of lack-of-social-skills that I would love to suss out in an interview. It's not like you are going to be asked to understand and implement a distributed algorithm for the first time in your life, they will take topics that should be at-worst adjacent to your knowledge base and ask you about them.
In an interview, I expect the interviewee, as someone claiming they have the capability to be my colleague, to then demonstrate that we can discuss, reason about, and explore in real-time solutions to problems. If you consider the mere act of collaborating with a colleague "performance art", it most certainly is a bad sign for how well your knowledge base will filter to the team thru any means except them reading your code.
You are indeed involved in a skill-sharing - well, demonstrating - session where you are meant to convey meaningful information about your abilities as a colleague to the audience. Your analogy has therefore directly fallen apart.
Also, I am not really sure what you are getting at with this language, but in general, if I was to have an interviewee who was to state the interview "fucking irritates" them, it would most certainly not be a successful interview.
Taking a good faith chomp at what you've said, representing an interview as "performance art" when it is probably trying to suss out your skills again seems like the kind of lack-of-social-skills that I would love to suss out in an interview. It's not like you are going to be asked to understand and implement a distributed algorithm for the first time in your life, they will take topics that should be at-worst adjacent to your knowledge base and ask you about them.
In an interview, I expect the interviewee, as someone claiming they have the capability to be my colleague, to then demonstrate that we can discuss, reason about, and explore in real-time solutions to problems. If you consider the mere act of collaborating with a colleague "performance art", it most certainly is a bad sign for how well your knowledge base will filter to the team thru any means except them reading your code.
> You are indeed involved in a skill-sharing - well, demonstrating - session where you are meant to convey meaningful information about your abilities as a colleague to the audience.
If you were the manager at an architectural firm, would you have a candidate come in and invite them to design a bridge on a whiteboard? Interestingly, at the age of about 8, I could do this by remembering what the structure looked like; simple memorization. At 12, I competed in a bridge building competition, using balsa wood; I could draw a bridge that used good basic design principles. At 16, I could spend hours drawing a realistic looking design, annotating lengths and angles, design features. I know NOTHING about bridge building. Never have, beyond the basic low hanging fruit that anyone who's taken a basic physics class will be able to derive.
There simply isn't time during an interview to demonstrate a full professional design pipeline in ANY meaningful way. It's simply bullshit. This is why handing out problems to people as they arrive at interviews like some fucked up high school pop quiz makes you and your company look like a fucked up high school, full of pedantic pedagogy.
> Also, I am not really sure what you are getting at with this language, but in general, if I was to have an interviewee who was to state the interview "fucking irritates" them, it would most certainly not be a successful interview.
What an odd response. I would ask them what about the interview "fucking irritated" them, because there might be something terribly stupid about our interview process, and bettering the interview process is extraordinarily important to a business.
> Taking a good faith chomp at what you've said, representing an interview as "performance art" when it is probably trying to suss out your skills again seems like the kind of lack-of-social-skills that I would love to suss out in an interview.
Social skills are a skillset highly valued for a social role at a company, like HR, Sales, Business Development, etc. Engineering skills are highly valued for engineering roles, such as Lead Engineer, Architect, R&D Lead, etc. You don't interview your sales people and hand them a paper asking them to derive the quadratic formula from first principles; that would be idiotic. Some might be able to do it, but then why are you even asking them to do BS that is unimportant for a newly hired sales person.
> to then demonstrate that we can discuss, reason about, and explore in real-time solutions to problems.
This is simply not how most fields in science and engineering work. There are times when it's appropriate and needed to discuss, plan, and share information, but that is a VERY small slice of the pie. Forcing people to discuss, realtime, how they are going about solving a problem of any complexity, speaks to an awful culture saturated with bureaucracy and groupthink. A sit-down chat or planning meeting on occasion is really important, but should gazing play-by-play? Disgusting.
If you were the manager at an architectural firm, would you have a candidate come in and invite them to design a bridge on a whiteboard? Interestingly, at the age of about 8, I could do this by remembering what the structure looked like; simple memorization. At 12, I competed in a bridge building competition, using balsa wood; I could draw a bridge that used good basic design principles. At 16, I could spend hours drawing a realistic looking design, annotating lengths and angles, design features. I know NOTHING about bridge building. Never have, beyond the basic low hanging fruit that anyone who's taken a basic physics class will be able to derive.
There simply isn't time during an interview to demonstrate a full professional design pipeline in ANY meaningful way. It's simply bullshit. This is why handing out problems to people as they arrive at interviews like some fucked up high school pop quiz makes you and your company look like a fucked up high school, full of pedantic pedagogy.
> Also, I am not really sure what you are getting at with this language, but in general, if I was to have an interviewee who was to state the interview "fucking irritates" them, it would most certainly not be a successful interview.
What an odd response. I would ask them what about the interview "fucking irritated" them, because there might be something terribly stupid about our interview process, and bettering the interview process is extraordinarily important to a business.
> Taking a good faith chomp at what you've said, representing an interview as "performance art" when it is probably trying to suss out your skills again seems like the kind of lack-of-social-skills that I would love to suss out in an interview.
Social skills are a skillset highly valued for a social role at a company, like HR, Sales, Business Development, etc. Engineering skills are highly valued for engineering roles, such as Lead Engineer, Architect, R&D Lead, etc. You don't interview your sales people and hand them a paper asking them to derive the quadratic formula from first principles; that would be idiotic. Some might be able to do it, but then why are you even asking them to do BS that is unimportant for a newly hired sales person.
> to then demonstrate that we can discuss, reason about, and explore in real-time solutions to problems.
This is simply not how most fields in science and engineering work. There are times when it's appropriate and needed to discuss, plan, and share information, but that is a VERY small slice of the pie. Forcing people to discuss, realtime, how they are going about solving a problem of any complexity, speaks to an awful culture saturated with bureaucracy and groupthink. A sit-down chat or planning meeting on occasion is really important, but should gazing play-by-play? Disgusting.
> If you were the manager at an architectural firm, would you have a candidate come in and invite them to design a bridge on a whiteboard?
Let's say I were.
> I know NOTHING about bridge building. Never have, beyond the basic low hanging fruit that anyone who's taken a basic physics class will be able to derive.
These would be exactly the kind of things the SOCIAL aspect of that whiteboard interview would dig into. Indeed, same with software engineering. I am sure that many people would be able to solve the whiteboard problem I give; that's quite directly not the point. Instead, the point is to see if they can demonstrate their ability to say things like:
* why did you use these lengths and angles? If I were to make this modification, would it have structural or design implications?
* I like the structure overall, but you have "<N>" of some substructure. Why did you pick that quantity? What changes would you make if the client requested more or less to keep a similar feel?
The person would then be able to demonstrate that they don't know NOTHING about bridge building and simply recreated a thing they luckily knew the shape of, but instead dig into first principles, emergent properties, tradespaces that emerged from their design decisions, and why they navigated said tradespace how they did.
> What an odd response
Right, I'm talking about how social skills are an important part of being in a workplace - you do, after all, interact with your coworkers up to 40h a week - and then this fellow tells a story about how he as a senior engineer told a new hire they were "fucking irritating" him as a demonstration of his superior social skills, and now you're trying to pull this? Unbelievable.
> It's simply bullshit
> makes you and your company look like a fucked up hiigh school, full of pedantic pedgagogy
> You don't have sales people do quadratic formulas
> Disgusting
You're quite unpleasant, and arguing directly in bad faith
> Engineering skills are highly valued for engineering roles
And the ability to communicate is a critical Engineering skill. How on earth everyone in this thread thinks asking an Engineer to talk for 45 minutes about a problem is analogous to asking a sales crew to derive a quadratic formula would've been beyond me, but I've seen about 100 strawmans in this thread.
Design work is not a VERY small slice of the pie unless you're entrenched in some org with humungous momentum and years of technical planning accomplished for you. In that case, you can probably afford to hire anti-social fucks who will just sit in their cave and code by themselves, never collaborating, failing to share critical business knowledge because talking to someone else isn't in their job description.
Let's say I were.
> I know NOTHING about bridge building. Never have, beyond the basic low hanging fruit that anyone who's taken a basic physics class will be able to derive.
These would be exactly the kind of things the SOCIAL aspect of that whiteboard interview would dig into. Indeed, same with software engineering. I am sure that many people would be able to solve the whiteboard problem I give; that's quite directly not the point. Instead, the point is to see if they can demonstrate their ability to say things like:
* why did you use these lengths and angles? If I were to make this modification, would it have structural or design implications?
* I like the structure overall, but you have "<N>" of some substructure. Why did you pick that quantity? What changes would you make if the client requested more or less to keep a similar feel?
The person would then be able to demonstrate that they don't know NOTHING about bridge building and simply recreated a thing they luckily knew the shape of, but instead dig into first principles, emergent properties, tradespaces that emerged from their design decisions, and why they navigated said tradespace how they did.
> What an odd response
Right, I'm talking about how social skills are an important part of being in a workplace - you do, after all, interact with your coworkers up to 40h a week - and then this fellow tells a story about how he as a senior engineer told a new hire they were "fucking irritating" him as a demonstration of his superior social skills, and now you're trying to pull this? Unbelievable.
> It's simply bullshit
> makes you and your company look like a fucked up hiigh school, full of pedantic pedgagogy
> You don't have sales people do quadratic formulas
> Disgusting
You're quite unpleasant, and arguing directly in bad faith
> Engineering skills are highly valued for engineering roles
And the ability to communicate is a critical Engineering skill. How on earth everyone in this thread thinks asking an Engineer to talk for 45 minutes about a problem is analogous to asking a sales crew to derive a quadratic formula would've been beyond me, but I've seen about 100 strawmans in this thread.
Design work is not a VERY small slice of the pie unless you're entrenched in some org with humungous momentum and years of technical planning accomplished for you. In that case, you can probably afford to hire anti-social fucks who will just sit in their cave and code by themselves, never collaborating, failing to share critical business knowledge because talking to someone else isn't in their job description.
I absolutely hate the interviews that are just an algo challenge and the interviewer is waiting on me to finish to evaluate my solution. I flip out, start thinking that I'm taking too long and it just goes downhill from there.
On the other hand, the questions where I can talk through a problem are by far my favorite. Every problem in our field can be talked about for days. There's just so much to discuss:
- different approaches
- performance considerations
- perceived complexity
- and a hundred tangents I could go on to
In my opinion, it is the interviewer's job to get the candidate to talk. If the candidate can't speak a word out of their mouth, the interviewer failed. When I interview I just want to get you to talk...tell me stories, tell me tangents, tell me the stupid stuff, tell me the smart stuff. Tell me what kind of solutions you prefer and why?
Reading your comment made me think that maybe this isn't the best approach. Maybe some people want me to shut up and just do the hard math problem on their own.
On the other hand, the questions where I can talk through a problem are by far my favorite. Every problem in our field can be talked about for days. There's just so much to discuss:
- different approaches
- performance considerations
- perceived complexity
- and a hundred tangents I could go on to
In my opinion, it is the interviewer's job to get the candidate to talk. If the candidate can't speak a word out of their mouth, the interviewer failed. When I interview I just want to get you to talk...tell me stories, tell me tangents, tell me the stupid stuff, tell me the smart stuff. Tell me what kind of solutions you prefer and why?
Reading your comment made me think that maybe this isn't the best approach. Maybe some people want me to shut up and just do the hard math problem on their own.
It doesn't seem difficult to msybf ask the candidate, either by phone or email or both, what or how they'd prefer to be interviewed.
Maybe interviewers could offer maybe say five methods or key topics and ask the candidate to choose at least two or three.
Maybe interviewers could offer maybe say five methods or key topics and ask the candidate to choose at least two or three.
Even worse is when you solve the problem in a way they didn't expect or plan on, and they spend the time derailing you off your solution and back to the one they understand.
I had a miserable interview experience once were I gave a tree processing solution in a functional, tail recursive way. They spent the interview sliding me back into a mutable, iterative approach, and it really through me off. Additionally, I don't think they even understood the functional solution. Half the time, it felt like I was needing to guess where they'd like me to go.
And we all know working and software development is nothing like that. At no point in that particular interview was I asked to describe projects I had worked on. When I left, they had no idea what I had even worked on previously because they didn't ask. They just asked a tree question, which someone could look up over an evening or weekend. So pointless.
I had a miserable interview experience once were I gave a tree processing solution in a functional, tail recursive way. They spent the interview sliding me back into a mutable, iterative approach, and it really through me off. Additionally, I don't think they even understood the functional solution. Half the time, it felt like I was needing to guess where they'd like me to go.
And we all know working and software development is nothing like that. At no point in that particular interview was I asked to describe projects I had worked on. When I left, they had no idea what I had even worked on previously because they didn't ask. They just asked a tree question, which someone could look up over an evening or weekend. So pointless.
The scariest part of all this is that people think this is a good way to filter candidates. It shows you that the company is populated by stress tolerant, herd mentality thinkers. I see them as a cross between army grunts and high school nerds. Simple fast logic, drawn to rules and systems, prone to being swayed by appeals to authority. Low sensitivity nervous systems meaning high stress tolerance and low ability to perceive, synthesise and abstract. Working with these guys is like wearing a straight jacket anyway - avoid like the plague. Unless you happen to fit that model, in which case go right ahead, you do you, but please be aware that there are other kinds of human out there who can add value outside of the cookie cutter.
That's an awful lot of insulting generalization you're labeling people with, simply for being able to handle stress better than you. You are also completely overlooking the people for whom stress-handling doesn't come into play because it's not a very stressful situation for them. (This set of people is interesting to me, because it seems plausible that "gets less stressed by being asked about algorithms and system design" might correlate to "has a higher skill level and finds the problems easier and simpler.")
How would you evaluate a candidate's problem solving skills? "Trust me, I can do it, I just can't show you" doesn't seem particularly compelling if I'm looking to hire someone.
How would you evaluate a candidate's problem solving skills? "Trust me, I can do it, I just can't show you" doesn't seem particularly compelling if I'm looking to hire someone.
And yet here and elsewhere, for the same comment I have more upvotes than downvotes.
Some of what you are saying is accurate, but it's dwarfed by the noise that the different kinds of brains that you interview are producing. One of the most fundamental properties of a brain is it's sensitivity to stimulation. This is how 'activated' it gets by sensory stimulation. A loud annoying sound for one person is barely noticed by another.
People who have reactive nervous systems are introverted, and more easily stressed. It's not a symptom of their upbringing (necessarily, although it can be due to early life trauma, early loss of the mother etc), it's not necessarily about how they think about the ease or difficulty of the interview process - it can merely be from the large amount of information they are perceiving about the emotional and cognitive state of the interviewer.
This sensitivity of the nervous system may not be what you are looking for and that is the nature of the world. But do not be surprised when those you exclude because of a fundamental aspect of their nature react to the exclusion negatively.
Sensitive people have a lot to give. Sensitivity correlates well with openness and openness brings in fresh ideas and new approaches. Openness also correlates with IQ.
The reasoning is that - aside from the studies that you can look up and read - that the sensitive nervous system is more alert to its environment and therefore more creative - as creativity seems to be, in part, the ability to combine environmental perceptions fluidly. In order to be creative you have to perceive a lot and let it sit. This kind of personality also tends to process events more deeply, thoughts spread further and wider. This is bad for straightforward logic, because it creates background noise, but it's great for getting the big picture.
High sensitivity is quite prevalent within the gifted population. You also find a significantly greater proportion of gifted people afflicted with anxiety, which is not surprisingly connected with reactivity - sensitivity. There is a theory of giftedness which describes the phenomena entirely in terms of excitabilities. By excluding the easily excitable from your organisation you are literally excluding the gifted. I am - if the raven matrices you can find inline are to be believed at the bottom end of gifted. And yet I can appear to be an idiot if run through the standard white board process. I deal with it using pregabalin which deadens the nervous system enough to remain functional during an interview. But I do contract work these days and dont have to jump through hoops anymore.
So by only including individuals like yourself you are excluding sources of creativity, you are limiting yourself to a particular kind of person and by extension limiting the kinds of thoughts and approaches your organisation can have.
It's similar and indeed perhaps intertwined with the exclusion that minorities of all kinds find within the workplace dominated by the kind of culture that sees whiteboard interviews as a good thing.
Some of what you are saying is accurate, but it's dwarfed by the noise that the different kinds of brains that you interview are producing. One of the most fundamental properties of a brain is it's sensitivity to stimulation. This is how 'activated' it gets by sensory stimulation. A loud annoying sound for one person is barely noticed by another.
People who have reactive nervous systems are introverted, and more easily stressed. It's not a symptom of their upbringing (necessarily, although it can be due to early life trauma, early loss of the mother etc), it's not necessarily about how they think about the ease or difficulty of the interview process - it can merely be from the large amount of information they are perceiving about the emotional and cognitive state of the interviewer.
This sensitivity of the nervous system may not be what you are looking for and that is the nature of the world. But do not be surprised when those you exclude because of a fundamental aspect of their nature react to the exclusion negatively.
Sensitive people have a lot to give. Sensitivity correlates well with openness and openness brings in fresh ideas and new approaches. Openness also correlates with IQ.
The reasoning is that - aside from the studies that you can look up and read - that the sensitive nervous system is more alert to its environment and therefore more creative - as creativity seems to be, in part, the ability to combine environmental perceptions fluidly. In order to be creative you have to perceive a lot and let it sit. This kind of personality also tends to process events more deeply, thoughts spread further and wider. This is bad for straightforward logic, because it creates background noise, but it's great for getting the big picture.
High sensitivity is quite prevalent within the gifted population. You also find a significantly greater proportion of gifted people afflicted with anxiety, which is not surprisingly connected with reactivity - sensitivity. There is a theory of giftedness which describes the phenomena entirely in terms of excitabilities. By excluding the easily excitable from your organisation you are literally excluding the gifted. I am - if the raven matrices you can find inline are to be believed at the bottom end of gifted. And yet I can appear to be an idiot if run through the standard white board process. I deal with it using pregabalin which deadens the nervous system enough to remain functional during an interview. But I do contract work these days and dont have to jump through hoops anymore.
So by only including individuals like yourself you are excluding sources of creativity, you are limiting yourself to a particular kind of person and by extension limiting the kinds of thoughts and approaches your organisation can have.
It's similar and indeed perhaps intertwined with the exclusion that minorities of all kinds find within the workplace dominated by the kind of culture that sees whiteboard interviews as a good thing.
We want you to think out loud. It's a different skillset but an important one when solving a problem in a group. You might be formulating an idea in your head, but if you say it out loud, even an incomplete or "wrong" thought might be useful to someone else in a real world situation, as it might trigger an insight for them.
Yes, it is generally a good thing to be able to communicate while you're working on a problem within a group.
But rarely does someone get posed a difficult technical design question on the job and on the spot where they've had little or no opportunity to think about it ahead of time. Usually there is a ton of contextual knowledge leading up to those group discussions vs. being asked a question in an interview and then be expected to perform at that same level.
But rarely does someone get posed a difficult technical design question on the job and on the spot where they've had little or no opportunity to think about it ahead of time. Usually there is a ton of contextual knowledge leading up to those group discussions vs. being asked a question in an interview and then be expected to perform at that same level.
Or where the stakes are so high, resulting in this level of terror and paralysis in such a high percentage of candidates. I get the goal and why it is appealing, but in the end these tactics are not useful and are inhumane.
It depends on the role. I've always worked in sysadmin/devops, so a lot of programming is under duress of an outage. Even for the "regular engineers", I need to know that they can fix a production bug in real time, or at least figure out what to roll back.
And even for the non-duress situation, there are plenty of situations during normal work where a senior engineer will have to jump in and contribute to a problem they've never seen before.
And even for the non-duress situation, there are plenty of situations during normal work where a senior engineer will have to jump in and contribute to a problem they've never seen before.
Really?
I prefer myself and my coworkers to pause and think for a few minutes if needed and come up with a coherent idea.
Gibberish doesn’t really trigger any insight to me, if not distracting. Well reasoned thoughts and coherent sentences do.
I prefer myself and my coworkers to pause and think for a few minutes if needed and come up with a coherent idea.
Gibberish doesn’t really trigger any insight to me, if not distracting. Well reasoned thoughts and coherent sentences do.
Working through a problem out loud is definitely a bit of a different skill from just solving a complex problem on your own. I do find that saying something along the lines of "OK now I just need a minute to think through this next part" goes over well enough and can give you a minute to relax and think though.
I think the issue is "a minute." You still feel pressured to get moving and keep talking, even if it buys you a little time. My favorite part of the job is whiteboarding with colleagues, but in an interview there is something that feels inherently adversarial about it. This isn't necessarily the fault of the interviewer either, it's just sort of baked into the format. There's a natural power discrepancy: the interviewer makes the final call, and they selected the problem to begin with, which means you can't possibly be on equal footing. That imbalance means you spend the whole time on the back foot. I'm not sure there's a better format available, but that's definitely an unnatural environment and can lead people including me to perform much differently than they do in a normal work environment. Unfortunately it may be the best we can do.
I think the better format is to have the interviewee talk through a complex problem that they have already solved, whether it's a side project, a work project they can talk about, or whatever. It balances out the power imbalance, and it showcases how well the interviewee can explain and talk through problems, design decisions, what they learned, what worked, what didn't, etc. through a problem they are already familiar with.
These whiteboarding interviews where the interviewers pose some random question does nothing for either party.
These whiteboarding interviews where the interviewers pose some random question does nothing for either party.
I really like that approach. The tradeoff is that you're asking someone to spend a non-trivial amount of personal time prepping and it opens the door a bit more to BS'ing and just memorizing Stackoverflow or whatever. But presumably your interviewers are skilled enough to sniff that out. I personally feel that would be a better way to showcase my abilities.
The absolute worst is when interviewers keep interrupting the candidate. It shows a complete lack of empathy and it shows that the interviewer wants the interviewee to answer the way the interviewer thinks it should be answered.
I am very mindful of this when I interview and I think it’s my job as the interviewer to strain myself to try to understand the interviewee, not the other way around. I quietly try to understand my interviewee and don’t interrupt unless there’s a real reason since I want her to think freely and not feel stressed.
I am very mindful of this when I interview and I think it’s my job as the interviewer to strain myself to try to understand the interviewee, not the other way around. I quietly try to understand my interviewee and don’t interrupt unless there’s a real reason since I want her to think freely and not feel stressed.
I've been on both ends of this exchange. I always gave candidates time to work through it without interruption when I was the interviewer. This didn't prevent the worst cases of nerves, of course, but hopefully I didn't make it unnecessarily worse.
Conversely, how they treat me as I go through the process is a great way to judge the engineering culture of the org. If the interviewer feels license to cut me off, jump in, correct me on syntax as I'm still writing, or just generally sprays frustration, it's a high-confidence signal that I'll get the same treatment as a new coworker, and I have ended interviews accordingly.
Conversely, how they treat me as I go through the process is a great way to judge the engineering culture of the org. If the interviewer feels license to cut me off, jump in, correct me on syntax as I'm still writing, or just generally sprays frustration, it's a high-confidence signal that I'll get the same treatment as a new coworker, and I have ended interviews accordingly.
My least favorite is "let's implement something that's really easy to implement if you've spent time thinking about it or working with it before", except you don't get the time to think about it and you're judged poorly for providing anything other than the "obvious" optimal solution.
Yeah, I have encountered this myself in the past as well -- a subject matter expert that has worked in the domain for a decade and asks you about how to solve problems in their domain (while the actual position is a generalist one).
Luckily, I was able to remedy this a little bit by brushing up on common system architectures used by companies that I was interviewing with.
Luckily, I was able to remedy this a little bit by brushing up on common system architectures used by companies that I was interviewing with.
> My least favourite interview technique has to be "talk me through a complex problem on a whiteboard".
Let's say an interviewer reads your resume and is interested in a project. The interviewer comes to the interview prepared with some questions. They ask you to start with a high level summary of the project. Then, ask you - "What choices did you have to make during the project?". Then, ask you some specific questions about these choices. In each of these questions, the interviewer is not supposed to "evaluate your answer" but rather gain an understanding of your project and choices to be made. At the end of the interview, the interviewer would walk away with two things - (1) a clear picture of the project almost as good as the interviewee and (2) a clear picture of the design decisions the interviewee made (options they had and why they chose one over the other).
The above interview is a conversation and you are allowed to be silent. The process will be communicated to you upfront. In fact, silence is encouraged because the interviewer would much rather give you time to think over a spur of the moment answer.
Let's say an interviewer reads your resume and is interested in a project. The interviewer comes to the interview prepared with some questions. They ask you to start with a high level summary of the project. Then, ask you - "What choices did you have to make during the project?". Then, ask you some specific questions about these choices. In each of these questions, the interviewer is not supposed to "evaluate your answer" but rather gain an understanding of your project and choices to be made. At the end of the interview, the interviewer would walk away with two things - (1) a clear picture of the project almost as good as the interviewee and (2) a clear picture of the design decisions the interviewee made (options they had and why they chose one over the other).
The above interview is a conversation and you are allowed to be silent. The process will be communicated to you upfront. In fact, silence is encouraged because the interviewer would much rather give you time to think over a spur of the moment answer.
I would love, as an interviewer, to be able to do this style of interview every time.
It's astonishing how often you get nothing useful as an answer. Things like "my boss suggested it" as the reason for choices being made, or "not really" when asked about alternatives considered, or problems they ran into...
The hypothetical project whiteboard interview gives people who haven't done sufficiently interesting past projects and opportunity to show that they can think about them, at least...
It's astonishing how often you get nothing useful as an answer. Things like "my boss suggested it" as the reason for choices being made, or "not really" when asked about alternatives considered, or problems they ran into...
The hypothetical project whiteboard interview gives people who haven't done sufficiently interesting past projects and opportunity to show that they can think about them, at least...
> Let's say an interviewer reads your resume and is interested in a project.
That's not what they were talking about. The complex problem is supposed to be novel to the interviewee.
> In fact, silence is encouraged because the interviewer would much rather give you time to think over a spur of the moment answer.
Yes, they certainly say that. But for a normal person trying to figure out a novel problem, ignoring everything that's going on and sitting there silent for 30 seconds or longer while the person who single-handedly can determine whether you pass or fail - often depending on their mood - stares at you is incredibly difficult. And if their mood isn't good, they absolutely see the silence as not knowing the answer and push for you to move on. I had it happen to me on a Facebook interview. I personally can't concentrate enough to think through a difficult, multi-layered problem in that situation. It is in no way reflective of what I can do while working.
Online coding tests are more than sufficient. And a discussion afterwards about said tests is sufficient to make sure there was no cheating.
That's not what they were talking about. The complex problem is supposed to be novel to the interviewee.
> In fact, silence is encouraged because the interviewer would much rather give you time to think over a spur of the moment answer.
Yes, they certainly say that. But for a normal person trying to figure out a novel problem, ignoring everything that's going on and sitting there silent for 30 seconds or longer while the person who single-handedly can determine whether you pass or fail - often depending on their mood - stares at you is incredibly difficult. And if their mood isn't good, they absolutely see the silence as not knowing the answer and push for you to move on. I had it happen to me on a Facebook interview. I personally can't concentrate enough to think through a difficult, multi-layered problem in that situation. It is in no way reflective of what I can do while working.
Online coding tests are more than sufficient. And a discussion afterwards about said tests is sufficient to make sure there was no cheating.
This is of course a great way to interview, when you're only concerned about assessing competence. It seems to me, however, that assessing competence is one of several concerns that the interviewer is expected to balance.
In particular, HR is expected to structure the hiring process in such a way as to avoid accusations of bias. In order to achieve this, they are expected to come up with precise criteria for hiring, which is at odds with anything vaguely qualitative or holistic, let alone ecological.
The result is that we strictly rank-order candidates using hard measures. In doing so we sacrifice the ecological validity of the test.
In particular, HR is expected to structure the hiring process in such a way as to avoid accusations of bias. In order to achieve this, they are expected to come up with precise criteria for hiring, which is at odds with anything vaguely qualitative or holistic, let alone ecological.
The result is that we strictly rank-order candidates using hard measures. In doing so we sacrifice the ecological validity of the test.
Those are great, I'm more talking about a complex problem you're given and have to whiteboard the solution while making sure not to be silent for too long or you'll interrupted or given a hint.
I was offering an alternative to the novel problem on the spot. I’m glad you like this procedure better : )
[deleted]
If you work at a company where 90% of your developer interactions are, let's go to a whiteboard and hash it out, it kind of is the most job-relevant interview question you can ask outside of writing code.
[deleted]
Am I the only one that prefers white boarding over 8 hr "take home". I refuse to work with any company that thinks that I should invest 8 hrs in unpaid work while they invest 20 mins of their time reviewing my take home.
So, it's a long PDF. Can someone summarize the recommendation from the study?
The abstract of a paper is an executive summary. In case, that is not enough - you can look into the press release [0].
[0] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
[0] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
Previous Discussion 8 months ago : https://news.ycombinator.com/item?id=23848039
Or, an alternative title: Is Water Wet?
You do realize that the correct answer to any headline that ends with a question mark, is "No."
So I guess water is not wet.
I will say that I am comfortable with people not wanting to work with me, because I don't fit their "cultural criteria."
Take-home tests are a good idea, but you still have the issue that so many folks here have mentioned:
They don't just want you to solve the problem. They want you to solve it the way they want. Orthogonal approaches are generally not met with support.
I once had a take-home, where I was asked to produce an iOS app that used a certain dependency (that I had never used), and was asked to do so, using MVVM.
In four hours, I had it done. It was a localized, ship-ready app. I didn't use MVVM, because UIKit is designed specifically for MVC, and using other models actually makes the app larger and more prone to problems. It was a tiny app, and I didn't want to add any code that wasn't 100% necessary (basic Quality 101). I also encapsulated the dependency (Quality 101, again).
Did I mention that I had never even looked at the dependency API, before starting that project?
During the process, my home had an electrical emergency, and almost burned down, so I guess there was a bit of stress.
I could smell the burning rubber, as they peeled out (although it could have been the electrical fire). They never asked me why I made the choices I did. I strongly suspect that I was never in consideration anyway. I think they were checking off a "interviewed old guy, and he sucked" checkbox on an EEOC form.
I handle stress quite well, indeed. A quick glance at my CV tells you that I worked for one of the top Japanese firms in the world, for 27 years.
They refine and mass-produce stress in Tokyo. If you last 27 years at a joint like that, you can handle stress.
The folks that actually get around to working with me, seem to enjoy the experience. Funny how that happens.
So I guess water is not wet.
I will say that I am comfortable with people not wanting to work with me, because I don't fit their "cultural criteria."
Take-home tests are a good idea, but you still have the issue that so many folks here have mentioned:
They don't just want you to solve the problem. They want you to solve it the way they want. Orthogonal approaches are generally not met with support.
I once had a take-home, where I was asked to produce an iOS app that used a certain dependency (that I had never used), and was asked to do so, using MVVM.
In four hours, I had it done. It was a localized, ship-ready app. I didn't use MVVM, because UIKit is designed specifically for MVC, and using other models actually makes the app larger and more prone to problems. It was a tiny app, and I didn't want to add any code that wasn't 100% necessary (basic Quality 101). I also encapsulated the dependency (Quality 101, again).
Did I mention that I had never even looked at the dependency API, before starting that project?
During the process, my home had an electrical emergency, and almost burned down, so I guess there was a bit of stress.
I could smell the burning rubber, as they peeled out (although it could have been the electrical fire). They never asked me why I made the choices I did. I strongly suspect that I was never in consideration anyway. I think they were checking off a "interviewed old guy, and he sucked" checkbox on an EEOC form.
I handle stress quite well, indeed. A quick glance at my CV tells you that I worked for one of the top Japanese firms in the world, for 27 years.
They refine and mass-produce stress in Tokyo. If you last 27 years at a joint like that, you can handle stress.
The folks that actually get around to working with me, seem to enjoy the experience. Funny how that happens.
My latest system design interview was: design this app. Dude was like "I'm expecting you to lead the conversation". I was like, bitch there's a million things to talk about, at some point you have to tell me where you want the convo to go. Needless to say I failed, when I'm probably overqualified for the job, but heh...
I don't think you're overqualified. I'm not a systems designer by any means but here's how I'd lead this conversation:
"Ok cool. Before we get started, let me understand some non functional requirements. How many concurrent users do we expect? Is brief downtime acceptable?"
Then you listen to the answer.
Then you say "ok cool. So given what we just estsblished, it's making me lean towards X type of architecture. Here are trade-offs A,B, and C that are involved. Does this make sense to you so far?"
You listen to their answer. They will 100% of the time indicate to you if you are on the right track or missing something.
Then you go "ok cool. So on the high level we are going to have (for example) a client, a server, a proxy, a database, etc . Given what we discussed, I think the most complex part of the system will be X so let me begin by thinking through that and then we can pop back up to other parts. Does that sound good to you?"
Etc.
I am not looking to make you feel bad but being able to have this conversation in this way is a baseline skill of a senior engineer. Compare that to what you actually did and ask yourself if it's you or the "bitch" that has room to improve here.
"Ok cool. Before we get started, let me understand some non functional requirements. How many concurrent users do we expect? Is brief downtime acceptable?"
Then you listen to the answer.
Then you say "ok cool. So given what we just estsblished, it's making me lean towards X type of architecture. Here are trade-offs A,B, and C that are involved. Does this make sense to you so far?"
You listen to their answer. They will 100% of the time indicate to you if you are on the right track or missing something.
Then you go "ok cool. So on the high level we are going to have (for example) a client, a server, a proxy, a database, etc . Given what we discussed, I think the most complex part of the system will be X so let me begin by thinking through that and then we can pop back up to other parts. Does that sound good to you?"
Etc.
I am not looking to make you feel bad but being able to have this conversation in this way is a baseline skill of a senior engineer. Compare that to what you actually did and ask yourself if it's you or the "bitch" that has room to improve here.
[deleted]
I had this for my current job. I was asked to design Facebook and it just ended up being nearly entirely about how to design a notification microservice. Got the job so it evidently worked, but I was just going on in a direction until told to stop.
[deleted]
The interview process is somewhat akin to the “stress” gatekeeping in other industries, where people in charge say “I’ve gone through this, so it makes sense that others would have to also or else we risk getting worse people”. While not really in the same ballpark it feels somewhat like the medical residency system, something that everyone knows is drastically unhealthy but it keeps getting propagated because people are afraid of what happens when you change it.
As someone with clinically diagnosed and treated anxiety the whiteboard interview is an absolute personal hell. I have extremely good performance ratings everywhere I’ve landed a job even in very high stakes actual work environments, but my opinion on the average interview process is it’s extremely discriminatory against people who deal with practically any mental health issue (which, let’s face it is probably quite prevalent in this industry).
As someone with clinically diagnosed and treated anxiety the whiteboard interview is an absolute personal hell. I have extremely good performance ratings everywhere I’ve landed a job even in very high stakes actual work environments, but my opinion on the average interview process is it’s extremely discriminatory against people who deal with practically any mental health issue (which, let’s face it is probably quite prevalent in this industry).
A similar rationale was used to justify hazing and other stupid things when I was in the Marines. "We do this the most utterly painful and stupid way because that was what was done to me as a Private, and now it's my turn."
There is no other field where the candidate has to pass an exam ever time they switch jobs. I suggest getting software engineers licensed by an independent party just once (maybe renew every 10 years.)
The only time I've ever had a panic attack was when I was white boarding some basic stuff in an interview while I was hungover. Obviously being hungover is non-ideal and one wouldn't expect the interview experience to account for that but nonetheless I have always found it interesting that it took that combination to put me over the edge.
I remember leaving as quickly as I could and thinking that I had to find a place to sit down as soon as possible. I made it to Washington Square Park and just sat at a bench for about 30 minutes feeling like I was slowly coming out of a tunnel. Never happened before nor since.
I remember leaving as quickly as I could and thinking that I had to find a place to sit down as soon as possible. I made it to Washington Square Park and just sat at a bench for about 30 minutes feeling like I was slowly coming out of a tunnel. Never happened before nor since.
Absolutely. I've had it happen. I have even undergone tests that shows I experience performance anxiety, and that it actually goes up the easier a question is.
I've had interview instances where my face felt flushed and my mind completely blank on a question I knew how to do but struggled on. And then 30 minutes after the interview finished, the answer is clear as day in my head because it was indeed something I knew and the anxiety was gone.
I feel there is a fair amount of discrimination against those of us who experience anxiety like this, but in a way, I don't care all that much because places that interview certain ways aren't places I want to work anyway.
I've had interview instances where my face felt flushed and my mind completely blank on a question I knew how to do but struggled on. And then 30 minutes after the interview finished, the answer is clear as day in my head because it was indeed something I knew and the anxiety was gone.
I feel there is a fair amount of discrimination against those of us who experience anxiety like this, but in a way, I don't care all that much because places that interview certain ways aren't places I want to work anyway.
This is one of the reasons, I prefer small take home assignments which can be later discussed with the interviewer about the pros and cons of the decisions made during coding. As an interviewer, you get more insight into what the candidate was thinking and whether they were able to actually implement something end to end. Leetcode style problems rarely give you that signal from candidates.
I've done those before, and found the employer always underestimates how much time they should take. I think it's a good idea, but I've done 3 where the employer said "this should take 2-3 hours", but I actually had to spend an entire weekend on it instead. Felt like a waste of time, especially for the ones I didn't even make it to next phase in.
[deleted]
Absolutely, every job states it should take a few hours but it NEVER takes a few hours if you want to deliver something that’s remotely competitive.
My current role’s take home was suppose to take “a few hours”, took the entire weekend. But hey, it worked out.
My current role’s take home was suppose to take “a few hours”, took the entire weekend. But hey, it worked out.
As a hiring manager who was once a developer who was absolutely terrified of the typical whiteboard/algorithm interview, take home assignments are my preferred method to judge tech ability. However, I often feel it's not fair to ask the candidate to invest their personal time in the interview process. So I'm glad to see some affirmation that it's the preferred method for at least some.
When I have done take homes, I've tried to keep the time requirements minimal and the project itself simple and hopefully even a bit fun. It might even be as simple as "here's a REST endpoint, pick the JS SPA framework of your choice and do something interesting with that endpoint." Even just a simple form with clean code that builds will be enough to get you hired if you're otherwise a good candidate. Because in the real world, most of of us are just building CRUD apps.
All that said, I only bother with this if the candidate is incredibly green. For anyone with decent experience I generally assume they're not lying on their resume, and an hour long chat about their projects and technology interests will tell you more than any technical assessment.
When I have done take homes, I've tried to keep the time requirements minimal and the project itself simple and hopefully even a bit fun. It might even be as simple as "here's a REST endpoint, pick the JS SPA framework of your choice and do something interesting with that endpoint." Even just a simple form with clean code that builds will be enough to get you hired if you're otherwise a good candidate. Because in the real world, most of of us are just building CRUD apps.
All that said, I only bother with this if the candidate is incredibly green. For anyone with decent experience I generally assume they're not lying on their resume, and an hour long chat about their projects and technology interests will tell you more than any technical assessment.
[deleted]
We discuss interviewing a lot here. Despite all the proposals things don't really change. FA(A)NG style interviews are just that - whiteboarding solutions to N algorithmic problems in M minutes (or minor variants of this). Either we accept this and move on with solid preparation or modify our priorities such that we're fine with not wanting the money and/or the quality of work (debatable) offered by these companies. At the end of the day, it's a choice and it's still in our hands.
[deleted]
Speaking from personal experience, it absolutely does; both on a mental level (the feeling of being watched and judged, leading to constant second-guessing) and on a physical level, where it invokes the fight-or-flight response. I'm sure the latter in particular was super useful to our ancient ancestors, for whom a shot of adrenaline during stressful situations could be the difference between survival and death. But the physical symptoms - elevated heart rate, dry mouth, tunnel vision, shaking, excess perspiration - are actively harmful to performance in a software engineering interview.
I have friends who, before upcoming situations they know will be stressful - interviews for engineers, performances for musicians, surgery for doctors - have obtained prescriptions for beta-blockers, which mitigate the physical symptoms of the fight-or-flight response for a few hours at a time.
* https://en.wikipedia.org/wiki/Beta_blocker#Anxiety
I have friends who, before upcoming situations they know will be stressful - interviews for engineers, performances for musicians, surgery for doctors - have obtained prescriptions for beta-blockers, which mitigate the physical symptoms of the fight-or-flight response for a few hours at a time.
* https://en.wikipedia.org/wiki/Beta_blocker#Anxiety
We have a rule for technical interviews that we want to "enable each candidate to show the best version of themselves". It seems simple and obvious, but I'm amazed by how happy and excited people are by our interview process. I constantly get emails saying it was such a relief or that they loved it.
It just seems so crazy that we pressure technical candidates the way we do, and expect decent results.
It just seems so crazy that we pressure technical candidates the way we do, and expect decent results.
That sounds great, but how do you achieve that?
Here's an HN comment about I made about it a while ago: https://news.ycombinator.com/item?id=24457170
A kooky theory I harbor is that corporate tech continues to employ algo whiteboard hazing ritual interviews because if they were to evaluate SWEs with tasks more closely resembling the expected day-to-day they would have to pay them an hourly rate under CA labor law or face liability exposure.
We use a 3 step process, sans whiteboard:
1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is. If they express interest by being prepared and asking good questions, we send them
2. a programming task. Choose 1 of 5 tasks. The tasks are not abstract problems or puzzles, but come out of the designs we've implemented. We ask for 100 lines of code and not more than 2 hours. They take as much calendar time in days as they need, most have full time jobs, and let us know when you're done. The candidate then comes back, typically in 2 to 7 days and hosts a code review in front of our team. I give strict instructions before hand: The purpose of the review is to learn how they thought through the problem and how they solved it, not to criticize their style and approach.
3. If we decide we like them to this point, the third interview is with managers from other functions. How well does the candidate communicate, come across to non-developers, express interest in the company and role, etc. It's a check point to look for concerning weaknesses, as well as get buy in from the broader organization.
We like this approach because it allows the candidate to code in a much more natural environment. They can take the time they need and comfortably solve. No one spends their professional career coding on a white board. This approach so far has yielded excellent results. We ask developers how much time they took in the programming task and why they chose the one they solved. The programming task is telling, not in the quality of their code and how long they took, but how much did they get into it? We have had the range from candidates who did not complete it at all and opted out, to candidates who stumped 40 year veterans with elegant code. In every case, we learn how well they can express their thoughts, and importantly, their level of love for the discipline. This is as important to me as any other attribute.
1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is. If they express interest by being prepared and asking good questions, we send them
2. a programming task. Choose 1 of 5 tasks. The tasks are not abstract problems or puzzles, but come out of the designs we've implemented. We ask for 100 lines of code and not more than 2 hours. They take as much calendar time in days as they need, most have full time jobs, and let us know when you're done. The candidate then comes back, typically in 2 to 7 days and hosts a code review in front of our team. I give strict instructions before hand: The purpose of the review is to learn how they thought through the problem and how they solved it, not to criticize their style and approach.
3. If we decide we like them to this point, the third interview is with managers from other functions. How well does the candidate communicate, come across to non-developers, express interest in the company and role, etc. It's a check point to look for concerning weaknesses, as well as get buy in from the broader organization.
We like this approach because it allows the candidate to code in a much more natural environment. They can take the time they need and comfortably solve. No one spends their professional career coding on a white board. This approach so far has yielded excellent results. We ask developers how much time they took in the programming task and why they chose the one they solved. The programming task is telling, not in the quality of their code and how long they took, but how much did they get into it? We have had the range from candidates who did not complete it at all and opted out, to candidates who stumped 40 year veterans with elegant code. In every case, we learn how well they can express their thoughts, and importantly, their level of love for the discipline. This is as important to me as any other attribute.
I love this approach. As someone who generally asks a lot of questions, your first round would give one a good idea if it's worth proceeding to other rounds or not. BTW, are you guys hiring?
Unfortunately I have done take-home assignments and then been ghosted by companies. If I am a candidate I don't know if you are going to ghost me or review my submission. It is tough because I do think this is a better signal than whiteboarding, but there is an asymmetric time commitment.
I can't speak for other companies, but the point at which candidates do the programming task is after they've come in and met 3 or 4 of us. It would be really awkward in my opinion to then not have them back after they've completed it. By that point, we really want to have them in and we hope they do really well. We have 4-5 engineers in the code review for 1-2 hours. That time more than matches the candidate's commitment, which I think is equitable.
I like your approach. Seems to solve the async time commitment problem. Also the candidate doesn't have to do the takehome if it seems like an obvious mismatch when they come in and meet the team.
This rules... I advocated for a similar approach with take home tests at my last position. I used this on candidates and never had a bad hire. Unfortunately this approach was overridden by more senior management in favor of on-site technical tests (programming tasks, IQ and whiteboard problems). The prevailing wisdom was that you couldn't trust people not to cheat on take home programming tasks - a proposition that I find entirely ridiculous and in any case would be quickly mitigated by the on-site code review you use.
We had a long stretch of resumes that were exaggerated, borderline lies, causing a bunch of wasted time, so we had to come up with a pre-screen. The fact that the resumes were basically false meant that trust probably wasn't something to rely on for a pre-screen. So, we came up with an easy 5 minute question, which seemed reasonable.
So far, our 5 minute test has failed spectacularly. Knowing what a good answer is requires too much interpretation of the subtleties of the answer, so it's not very useful for the managers who may not be the greatest at coding.
I think we'll tweak the question a bit, but we're kinda stuck.
How do others handle pre-screening?
So far, our 5 minute test has failed spectacularly. Knowing what a good answer is requires too much interpretation of the subtleties of the answer, so it's not very useful for the managers who may not be the greatest at coding.
I think we'll tweak the question a bit, but we're kinda stuck.
How do others handle pre-screening?
Sorry but no way. You send me a programming task you expect me to do in my personal time, and you’ve just revealed you practice extreme ageism, discriminate against parents or people with elder care responsibilities, and many more negative and disrespectful signs. It shows excessive entitlement on your side, indicating a poor working environment. Out of consideration for finding a healthy, reasonable employer, I would reject you immediately, and so should everyone. It’s not a matter of some candidates liking your process and not others - the process you describe solely perpetuates destructive and exploitative interview behaviors for everyone, even people free of outside responsibilities who can have the luxury to make room for a multi-hour take home exam within a month or two of being assigned.
The median age of my team is 60.
We give candidates all the time they need, as in a week or more. We explain the process when we ask them to interview, and if they prefer a white board interview instead of a take home, we can do that. So far, no one has taken that option.
You're right that some people object to being asked to do programming (though none that we've asked to come in have objected) and some people think they should be compensated, which we don't do. I point out that the code review, for example, takes my team of 4-5 engineers 1-2 hours of their time, so we are matching the candidate's time spent with my team's time.
In any case, it's certainly not for everyone, and we're flexible.
We give candidates all the time they need, as in a week or more. We explain the process when we ask them to interview, and if they prefer a white board interview instead of a take home, we can do that. So far, no one has taken that option.
You're right that some people object to being asked to do programming (though none that we've asked to come in have objected) and some people think they should be compensated, which we don't do. I point out that the code review, for example, takes my team of 4-5 engineers 1-2 hours of their time, so we are matching the candidate's time spent with my team's time.
In any case, it's certainly not for everyone, and we're flexible.
None of this makes it better. “We give candidates all the time they need” (which you suggest is something insanely short notice, like 1-2 weeks) is not an answer. What about people who cannot give you time outside of 1-2 hour blocks within business hours of a weekday, as in they can never work on your assignment apart from that. You’re just openly admitting that you won’t assess that whole cohort of humans. I urge you to reconsider. You are determining a process that you happen to fancy, maybe because it makes you feel like you’re a visionary of interview process or you’re going against the trend, but you’ve got blinders on about your commitment to this and what it means for people who just happen to have obligations.
The standard whiteboard hazing trivia is bad enough, and 4+ hour long marathon on-sites that require weeks of study and taking vacation days just to interview are ridiculous. But your method is not much better. You’re still perpetuating this myth that somehow, unlike virtually every other profession on the planet, software engineering requires intense examination of on-the-spot demonstrated skill, rather than simple conversation probing past experience and domain knowledge.
We just do interviewing completely ass backwards in software. Part to assert gatekeeper superiority, part to judge fealty. It’s just bananas.
The standard whiteboard hazing trivia is bad enough, and 4+ hour long marathon on-sites that require weeks of study and taking vacation days just to interview are ridiculous. But your method is not much better. You’re still perpetuating this myth that somehow, unlike virtually every other profession on the planet, software engineering requires intense examination of on-the-spot demonstrated skill, rather than simple conversation probing past experience and domain knowledge.
We just do interviewing completely ass backwards in software. Part to assert gatekeeper superiority, part to judge fealty. It’s just bananas.
To be honest, if a candidate told me they didn't have time for a take home, I'd arrange to complete the interview process in times-of-day that works for them. The purpose was to avoid 'coding in front of a firing squad', and we could achieve a similar outcome by leaving the room while they work.
BTW, I will consider the approach of simple conversation probing past experience and domain knowledge. I infer that would be in lieu of a coding demonstration. If that has worked for you, any detail that you can fill in would be appreciated.
BTW, I will consider the approach of simple conversation probing past experience and domain knowledge. I infer that would be in lieu of a coding demonstration. If that has worked for you, any detail that you can fill in would be appreciated.
Sounds like a place worth working for
Kind of disappointing that all the top rated comments here are about people's personal opinions about interviewing, and don't engage with the pdf at all. What I found most interesting was Table 1 - it shows that in the unsupervised interview the average score of women was 3/3, while in the supervised version it was less than 1/3. Small sample sizes, but still seems suggestive. (I CTRL-Fed but didn't find much discussion of it in the paper.)
Anyway, other than that interview where I was super stressed and tanked it, all my others have been nothing like it. Just goes to show how stress and a loss of confidence can wreck your chances.