Tech Interviews Are Stupid(gomakethings.com)
gomakethings.com
Tech Interviews Are Stupid
https://gomakethings.com/tech-interviews-are-stupid/
159 comments
Agree with this. In my experience interviewers don't mind if you would normally look something up, but you need to state "hmm I always forget which direction Array.sort sorts based on your return value" and they'll usually say something like "yeah me too, let's try it this way and flip it if needed." If you say nothing and freeze or implement it incorrectly without making it clear that you half expected it, the same situation feels much worse.
In general, I've been seeing some conflation between the "tech interviews are broken" sentiment and the "tech interviews are too hard" sentiment. There's probably some truth to both, and some specific things that both interviewers and candidates can do differently to make more interviews more productive. In all cases, being calm and congenial and trying to put the other person at ease (even if you're the candidate) can work wonders.
In general, I've been seeing some conflation between the "tech interviews are broken" sentiment and the "tech interviews are too hard" sentiment. There's probably some truth to both, and some specific things that both interviewers and candidates can do differently to make more interviews more productive. In all cases, being calm and congenial and trying to put the other person at ease (even if you're the candidate) can work wonders.
Why don't you ask the candidate about problems that they've solved recently though? Why give them a specific problem out of the blue that they might not have any experience with? Wouldn't it be better to ask them about the hardest stuff they've recently done and then ask them about what constraints and tradeoffs they hit and things like that?
I do both. The problem with asking only about experience is - as surprising as it may seem - people are often not prepared for this question. This often means they just talk about a highly specific mundane technical task they're currently working on (for example, I've had answers like "implement redux reducers for my project") and when pressed about challenges, they can't come up with anything and just wing it with some generic canned response (e.g. "the challenge was using APIs from backend team to make the product work" or some other mundane aspect that one typically would expect to be present in every such job). Another variation is that they can't organize their thoughts quickly enough to summarize effectively, so they ramble semi-incoherently for 10 minutes about the specifics of their tech debt.
Mind you, while answers can be highly uninspiring, they often help contextualize where a candidate is in the spectrum. It's easy to spot ticket-closers with this question for example, and conversely you can quickly spot highly opinionated individuals. But like in scientific studies, there's different value in self-reported data vs data you collect from your own observation. There's plenty of "talk-the-talk" people that can articulate prose extremely well, but struggle with the most basic of technical tasks when it comes to actually writing some code. This is surprisingly common with people pivoting from startup jack-of-all-trade roles to highly specialized roles in bigcos.
Mind you, while answers can be highly uninspiring, they often help contextualize where a candidate is in the spectrum. It's easy to spot ticket-closers with this question for example, and conversely you can quickly spot highly opinionated individuals. But like in scientific studies, there's different value in self-reported data vs data you collect from your own observation. There's plenty of "talk-the-talk" people that can articulate prose extremely well, but struggle with the most basic of technical tasks when it comes to actually writing some code. This is surprisingly common with people pivoting from startup jack-of-all-trade roles to highly specialized roles in bigcos.
I don't understand why we collectively treat interview questions as this big gotcha trade secret we have to spring on interviewees in real time. It feels like a masochistic hazing ritual.
I've taken to pre-emailing the general structure of questions I expect to ask ~48 hours before the interview and I've found that a) the interviewees are enormously thankful (which helps with closing candidates) and b) the quality of signal I get is much higher.
I've taken to pre-emailing the general structure of questions I expect to ask ~48 hours before the interview and I've found that a) the interviewees are enormously thankful (which helps with closing candidates) and b) the quality of signal I get is much higher.
I think there are two answers: the bad answer is that people think of interviews as school finals, with all the same connotations about potential for cheating, cramming, memorization-without-retention, etc.
The good answer (IMHO) is that the format allows the interviewer to evaluate how a candidate deals with prioritizing conflicts of interest (i.e. the desire to write clean code vs desire to just get something out due time constraints pressure). It also opens up opportunities to explore refactoring, whereas a primed candidate might just bang out "perfect" code the first time around, and not leave much room for evaluating their iterative process.
The good answer (IMHO) is that the format allows the interviewer to evaluate how a candidate deals with prioritizing conflicts of interest (i.e. the desire to write clean code vs desire to just get something out due time constraints pressure). It also opens up opportunities to explore refactoring, whereas a primed candidate might just bang out "perfect" code the first time around, and not leave much room for evaluating their iterative process.
What exactly do you mean by the general structure of questions? How much details do you send out before the interview?
Definitely both. This is the part of the interview that I get the strongest signal from. If they can still remember a problem, failed and working solution with great detail some time after the situation then they must have understood it at a level more than lookup trial and error fixes. It's great when you can riff off each other thinking about various avenues of discovery, potential causes and fixes.
> It's about how they think, break things down, compose
> the parts into a solution
That's fine too. Often if I'm thinking out loud it's for the benefit of the interviewer so they don't get bored or confused later when I write a bunch of stuff after a long period of silence.
I also tend to do a lot of scribbling of intermediate picture/thoughts that I wipe and redo so it's not a complete void.
Where it can penalize you is if you don't finish and not having talked/shown your work to that point doesn't count for as much.
The best tech interview is if you describe the strategy, algorithm outline and datastructures that you would be using and the interviewer says, "that's good enough, next question".
I also tend to do a lot of scribbling of intermediate picture/thoughts that I wipe and redo so it's not a complete void.
Where it can penalize you is if you don't finish and not having talked/shown your work to that point doesn't count for as much.
The best tech interview is if you describe the strategy, algorithm outline and datastructures that you would be using and the interviewer says, "that's good enough, next question".
Posts like this pop up pretty frequently. The format is usually…
> I hate tech interviews, they’re a poor proxy for real work.
> Here are some ways they’re silly (some examples follow).
> A good interview would not have these bad features, maybe someone should do that.
What I’ve observed in 12 years of practice is —
It works; things go south quickly if you don’t do rigorous technical interviewing.
There is huge variance between interviewers, in both accuracy and bedside manner.
The most effective signal by far (close to 100%) is: someone I already trust technically who has worked with them says they’re good, and is willing to stake their reputation on it. I enthusiastically skip the more tedious parts of interviewing in these cases.
Interesting things I’ve seen, experienced, or practiced:
Long trial contracts to hire, no exceptions. Buys you accuracy at the cost of reduced candidate pool.
Willingness to fire quickly (first 30 days) — in practice similar to above. This is often advocated for but rarely implemented.
De-emphasize interview, heavily game-plan and weigh reference calls; do a lot of them (5+). This works if you have the technique to blow past the boilerplate endorsements.
> I hate tech interviews, they’re a poor proxy for real work.
> Here are some ways they’re silly (some examples follow).
> A good interview would not have these bad features, maybe someone should do that.
What I’ve observed in 12 years of practice is —
It works; things go south quickly if you don’t do rigorous technical interviewing.
There is huge variance between interviewers, in both accuracy and bedside manner.
The most effective signal by far (close to 100%) is: someone I already trust technically who has worked with them says they’re good, and is willing to stake their reputation on it. I enthusiastically skip the more tedious parts of interviewing in these cases.
Interesting things I’ve seen, experienced, or practiced:
Long trial contracts to hire, no exceptions. Buys you accuracy at the cost of reduced candidate pool.
Willingness to fire quickly (first 30 days) — in practice similar to above. This is often advocated for but rarely implemented.
De-emphasize interview, heavily game-plan and weigh reference calls; do a lot of them (5+). This works if you have the technique to blow past the boilerplate endorsements.
> Long trial contracts to hire
I don't understand these. If I'm going to do this (as a candidate) then why not just do independent consulting / contracting and charge a higher rate?
Only once was I offered a contract to hire and I assumed it meant the hiring manager hadn't yet secured the FTE req. When I asked for clarification he just said "that's how we do things here". I explained that a significant portion of their pitch was spent talking about their stellar benefits when they were trying to recruit me and now I wouldn't receive any of those. He said I'd get those in 6 months. I took it as a red flag and passed.
I don't understand these. If I'm going to do this (as a candidate) then why not just do independent consulting / contracting and charge a higher rate?
Only once was I offered a contract to hire and I assumed it meant the hiring manager hadn't yet secured the FTE req. When I asked for clarification he just said "that's how we do things here". I explained that a significant portion of their pitch was spent talking about their stellar benefits when they were trying to recruit me and now I wouldn't receive any of those. He said I'd get those in 6 months. I took it as a red flag and passed.
> The most effective signal by far (close to 100%) is: someone I already trust technically who has worked with them says they’re good, and is willing to stake their reputation on it. I enthusiastically skip the more tedious parts of interviewing in these cases.
This works in small companies where somebody you refer will likely work directly with you and your reputation is at stake. This breaks at around 100 people or so. If you refer somebody to big company, chances are you will never work with them directly and your reputation is not at stake. Why not help out a unqualified friend at a cost to big faceless company?
This works in small companies where somebody you refer will likely work directly with you and your reputation is at stake. This breaks at around 100 people or so. If you refer somebody to big company, chances are you will never work with them directly and your reputation is not at stake. Why not help out a unqualified friend at a cost to big faceless company?
I’ve seen it work in companies at scale, though it’s not universal. Teams are still on the order of 20 or so and the dynamics still apply. Beyond that, you’re right — but even at huge companies a lot of exec hiring is done that way.
I got hired into a big i-bank through a couple of Facebook messages. We ticked all the HR boxes but it was a done deal. Happens all the time
I got hired into a big i-bank through a couple of Facebook messages. We ticked all the HR boxes but it was a done deal. Happens all the time
> a lot of exec hiring is done that way
that's fine because they don't do jack anyway, a good exec just need to not come up with stupid ideas and should let the people who know stuff run everything.
that's fine because they don't do jack anyway, a good exec just need to not come up with stupid ideas and should let the people who know stuff run everything.
Probably very controversial opinion, but so far, my favorite type of interviews have actually been take-home problems.
Of course, it sucks if you've invested time into those, and get ghosted afterwards - but luckily I have yet to experience that.
Some people excel on taking test, some people excel on projects, others in both - I've personally never been the type to perform well on banging out code on whiteboards, but I will perform well on projects.
edit: Let me expand a bit on this
All my take-home problems have been very reasonable. Usually they've been more big-picture / architecture oriented, and not some specific implementation - some coding, some presentation, usually takes 2-6 hours to complete, depending on how much you put into it.
It is of course unfortunate if someone decides to code up something for a whole weekend, like 20-30 hours, only to get ghosted or similar. I guess it comes down to identifying free work from actual interview problems.
Of course, it sucks if you've invested time into those, and get ghosted afterwards - but luckily I have yet to experience that.
Some people excel on taking test, some people excel on projects, others in both - I've personally never been the type to perform well on banging out code on whiteboards, but I will perform well on projects.
edit: Let me expand a bit on this
All my take-home problems have been very reasonable. Usually they've been more big-picture / architecture oriented, and not some specific implementation - some coding, some presentation, usually takes 2-6 hours to complete, depending on how much you put into it.
It is of course unfortunate if someone decides to code up something for a whole weekend, like 20-30 hours, only to get ghosted or similar. I guess it comes down to identifying free work from actual interview problems.
I've only ever had bad experiences.
* Here's a problem that takes 6 hours now do it in two.
* Ghosted
* Bizarre feedback that suggests they mixed me up with a different candidate.
* Emailed while I'm half way through the task to let me know that the position has been filled.
* We aren't prepared to spend 5 minutes looking at your portfolio to see if you can code to a base level but please do our take home problem that takes an hour and a half to prove that you can code to a base level.
* Here's a problem that takes 6 hours now do it in two.
* Ghosted
* Bizarre feedback that suggests they mixed me up with a different candidate.
* Emailed while I'm half way through the task to let me know that the position has been filled.
* We aren't prepared to spend 5 minutes looking at your portfolio to see if you can code to a base level but please do our take home problem that takes an hour and a half to prove that you can code to a base level.
At my company, we usually do the more in-person interview, but if we suspect you have potential and your performance is negatively impacted by the stress of the interview process, we propose a project interview to wrap up the decision.
Since it's more time consuming we don't offer it up front, most people don't like them, but it's a useful tool to have to size up a candidate.
The post-project debrief provides a lot of information on how you think and how you work, the end result is not expected to be perfect, or even complete, as long as the approach is reasonable and we can picture if the person-team fit is there.
Since it's more time consuming we don't offer it up front, most people don't like them, but it's a useful tool to have to size up a candidate.
The post-project debrief provides a lot of information on how you think and how you work, the end result is not expected to be perfect, or even complete, as long as the approach is reasonable and we can picture if the person-team fit is there.
The problem with take home interviews as I see it is that it costs the candidate X time and the company roughly 0 time. This incentives companies to give them to anyone and everyone. So you may be considered a marginal candidate who has little chance of passing unless you prove to be an utter genius in the take home. With in person interviews you'd just get rejected but with a take home they have nothing to lose by giving it to you. So it's a system that can work when few companies do it but utterly implodes at scale.
I've had some take home interview assignments that were paid at my freelancing rate. In my opinion that's win-win; the company has the right incentives and also saves engineering time since interviews are shorter while in the worst case the candidate made a little money on the side.
Not controversial so long as they pay you for your time. This should be standard practice IMO
Personally I don't find making my tax situation more complicated doesn't help with the stress of job applications. I did a work trial last year and it added a whole new layer of anxiety about figuring out how i'm supposed to declare ad-hoc income (which in this case was foreign, making it even more complicated).
If you're used to something like the UK's PAYE system, being paid for interviewing is far from a no-brainer.
If you're used to something like the UK's PAYE system, being paid for interviewing is far from a no-brainer.
Offering a “or we give it to charity” option would have helped I’m guessing.
In this case, given the choice, i'd just have opted to do a few more interviews rather than investing a week in a work trial. I was burning through my savings due to being unemployed for ages, so exchanging a week of my time for a charitable donation wasn't really on the cards -- i've no doubt they would have done that if i'd asked.
Not to mention that I now consider mandatory work trials to be an unethical practice. The stress of an interview, but stretched to last an entire week. A week where you feel you can't make any mistakes. I'll never do another one, it's not worth it.
Not to mention that I now consider mandatory work trials to be an unethical practice. The stress of an interview, but stretched to last an entire week. A week where you feel you can't make any mistakes. I'll never do another one, it's not worth it.
Sorry I misunderstood or misread. For a weeks work you definitely want to be paid!
I’d view a week trial as a knee jerk “skip this opportunity” signal.
I’d view a week trial as a knee jerk “skip this opportunity” signal.
The fundamental problem with take homes is that they produce a huge asymmetry in the cost of interviewing. The company can waste hours of random people's time without having to spend a single minute on them. So while I understand that paying someone for a take home complicates things for some individuals, some kind of contractual agreement that the company will reciprocate some amount of good faith effort is necessary to avoid abuse. Payment is basically a proof of work, and money deposited in my account is a sure way I know of that a company expended some resource on my part. I'd say that even just offering to pay standard contract rates would make me more likely to actually do a take home.
Do they pay you on how well they grade you or do they pay a flat amount?
Either they grade you at 0 and still hire you or I can make some extra cash turning in a bunch of empty applications.
Either they grade you at 0 and still hire you or I can make some extra cash turning in a bunch of empty applications.
Perhaps both parties should pay, i.e. 50/50.
Well it is a single-sided risk reduction strategy. The candidate presumably is convinced that she can fulfill the role regardless of the test.
Then does the candidate pay for the part of the time spent explaining the company and role?
[deleted]
The other big problem with homework is that they are most often used in addition to the leetcode round, not instead of. So double the fun!
>takes 2-6 hours to complete
screw that
screw that
Hold on, how are they “fundamentally broken” if the author is happy with the format with just a slight tweak (allowing use of Google, which many places do already)?
I see that phrase a lot and it’s often deployed without a great deal of thought, which is a good indicator that what I’m about to read is a pretty low-effort article, blog post or tweet.
I see that phrase a lot and it’s often deployed without a great deal of thought, which is a good indicator that what I’m about to read is a pretty low-effort article, blog post or tweet.
I think the author is trying to say that interviews are testing for the wrong thing (ability to solve tricky problems quickly, on the fly, and with no help, which is unrealistic in the workplace) when what they should be testing for is good analytical and research skills, with the assistance of the internet available, just like in real life.
Realism is criminally underrated in interviewing.
Most likely because an interview is not a probationary period where the candidate does actual work. Neither can it simulate the important parts of doing the work. So you have to look for proxies you believe correlate well with being able to do work and test for those instead. Things that can be reasonable measured in 45 minute chunks, thus the proxies you pick will not at all look like someone doing actual work, they will be far removed from that. Hence they will not be "realistic". But that doesn't mean they aren't informative.
So while it's true that having a candidate be able to write code that implements a stack and or whiteboard a simple app is not a realistic scenario of what they will be doing day to day, if they are able to do that well than they are much more likely to be able to do the work than if they are unable to do it well. So you use that as a proxy. Then 5 people interview for an hour, each using a different proxy, and you look at all the information the proxies are telling you to decide whether the candidate should be hired.
So while it's true that having a candidate be able to write code that implements a stack and or whiteboard a simple app is not a realistic scenario of what they will be doing day to day, if they are able to do that well than they are much more likely to be able to do the work than if they are unable to do it well. So you use that as a proxy. Then 5 people interview for an hour, each using a different proxy, and you look at all the information the proxies are telling you to decide whether the candidate should be hired.
>Neither can it simulate the important parts of doing the work.
For sure it can. You can combine problem solving, ability to read code, write tests, avoid common pitfalls, clarify requirements all in the space of 45 mins.
I typically do this by taking a chunk of real code out of the code base, uncoupling it so it can run on its own and writing some stories.
For sure it can. You can combine problem solving, ability to read code, write tests, avoid common pitfalls, clarify requirements all in the space of 45 mins.
I typically do this by taking a chunk of real code out of the code base, uncoupling it so it can run on its own and writing some stories.
Without the background information you are not really simulating what happens. But I also took some examples from our code base, abstracted them a bit to remove dependencies on our libraries, simplified the logic and used them in interviews as a "find the bug" type question. I liked that it because you have to read the code, be able to describe what it is trying to do, and then figure out where it goes wrong. Those types of questions have served me well and I always try to ask one when I am asked to interview.
But that also was not a realistic description of what happens during a regular workday, but it tries to cover one of the (many) skills you do need, so I think it was a decent proxy.
I am not advocating for lots of weird trick questions or brainteasers, but simulating the entire process of design, implementation, test, debug, document, sprint planning and the like. You can't realistically do that in an interview, you pick a few proxies and hope for the best.
But that also was not a realistic description of what happens during a regular workday, but it tries to cover one of the (many) skills you do need, so I think it was a decent proxy.
I am not advocating for lots of weird trick questions or brainteasers, but simulating the entire process of design, implementation, test, debug, document, sprint planning and the like. You can't realistically do that in an interview, you pick a few proxies and hope for the best.
It sounds like you achieved a close enough approximation.
I find interviews like this give very strong (& often surprising) signals.
I don't really object as much to people who try to approximate and don't quite get there than people who decide to just test something that nobody would ever actually do in their job.
It's like the difference between having buggy code and a product that just does completely the wrong thing.
I find interviews like this give very strong (& often surprising) signals.
I don't really object as much to people who try to approximate and don't quite get there than people who decide to just test something that nobody would ever actually do in their job.
It's like the difference between having buggy code and a product that just does completely the wrong thing.
Neither can it simulate the important parts of doing the work.
I disagree with this. With some effort, you can create a small sandbox app that recreates an exact bug or existing implementation that needs a a feature added to it straight from your day to day work.
You can even translate your obscure problem domain into the appropriate analogy.
I disagree with this. With some effort, you can create a small sandbox app that recreates an exact bug or existing implementation that needs a a feature added to it straight from your day to day work.
You can even translate your obscure problem domain into the appropriate analogy.
A small pet peeve of mine is when someone criticizes the interview process without having been on both sides, and assumes that they know what the acceptance criteria is. This article is a good example of that:
> They want to get a sense for what candidates actually know versus what they’ve copy/pasted from the web
This is not quite right. It's right in the sense that if your resume says 3 years of JS experience and you're spending 20% of the time fumbling with unmatched brackets, it says something about you. It's right in the sense that if you say you've done React at two previous jobs but proceed to call setState multiple times in series and then scratch your head when stuff doesn't work, that says something about you. Etc. However, it doesn't mean what a lot of newbies seem to think (that when receiving some leetcode-like task, they're supposed to have the exact ideal answer memorized)
In my interviews, one of the first things I tell candidates is in fact "feel free to google stuff, I'm not here to test whether you've memorized stuff, I'm more interested in how you approach problems".
At least for FAANG/bay area tech, good interviewers looks for signals, and not just purely technical rote memorization: on a basic level, one looks at whether the candidate has working familiarity w/ syntax and APIs that they ought to know, but on deeper levels how they think about code structure, refactoring, testability concerns, design trade-offs, how effectively they communicate (both speaking and listening to feedback), etc. There may also be smaller considerations for legitimate "fit" factor, for example whether a candidate is more startup oriented or more enterprise oriented (this mostly matters when you think about their career aspirations vs what the job is able to provide in terms of challenges).
You can get a pretty decent feel for how "senior" a candidate is, and then acceptance becomes more a question of placing the candidate on a sliding scale based on how demanding of a role one is trying to fill.
The one criticism I have about the interview process is that many interviewers don't tweak their expectations based on level quite as much as they should, especially if they only hire sparsely for low seniority levels. For a junior hire, there should be an expectation that some ramp up time will be required. Expecting upfront solid experience with a laundry list of technologies from them does a service to no one.
> They want to get a sense for what candidates actually know versus what they’ve copy/pasted from the web
This is not quite right. It's right in the sense that if your resume says 3 years of JS experience and you're spending 20% of the time fumbling with unmatched brackets, it says something about you. It's right in the sense that if you say you've done React at two previous jobs but proceed to call setState multiple times in series and then scratch your head when stuff doesn't work, that says something about you. Etc. However, it doesn't mean what a lot of newbies seem to think (that when receiving some leetcode-like task, they're supposed to have the exact ideal answer memorized)
In my interviews, one of the first things I tell candidates is in fact "feel free to google stuff, I'm not here to test whether you've memorized stuff, I'm more interested in how you approach problems".
At least for FAANG/bay area tech, good interviewers looks for signals, and not just purely technical rote memorization: on a basic level, one looks at whether the candidate has working familiarity w/ syntax and APIs that they ought to know, but on deeper levels how they think about code structure, refactoring, testability concerns, design trade-offs, how effectively they communicate (both speaking and listening to feedback), etc. There may also be smaller considerations for legitimate "fit" factor, for example whether a candidate is more startup oriented or more enterprise oriented (this mostly matters when you think about their career aspirations vs what the job is able to provide in terms of challenges).
You can get a pretty decent feel for how "senior" a candidate is, and then acceptance becomes more a question of placing the candidate on a sliding scale based on how demanding of a role one is trying to fill.
The one criticism I have about the interview process is that many interviewers don't tweak their expectations based on level quite as much as they should, especially if they only hire sparsely for low seniority levels. For a junior hire, there should be an expectation that some ramp up time will be required. Expecting upfront solid experience with a laundry list of technologies from them does a service to no one.
This is one of my favourite tweets: https://twitter.com/mxcl/status/608682016205344768
I've seen some rather complex problems being asked in interviews. I've always wondered why my credentials don't seem to matter when it comes to programming skills. I have a masters degree where i did some very tricky algorithms and I even open-sourced it for all to see. But every time I've been looking for a job, i have to rehearse some leetcode problems so that i can put on a good show for 45 minutes.
I get that deep-dives matter a lot and i can respect some of those exercises. These are crucial for senior positions since ideas are more important than code. Then I really get the life story kind of interviews because relating to coworkers and culture fit are hugely important.
It's just the dang leetcode that ticks us all off isn't it?
I've seen some rather complex problems being asked in interviews. I've always wondered why my credentials don't seem to matter when it comes to programming skills. I have a masters degree where i did some very tricky algorithms and I even open-sourced it for all to see. But every time I've been looking for a job, i have to rehearse some leetcode problems so that i can put on a good show for 45 minutes.
I get that deep-dives matter a lot and i can respect some of those exercises. These are crucial for senior positions since ideas are more important than code. Then I really get the life story kind of interviews because relating to coworkers and culture fit are hugely important.
It's just the dang leetcode that ticks us all off isn't it?
People cheat their way through credentials all the time, having a masters degree doesn't tell you anything about a person except they maybe sat in a room on their phone while someone discussed the nuances of Church's Thesis. And most interviewers aren't valued enough by a company to really give a shit who you are, there is just too much happening and too many interviews, so they just can't be asked to look at your Github.
The game and well are poisoned at many levels. There's just too much money on the table at most modern tech companies to trust anything.
The game and well are poisoned at many levels. There's just too much money on the table at most modern tech companies to trust anything.
Bingo.
I do quite a few coding interviews for my company and, frankly, the pass rate on the coding interview is dismal. It's one of our most effective filters for bad candidates.
What sort of torturous questions do I ask? Merge 2 sorted arrays is one of my favorite.
How many people can pass that? less than 50%. A good number of people don't even get to the "concat the 2 arrays into 1 array and sort" sort of solution. It's really that bad. These are people with "15 years of software development experience" failing really basic coding problems.
I really wish I didn't have to ask that sort of question, but at the same time, I don't want to work with anyone that can't solve it (even "under pressure").
The fact is, a lot of programmers will go off the deep end with simple problems. This joke[1] still relatively accurately describes the mentality of a bunch of programmers in the industry.
[1] http://4.bp.blogspot.com/-Hk1mt-RKYLc/UOkxShm6NrI/AAAAAAAACq...
I do quite a few coding interviews for my company and, frankly, the pass rate on the coding interview is dismal. It's one of our most effective filters for bad candidates.
What sort of torturous questions do I ask? Merge 2 sorted arrays is one of my favorite.
How many people can pass that? less than 50%. A good number of people don't even get to the "concat the 2 arrays into 1 array and sort" sort of solution. It's really that bad. These are people with "15 years of software development experience" failing really basic coding problems.
I really wish I didn't have to ask that sort of question, but at the same time, I don't want to work with anyone that can't solve it (even "under pressure").
The fact is, a lot of programmers will go off the deep end with simple problems. This joke[1] still relatively accurately describes the mentality of a bunch of programmers in the industry.
[1] http://4.bp.blogspot.com/-Hk1mt-RKYLc/UOkxShm6NrI/AAAAAAAACq...
I got to wonder, are all those people really incompetent? See, another take is that they all clutch when being interviewed. And maybe not every interview; maybe it matters how the interviewer approaches it.
It's not 'under pressure'. Real pressure is schedules and deployments and compute constraints. Not somebody looking over their shoulder.
It's not 'under pressure'. Real pressure is schedules and deployments and compute constraints. Not somebody looking over their shoulder.
I can definitely write code. I've been doing it quite a while. I've been the "smart guy" to go to with hard problems, at more than one place. Feedback has been very good at most places I've worked. I'm the guy who figures out weird platforms and things like that. I also clean up alright and can talk to people. I'm not bad at the job, and would be, so far as the technical side of things, a good hire for probably 95+% of all software development jobs there are that even somewhat fit in the set of things I've already worked on (which is broad). I'm good under on-the-job pressure (great, even—I kind of like emergencies, in a perverse way. Clears the mind, demands focus on a clear goal.)
I am also 100% certain that I've given interviewers the impression I don't know how to code whatsoever on at least three occasions, including as recently as a couple years ago. Hell, I'll probably do it at least a couple more times before my career's up. I guarantee that the interviewers were chatting after and were like, "see this is why we have to do this, because you get guys like that lying piece-of-shit yahoo, who've somehow had a full career and have great references and can talk about software and processes and architecture and all that, but inexplicably can't code at all".
I'm terribly self-conscious when people watch me code in the context of an interview where they're explicitly judging me so I get very slow even at using tools I'm familiar with ("I think that shortcut does what I want but... what if it doesn't? I'll just do it the slow way, oh no, now I look clumsy and stupid, fuck fuck fuck!"), and I tend to do stuff like forget the syntax to invoke a method in [language X that I've written 500 lines of in the last two weeks] when I start writing with a blank file (no existing syntax to crib off of, and c'mon, can you ask to google shit that basic in an interview? LOL no). Basically I have intense social anxiety but only for specific parts of interviews.
What I can do is talk through architecture, and any amount of conceptually-how-things-fit-together I have in my head, including low-level stuff. I can do 100% of the talking parts of an interview just fine. I'm told I come off exceptionally good in that part, actually (I can't tell, I just go by feedback). I'm sure this just reinforces the impression that I'm some kind of god-tier social manipulator who's, for some unknown reason, decided to scam his way into mid-tier tech jobs rather than applying such outstanding bullshitting skills to something that would pay for them more directly, and probably even be compensated higher if I were that good.
I am also 100% certain that I've given interviewers the impression I don't know how to code whatsoever on at least three occasions, including as recently as a couple years ago. Hell, I'll probably do it at least a couple more times before my career's up. I guarantee that the interviewers were chatting after and were like, "see this is why we have to do this, because you get guys like that lying piece-of-shit yahoo, who've somehow had a full career and have great references and can talk about software and processes and architecture and all that, but inexplicably can't code at all".
I'm terribly self-conscious when people watch me code in the context of an interview where they're explicitly judging me so I get very slow even at using tools I'm familiar with ("I think that shortcut does what I want but... what if it doesn't? I'll just do it the slow way, oh no, now I look clumsy and stupid, fuck fuck fuck!"), and I tend to do stuff like forget the syntax to invoke a method in [language X that I've written 500 lines of in the last two weeks] when I start writing with a blank file (no existing syntax to crib off of, and c'mon, can you ask to google shit that basic in an interview? LOL no). Basically I have intense social anxiety but only for specific parts of interviews.
What I can do is talk through architecture, and any amount of conceptually-how-things-fit-together I have in my head, including low-level stuff. I can do 100% of the talking parts of an interview just fine. I'm told I come off exceptionally good in that part, actually (I can't tell, I just go by feedback). I'm sure this just reinforces the impression that I'm some kind of god-tier social manipulator who's, for some unknown reason, decided to scam his way into mid-tier tech jobs rather than applying such outstanding bullshitting skills to something that would pay for them more directly, and probably even be compensated higher if I were that good.
Ok, so here's my advice for you in that sort of situation.
Just talk. A lot. What are you thinking, why are you thinking it? What are the alternatives you are thinking about. All of these will give your interviewer a good opportunity to give you feedback and can make the problem a lot easier than you might be expecting.
The more you talk, the better.
Just talk. A lot. What are you thinking, why are you thinking it? What are the alternatives you are thinking about. All of these will give your interviewer a good opportunity to give you feedback and can make the problem a lot easier than you might be expecting.
The more you talk, the better.
Yeah, I know, I do try to do that. I just don't really interview often enough to get much better at getting over the anxiety, because despite that I've never really had trouble finding good jobs. I only bring it up to confirm that yes, some of those "fakers" (maybe lots of them) aren't faking, they're just exceptionally shit at a particular kind of interview. Truly, I expect that the set of people who can do a really good job of talking the talk but aren't actually competent at writing code is pretty small, because faking being able to program and talk off-the-cuff about technical topics reflects a fair amount of talent at something else that ought to pay pretty well on its own, in other roles.
I think the way those algo-quiz or live-code-a-thing filters are used tends to catch people who can't talk well about it or do the job (this is the majority of would-be "bad hires", and you'd catch them anyway), some set who are afflicted with extremely bad general social anxiety so who are just fucked at interviews generally (I feel bad for those folks, especially since some of them are probably fine in normal work-social situations); the ones like me who can do the job, and can talk about it fluently, but suck at "coding interviews"; plus some very small set of actual social geniuses who've decided lying their way into so-so tech jobs is what they want to do with their lives, and have the talent (at faking it) to back that up, which are the ones people seem to think are super-common in the wild, for some reason, which seems very implausible to me for the simple reason that, again, that's a skill that's per se valuable elsewhere, if you're actually that good.
(frankly, from what I've seen on the other side of the table, I suspect you could weed out a very high percentage of actual-fakers by just blanket rejecting anyone with any kind of graduate degree in CS, who's applying for a mediocre position rather than something flashier and better-paying, though obviously that would come with its own set of pretty bad false negatives and I wouldn't actually do it, and anyway even those people usually don't "talk the talk" well enough to be convincing, if you get them off-script)
[EDIT] incidentally, FAANG and finance and a few of their peers are the ones where these kinds of aggressive weed-outs do sorta make sense, because their comp is so high that those "social geniuses" I mentioned might actually make more money faking their way into those tech jobs than applying their outstanding non-tech skills more directly to another role, such that it's worth it despite the stress and risk of being "found out".
I think the way those algo-quiz or live-code-a-thing filters are used tends to catch people who can't talk well about it or do the job (this is the majority of would-be "bad hires", and you'd catch them anyway), some set who are afflicted with extremely bad general social anxiety so who are just fucked at interviews generally (I feel bad for those folks, especially since some of them are probably fine in normal work-social situations); the ones like me who can do the job, and can talk about it fluently, but suck at "coding interviews"; plus some very small set of actual social geniuses who've decided lying their way into so-so tech jobs is what they want to do with their lives, and have the talent (at faking it) to back that up, which are the ones people seem to think are super-common in the wild, for some reason, which seems very implausible to me for the simple reason that, again, that's a skill that's per se valuable elsewhere, if you're actually that good.
(frankly, from what I've seen on the other side of the table, I suspect you could weed out a very high percentage of actual-fakers by just blanket rejecting anyone with any kind of graduate degree in CS, who's applying for a mediocre position rather than something flashier and better-paying, though obviously that would come with its own set of pretty bad false negatives and I wouldn't actually do it, and anyway even those people usually don't "talk the talk" well enough to be convincing, if you get them off-script)
[EDIT] incidentally, FAANG and finance and a few of their peers are the ones where these kinds of aggressive weed-outs do sorta make sense, because their comp is so high that those "social geniuses" I mentioned might actually make more money faking their way into those tech jobs than applying their outstanding non-tech skills more directly to another role, such that it's worth it despite the stress and risk of being "found out".
:D Way to peg where I work. (I'm in finance).
I could definitely see ditching the coding questions on a lower paying job or a small company. You'd learn a lot more by bringing someone in for a month long probation (so long as they can talk the talk through the interview).
I could also see doing different interviews if what I did didn't have a lot of algorithm writing requirements. It wouldn't make sense for someone to consider algorithmic efficiency if what they are doing all day long is writing basic CRUD apps.
For us, we've got a lot of specific finance logic and calculations that we need to implement. It's not (always) straight forward "write some crud".
I could definitely see ditching the coding questions on a lower paying job or a small company. You'd learn a lot more by bringing someone in for a month long probation (so long as they can talk the talk through the interview).
I could also see doing different interviews if what I did didn't have a lot of algorithm writing requirements. It wouldn't make sense for someone to consider algorithmic efficiency if what they are doing all day long is writing basic CRUD apps.
For us, we've got a lot of specific finance logic and calculations that we need to implement. It's not (always) straight forward "write some crud".
I'd love a different solution, but this is about the best I've got. I can certainly see that there are some false negatives, but to me, I'd rather take that risk than take the risk of hiring someone that's a true negative. Having worked with some in the past, I really don't want to touch code written by a dev that can't code.
I'm not sure how people couldn't merge two sorted arrays when a competent fifth grader could figure that question out without ever touching a computer.
Some people, even those with 100s of hours of schooling, just never learn how to think computationally. They just "fake it till they make it", which really just means they have no ability to provide value to any company.
Some people, even those with 100s of hours of schooling, just never learn how to think computationally. They just "fake it till they make it", which really just means they have no ability to provide value to any company.
Did you know that there are Go problems that professional players can't solve, but that newbies solve trivially? Becoming a professional go player creates blind spots, basically any board that could never happen in a real game just don't make any sense to someone at that level.
And while, I don't equate merging 2 sorted arrays with being an unsolvable problem for an experienced dev, I have personally experienced similar increasing difficulty to toy problems as I gain increasing real world experience. Common things like timestamps and full names are not as simple as they seem, and so now, I'm less likely to just jump to an answer that would have seemed obvious to me 20 years ago. And it's not like I come up with the solution and discard it, it's more like the previously obvious solution is no longer the first things I think about.
And while, I don't equate merging 2 sorted arrays with being an unsolvable problem for an experienced dev, I have personally experienced similar increasing difficulty to toy problems as I gain increasing real world experience. Common things like timestamps and full names are not as simple as they seem, and so now, I'm less likely to just jump to an answer that would have seemed obvious to me 20 years ago. And it's not like I come up with the solution and discard it, it's more like the previously obvious solution is no longer the first things I think about.
In my 20+ year career as a web developer and software engineer, I have never ONCE had to merge two sorted arrays as part of my job.
If you asked me that question in a whiteboard interview without access to my favorite IDE, a REPL, or google, there is a good possibility that I would fail to impress you.
Never mind the fact that I have been involved with the development of multiple highly successful, multi-million dollar projects where I wrote massive portions of the codebase.
At my current company, they didn't give me a code test or anything like that. They just gave me a small project as contract work (migrating some of their infrastructure to the cloud). I knocked it out, they paid me for the work, and then they hired me full time. I've been here for two and a half years and the CEO once publicly stated that hiring me was one of the best things the previous CTO ever did.
If you asked me that question in a whiteboard interview without access to my favorite IDE, a REPL, or google, there is a good possibility that I would fail to impress you.
Never mind the fact that I have been involved with the development of multiple highly successful, multi-million dollar projects where I wrote massive portions of the codebase.
At my current company, they didn't give me a code test or anything like that. They just gave me a small project as contract work (migrating some of their infrastructure to the cloud). I knocked it out, they paid me for the work, and then they hired me full time. I've been here for two and a half years and the CEO once publicly stated that hiring me was one of the best things the previous CTO ever did.
> In my 20+ year career as a web developer and software engineer, I have never ONCE had to merge two sorted arrays as part of my job.
Cool. It doesn't come up in my job either. That's not the point of the question. The point of the question is "given an algorithm specification, can you implement it" and that problem, particularly at my current work place, comes up very frequently.
> If you asked me that question in a whiteboard interview without access to my favorite IDE, a REPL, or google, there is a good possibility that I would fail to impress you.
All my coding interviews are done via hackerrank which has an OK IDE. I generally also allow googling or questions from the interviewer.
> Never mind the fact that I have been involved with the development of multiple highly successful, multi-million dollar projects where I wrote massive portions of the codebase.
> At my current company, they didn't give me a code test or anything like that. They just gave me a small project as contract work (migrating some of their infrastructure to the cloud). I knocked it out, they paid me for the work, and then they hired me full time. I've been here for two and a half years and the CEO once publicly stated that hiring me was one of the best things the previous CTO ever did.
Congratulations!
Honestly, I think you are letting this sort of question intimidate you too much. It is an easy question to answer and if you sat down for a few minutes I'm sure you could solve it. It's not a trick question. It's not one that's designed to filter someone of your qualifications. It's simply a "can you write an algorithm" question. Something that's really important in finance because a lot of what we deal with (particularly where I'm at, financial reporting) is writing code that matches financial models and algorithms. There's not a library you can reach for that computes everything we do.
Give it a shot in your favorite language and I think you'll be pleasantly surprised at how easily you can write a solution.
Cool. It doesn't come up in my job either. That's not the point of the question. The point of the question is "given an algorithm specification, can you implement it" and that problem, particularly at my current work place, comes up very frequently.
> If you asked me that question in a whiteboard interview without access to my favorite IDE, a REPL, or google, there is a good possibility that I would fail to impress you.
All my coding interviews are done via hackerrank which has an OK IDE. I generally also allow googling or questions from the interviewer.
> Never mind the fact that I have been involved with the development of multiple highly successful, multi-million dollar projects where I wrote massive portions of the codebase.
> At my current company, they didn't give me a code test or anything like that. They just gave me a small project as contract work (migrating some of their infrastructure to the cloud). I knocked it out, they paid me for the work, and then they hired me full time. I've been here for two and a half years and the CEO once publicly stated that hiring me was one of the best things the previous CTO ever did.
Congratulations!
Honestly, I think you are letting this sort of question intimidate you too much. It is an easy question to answer and if you sat down for a few minutes I'm sure you could solve it. It's not a trick question. It's not one that's designed to filter someone of your qualifications. It's simply a "can you write an algorithm" question. Something that's really important in finance because a lot of what we deal with (particularly where I'm at, financial reporting) is writing code that matches financial models and algorithms. There's not a library you can reach for that computes everything we do.
Give it a shot in your favorite language and I think you'll be pleasantly surprised at how easily you can write a solution.
Oh, don't get me wrong. I have done my fair share of "leet" and toy code interview problems over the years, and for a while I even used them when I was interviewing and hiring other devs myself.
But then I had some personal experiences that changed my perspective...
There were a couple of jobs that I applied for where I was perfect for the role. Every single piece of technology they used and listed in the job description was something that I had mastered and used with success at other companies.
I got turned down for those roles, however, because I didn't ace their stupid leet-code bubble sort style questions during their interviews. These were trick problems that had NOTHING to do with the actual day-to-day work I would have been doing.
There were a hundred other things they could have looked at to determine that I was a good fit. They could have asked me to show some of my other work. They could have looked at the 100+ projects in my Github. They could have looked at the 20+ written recommendations on my LinkedIn profile all vouching for me (including glowing recommendations from other developers I worked with over the years, many of whom are now working at companies such as Facebook, Apple, and Google.
But nope. Instead of doing any of that, they would rather rely on some leet-code trick questions to gauge my abilities. As a result, they missed the forest for the trees and they lost out on what could have potentially been a perfect fit and a valuable hire.
So yeah. That's why I don't do that type of interview anymore.
If the job is actually for a role that literally solves algorithm problems on a day-to-day basis, then a code test for algorithm problems makes sense. If not, then please don't use them!
But then I had some personal experiences that changed my perspective...
There were a couple of jobs that I applied for where I was perfect for the role. Every single piece of technology they used and listed in the job description was something that I had mastered and used with success at other companies.
I got turned down for those roles, however, because I didn't ace their stupid leet-code bubble sort style questions during their interviews. These were trick problems that had NOTHING to do with the actual day-to-day work I would have been doing.
There were a hundred other things they could have looked at to determine that I was a good fit. They could have asked me to show some of my other work. They could have looked at the 100+ projects in my Github. They could have looked at the 20+ written recommendations on my LinkedIn profile all vouching for me (including glowing recommendations from other developers I worked with over the years, many of whom are now working at companies such as Facebook, Apple, and Google.
But nope. Instead of doing any of that, they would rather rely on some leet-code trick questions to gauge my abilities. As a result, they missed the forest for the trees and they lost out on what could have potentially been a perfect fit and a valuable hire.
So yeah. That's why I don't do that type of interview anymore.
If the job is actually for a role that literally solves algorithm problems on a day-to-day basis, then a code test for algorithm problems makes sense. If not, then please don't use them!
Notably, he was never asked to invert a binary tree and I don't believe he ever got confirmation about why he was rejected. Given how he responded, I wouldn't be stunned if the answer was not his technical performance.
I have definitely been part of interview processes where the team unanimously rejected experienced candidates who express visceral negative and incredulous reactions to being asked to demonstrate basic programming proficiency. The story here has hints that it may be in this category. Some candidates with decades of experience expect to immediately have philosophical discussion about where software is headed, and offered their choice of management/architect positions without talking about code, and get upset when the interview starts out with some softballs. The irony is they get ejected before the interview gets to the good stuff, we do ask about philosophy and management, but a bad attitude can take that off the table very quickly.
I was also a fan of this tweet until I found out what inverting a binary tree actually is. Being asked to code relatively complicated algorithms like balancing a binary tree is one thing, but this guy essentially failed FizzBuzz. At that point it’s not about testing the limits of your candidate but filtering out the low hanging fruit.
Obviously I don’t have the whole story of the interview, but it makes me a little worried for homebrew.
Obviously I don’t have the whole story of the interview, but it makes me a little worried for homebrew.
No, it's not FizzBuzz. It's a recursive algorithm, which FizzBuzz isn't. (Or doesn't have to be. Let's not cite crazy repos with dozens of solutions deliberately obfuscated.) That automatically takes it out of FizzBuzz range.
If you're a functional programmer, it's probably something you can just bang out. I've done functional programming. But it's been a while. In an interview situation, I'd definitely need some time to struggle through it.
What gives? I claim to be a senior developer of ~25 years experience? Why can't I just bang out the roughly ten-line solution it would be in Haskell, including the requisite data declarations? Well, in my 25 years of experience, yes, I've covered everything that is necessary to do that problem, but I've got to drag it out of storage and rehydrate it manually, because I don't do this sort of thing every day lately. Or ever, really, because what senior dev is running around implementing Computer Science 301 data structures every day anymore? Pretty much just one currently implementing a brand new language that needs those basic structures in its standard library.
I'd get there. It wouldn't even necessarily take long. But I can understand why an interviewer asking this question ten times a week who knows the answer forward and backward might find taking even 5 minutes to be an unimpressive performance for something that such an expert should clearly just be able to scribble down in one shot, right?
And I'm gonna be honest. If you ask me to balance a possibly-unbalanced sorted binary tree, I'm basically going to be re-deriving the answer from scratch. Yeah, I covered it. Probably even did a homework assignment on it in C. But my job for the past 25 years has not been implementing balanced binary trees.
If you're a functional programmer, it's probably something you can just bang out. I've done functional programming. But it's been a while. In an interview situation, I'd definitely need some time to struggle through it.
What gives? I claim to be a senior developer of ~25 years experience? Why can't I just bang out the roughly ten-line solution it would be in Haskell, including the requisite data declarations? Well, in my 25 years of experience, yes, I've covered everything that is necessary to do that problem, but I've got to drag it out of storage and rehydrate it manually, because I don't do this sort of thing every day lately. Or ever, really, because what senior dev is running around implementing Computer Science 301 data structures every day anymore? Pretty much just one currently implementing a brand new language that needs those basic structures in its standard library.
I'd get there. It wouldn't even necessarily take long. But I can understand why an interviewer asking this question ten times a week who knows the answer forward and backward might find taking even 5 minutes to be an unimpressive performance for something that such an expert should clearly just be able to scribble down in one shot, right?
And I'm gonna be honest. If you ask me to balance a possibly-unbalanced sorted binary tree, I'm basically going to be re-deriving the answer from scratch. Yeah, I covered it. Probably even did a homework assignment on it in C. But my job for the past 25 years has not been implementing balanced binary trees.
Inverting a binary tree is essentially trivia in the context of most programming, honestly, if I had to do it without a library I'd be extremely worried I'd fuck it up in some subtle way. Max Howell obviously has a strong understanding of file systems, web protocols, package management, and a whole other suite of skills I'm too stupid to understand.
The fact it was trivia irrelevant to most programmers is precisely why it was such a brain dead question.
Asking what FizzBuzz is is similarly useless and would also make an abysmal question.
Asking what FizzBuzz is is similarly useless and would also make an abysmal question.
"until I found out what inverting a binary tree actually is"
So do you consider youself a terrible, unhire-able developer? Presumable your still has quadrupled since you found out this information?
So do you consider youself a terrible, unhire-able developer? Presumable your still has quadrupled since you found out this information?
No, but I assume that the inteviewer will elaborate if you ask them what they mean. I don't consider people who don't know what FizzBuzz is terrible developers, not knowing how to implement it after an explanation is a red flag though.
We don't know the full story behind that tweet. Inverting a binary tree is not exactly a difficult problem. They could have even spoon fed the author with "Ok, here's the code to traverse a binary tree. This is what it means to invert a subtree. How would you invert the rest of the tree?"
I've had plenty of candidates with great credentials who utterly bombed relatively simple coding problems. Not fizz-buzz but also not that much more difficult.
The tech interview process is different for every company but it's definitely not broken.
As a CTO, one of the things I look for is experience. You can look up all the "basic shit" you want but if we're talking about a specific problem, I want you to be able to give/walk me through a solution even if we don't finish the problem. If I'm interviewing a senior engineer, I want you to be able to show your seniority one way or another, if we talk about architecting something, I want you to be able to talk about why we want to solve it this way, pros-cons etc. Experience also shows me that it wouldn't take you a long time to solve the problem, and in our industry, at least my experience in startups, speed is key.
As a CTO, one of the things I look for is experience. You can look up all the "basic shit" you want but if we're talking about a specific problem, I want you to be able to give/walk me through a solution even if we don't finish the problem. If I'm interviewing a senior engineer, I want you to be able to show your seniority one way or another, if we talk about architecting something, I want you to be able to talk about why we want to solve it this way, pros-cons etc. Experience also shows me that it wouldn't take you a long time to solve the problem, and in our industry, at least my experience in startups, speed is key.
In the macro, there is a ton of demand for engineers yet short supply (particularly for universities). Thus the very high false negative rate observed at nearly every company indicates the interview system is broken.
What would help is more disclosure from hiring managers and interviewers about what they want and the success rates of candidates. That information is still largely today only available to hiring managers and executives. We need less information arbitrage for a more efficient hiring system.
What would help is more disclosure from hiring managers and interviewers about what they want and the success rates of candidates. That information is still largely today only available to hiring managers and executives. We need less information arbitrage for a more efficient hiring system.
More information just makes it more difficult to filter out the scripters trying to automate their job applications. That's the biggest issue I have seen, personally. There's too much noise on the channel, and giving any advantage to the CV spammers is not going to make it less noisy.
> Thus the very high false negative rate observed at nearly every company indicates the interview system is broken.
What high false negative rate? Can you point to some data?
By "false negative", are you referring to rejecting a good candidate? Are false negatives (rejecting a good candidate) more important than false positives (hiring a bad candidate)?
> We need less information arbitrage for a more efficient hiring system.
How do you define efficient? Whose efficiency are you considering, the company or the candidate? Why should efficiency be a primary goal of the hiring system, as opposed to simply finding the best candidates in a reasonable time frame?
What high false negative rate? Can you point to some data?
By "false negative", are you referring to rejecting a good candidate? Are false negatives (rejecting a good candidate) more important than false positives (hiring a bad candidate)?
> We need less information arbitrage for a more efficient hiring system.
How do you define efficient? Whose efficiency are you considering, the company or the candidate? Why should efficiency be a primary goal of the hiring system, as opposed to simply finding the best candidates in a reasonable time frame?
The cost of a false negative to the employer is quite low (missing on a good candidate) compared to the cost of a false positive (hiring a bad one).
Engineers are of course not a uniform thing and every role is looking for different experiences and traits, so you’ll get a lot of good engineers knocked back from interviews who go on to get a job somewhere else. This might be trite but thought it is worth mentioning as what seems “unfair” can often just be this.
In my experience, speed is the opposite of key for longer term success. Speed is how my companies have ended up with designs and implementations that weren't maintainable and/or had big gaps in them.
I'm sure the usefulness of speed varies by context
I'm sure the usefulness of speed varies by context
> most tech role interviews involve some sort of coding challenge
He is talking about coding challenges specifically.
He is talking about coding challenges specifically.
> Experience also shows me that it wouldn't take you a long time to solve the problem, and in our industry, at least my experience in startups, speed is key.
If you feel that way, here's a viewpoint from Derek Sivers which talks about "Idea being a multiple of execution" [0]. If you genuinely believe that "solving a problem within a few minutes" can be reached, I'm going to ask you - What problems are you asking people to solve in a limited time frame?
[0] https://sive.rs/multiply
If you feel that way, here's a viewpoint from Derek Sivers which talks about "Idea being a multiple of execution" [0]. If you genuinely believe that "solving a problem within a few minutes" can be reached, I'm going to ask you - What problems are you asking people to solve in a limited time frame?
[0] https://sive.rs/multiply
I'm curious about the reasoning behind these statements.
For example:
> if we're talking about a specific problem, I want you to be able to give/walk me through a solution even if we don't finish the problem"
Why does this matter? Why do you want this? How do you correlate what you're stating you want here with actual job performance?
For example:
> if we're talking about a specific problem, I want you to be able to give/walk me through a solution even if we don't finish the problem"
Why does this matter? Why do you want this? How do you correlate what you're stating you want here with actual job performance?
> Why does this matter? Why do you want this? How do you correlate what you're stating you want here with actual job performance?
It matters to me because we want to be able to show that we can think through solutions. If we're building a feature that has something to do with streaming video, it would be a plus if you've shown that you have experience building streaming services or at least some knowledge along the topic. This should correlate with job performance because of the experience and working on this topic is already known. Now, there would be a case where we have 0 experience with this problem space but again it comes down to thinking through solutions and thinking about race conditions, side-effects etc and being able to identify these.
It matters to me because we want to be able to show that we can think through solutions. If we're building a feature that has something to do with streaming video, it would be a plus if you've shown that you have experience building streaming services or at least some knowledge along the topic. This should correlate with job performance because of the experience and working on this topic is already known. Now, there would be a case where we have 0 experience with this problem space but again it comes down to thinking through solutions and thinking about race conditions, side-effects etc and being able to identify these.
If the title was a comment, it would be breaking the site guidelines.
More importantly, it's throwing out absolute opinions on why interviews are broken without demonstrating a real problem. There have been hundreds of articles saying hiring/interviews are broken because <logic/opinion>, and almost none that point to data showing that companies are not able to hire developers, nor that developers are statistically unable to get jobs.
Interviews can be difficult and frustrating for both sides, but that doesn't mean they're not working. Despite the fact that the article is correct that looking things up is routine in a dev job, I still may prefer to hire the candidate who doesn't need to during my interview question that was specifically designed to not really need to.
I don't know how often coding challenges disallow using the internet, but the last time I interviewed it was allowed. I've seen lots of companies allow use of any language, and spend your time searching the internet if you want.
More importantly, it's throwing out absolute opinions on why interviews are broken without demonstrating a real problem. There have been hundreds of articles saying hiring/interviews are broken because <logic/opinion>, and almost none that point to data showing that companies are not able to hire developers, nor that developers are statistically unable to get jobs.
Interviews can be difficult and frustrating for both sides, but that doesn't mean they're not working. Despite the fact that the article is correct that looking things up is routine in a dev job, I still may prefer to hire the candidate who doesn't need to during my interview question that was specifically designed to not really need to.
I don't know how often coding challenges disallow using the internet, but the last time I interviewed it was allowed. I've seen lots of companies allow use of any language, and spend your time searching the internet if you want.
We literally have thousands of research studies demonstrating that place of birth, income, gender, and even names bias hiring.
If it was this prilliant data driven meritocratic nirvana of scientific precision, we would never have issues with discrimination again minorities, the poor and people who looks funny and have tatoos.
The burden of proof is on folks that keep pushing these silly tasks, where is the data "we've added some stupid questions to our interviews, and now we are doing twice better"?
If it was this prilliant data driven meritocratic nirvana of scientific precision, we would never have issues with discrimination again minorities, the poor and people who looks funny and have tatoos.
The burden of proof is on folks that keep pushing these silly tasks, where is the data "we've added some stupid questions to our interviews, and now we are doing twice better"?
Oh I'd agree there's cultural bias issues, but that is a complete straw man in this context. The article wasn't about cultural bias, nor was my reply.
Meritocracy, incidentally, is a term that was invented to demonstrate bias, not a signifier of the ideal system. https://en.wikipedia.org/wiki/Meritocracy#Early_definitions
> The burden of proof is on folks that keep pushing these silly tasks
This strikes me as a somewhat bizarre and myopic way of thinking about jobs. There is no burden of proof at all, because there is no claim being made. The deal here is someone will pay you money, if you agree to do the work they ask. This opportunity is available to people who make it through the interviews in order to demonstrate they meet the qualifications for the job. Companies that are hiring are the sole decider of what these qualifications are, and can choose to have them demonstrated in any way they choose (up to what we agree are legal limits.) If you don't want money, then don't submit to "silly tasks". (I've never witnessed silly tasks in my interviewing, ever. I've had questions for which I don't have experience, and questions that are too hard, but the questions have always been pretty focused on figuring out what I know and how good of a programmer I am, and by extension how much they are going to pay me...)
Meritocracy, incidentally, is a term that was invented to demonstrate bias, not a signifier of the ideal system. https://en.wikipedia.org/wiki/Meritocracy#Early_definitions
> The burden of proof is on folks that keep pushing these silly tasks
This strikes me as a somewhat bizarre and myopic way of thinking about jobs. There is no burden of proof at all, because there is no claim being made. The deal here is someone will pay you money, if you agree to do the work they ask. This opportunity is available to people who make it through the interviews in order to demonstrate they meet the qualifications for the job. Companies that are hiring are the sole decider of what these qualifications are, and can choose to have them demonstrated in any way they choose (up to what we agree are legal limits.) If you don't want money, then don't submit to "silly tasks". (I've never witnessed silly tasks in my interviewing, ever. I've had questions for which I don't have experience, and questions that are too hard, but the questions have always been pretty focused on figuring out what I know and how good of a programmer I am, and by extension how much they are going to pay me...)
Your reply is written as if a system that is trustworthy is being accused of being untrustworthy.
Instead, a system that we know runs poorly in 10 ways is claimed to be run poorly in 11 ways.
That is a dramatic difference and this would be a valid argument in a court of law - if the system was never run properly, why do you expect it runs properly when it comes to this particular matter? Why would anyone put their faith in it?
Instead, a system that we know runs poorly in 10 ways is claimed to be run poorly in 11 ways.
That is a dramatic difference and this would be a valid argument in a court of law - if the system was never run properly, why do you expect it runs properly when it comes to this particular matter? Why would anyone put their faith in it?
> That is a dramatical difference and this would be a valid argument in a court of law
What are you talking about?
> Why would anyone put their faith in it?
They need money, maybe?
Because software developers are, by and large, employed (not to mention paid well)?
Because the economy is functioning?
I don't understand what you're suggesting, what is the alternative to interviewing? What do you want fixed? Why are you coming down hard on me for suggesting that not being allowed to Google the answer to your interview question isn't a serious problem that needs solving? How would you hire people if you had a company?
What does trust have to do with interviewing? Does one need to have trust in order to get a job? Do you have a job? Aside from cultural bias, what are examples of the system breaking trust in large numbers that demonstrate that hiring is "broken"? Where is the data that shows this?
Since you want to talk about trust, why should a company trust your or your claim of being qualified for a job without asking you to demonstrate you're qualified, in a time-bound and resource-bound setting? The interview questions you called "silly tasks" are nothing more than a request to demonstrate a very small amount of your claimed education and ability. Is that unreasonable? Why or why not?
What are you talking about?
> Why would anyone put their faith in it?
They need money, maybe?
Because software developers are, by and large, employed (not to mention paid well)?
Because the economy is functioning?
I don't understand what you're suggesting, what is the alternative to interviewing? What do you want fixed? Why are you coming down hard on me for suggesting that not being allowed to Google the answer to your interview question isn't a serious problem that needs solving? How would you hire people if you had a company?
What does trust have to do with interviewing? Does one need to have trust in order to get a job? Do you have a job? Aside from cultural bias, what are examples of the system breaking trust in large numbers that demonstrate that hiring is "broken"? Where is the data that shows this?
Since you want to talk about trust, why should a company trust your or your claim of being qualified for a job without asking you to demonstrate you're qualified, in a time-bound and resource-bound setting? The interview questions you called "silly tasks" are nothing more than a request to demonstrate a very small amount of your claimed education and ability. Is that unreasonable? Why or why not?
"What are you talking about?"
The following statement is not very convincing: "maybe hiring is broken up when it comes to culture, gender, race, personal connections and class, but when it comes to programming tests, we certainly have it right"
Mistakes in other areas indicate mistakes are very likely in this area. Parallel with courts was meant to illustrate that
"They need money, maybe?"
If thsts the only reason, they won't put their faith in it, they will game the system to the maximum possible degree.
I think what we are doing in this industry is idiotic, we do not ask surgeons to do a 30-minute surgery on a dog to test their skills in surgery on people, we do not ask structural engineers to make a bridge out of matchsticks to test their skill.
Maybe we need qualifications that can actually be trusted, thats one possible solution.
The following statement is not very convincing: "maybe hiring is broken up when it comes to culture, gender, race, personal connections and class, but when it comes to programming tests, we certainly have it right"
Mistakes in other areas indicate mistakes are very likely in this area. Parallel with courts was meant to illustrate that
"They need money, maybe?"
If thsts the only reason, they won't put their faith in it, they will game the system to the maximum possible degree.
I think what we are doing in this industry is idiotic, we do not ask surgeons to do a 30-minute surgery on a dog to test their skills in surgery on people, we do not ask structural engineers to make a bridge out of matchsticks to test their skill.
Maybe we need qualifications that can actually be trusted, thats one possible solution.
> Then they won't put their faith in it, they will game the system to the maximum possoble degree.
Hence the request to demonstrate basic proficiency?
You seem to be inventing reasons to criticize and argue with me about subjects that neither I nor the article I responded to brought up. I didn't claim that programming tests are "right", so this is like your 4th straw man in a row. You don't seem to be answering any questions about how to improve the situation. Aside from bias, you haven't yet demonstrated a problem with hiring from the company side or the candidate side that has data to back it up.
What would you do to improve your perceived imperfections in the interview and hiring process?
> I think what we are doing in this industry is idiotic, we do not ask surgeons to do a 30-minute surgery on a dog to test their skills in surgery on people, we do not ask structural engineers to make a bridge out of matchsticks to test their skill.
This comment makes it seem like you are ignorant of and not even remotely aware of what the qualifications and interviews are like in other fields, and how much worse and more difficult they are than software. Surgeons go through years of low paid residency, and accreditation, and they suffer a whole battery of knowledge based tests where they're not allowed to Google the answer. Same for structural engineering. The fields of surgery and structural engineering both have cultural bias problems, just like software.
Hence the request to demonstrate basic proficiency?
You seem to be inventing reasons to criticize and argue with me about subjects that neither I nor the article I responded to brought up. I didn't claim that programming tests are "right", so this is like your 4th straw man in a row. You don't seem to be answering any questions about how to improve the situation. Aside from bias, you haven't yet demonstrated a problem with hiring from the company side or the candidate side that has data to back it up.
What would you do to improve your perceived imperfections in the interview and hiring process?
> I think what we are doing in this industry is idiotic, we do not ask surgeons to do a 30-minute surgery on a dog to test their skills in surgery on people, we do not ask structural engineers to make a bridge out of matchsticks to test their skill.
This comment makes it seem like you are ignorant of and not even remotely aware of what the qualifications and interviews are like in other fields, and how much worse and more difficult they are than software. Surgeons go through years of low paid residency, and accreditation, and they suffer a whole battery of knowledge based tests where they're not allowed to Google the answer. Same for structural engineering. The fields of surgery and structural engineering both have cultural bias problems, just like software.
I do have data: I and a friend of mine have refused to do coding tests. It was really funny because frequently hiring folks start acting as if I insulted him. And we could still find a job, and it's not overrun by idiots or incompetents. The standards are not 'lower' and the company does not perform worse or pay worse.
So why can we hire a doctor without asking him to transplant a dog's kidgey during an interview, but not a developer?
So why can we hire a doctor without asking him to transplant a dog's kidgey during an interview, but not a developer?
> So why can we hire a doctor without asking him to transplant a dog's kidgey during an interview, but not a developer?
This is false. Your assertion that doctors don't perform in order to demonstrate their qualifications is wrong. They do have to perform multiple dissections and surgeries before they can get hired anywhere.
> I do have data: I and a friend...
That's not data, that's personal experience.
Your "data" is proving my point that coding tests aren't causing a hiring problem.
Last chance, third time I'm asking: what would you do to fix it?
This is false. Your assertion that doctors don't perform in order to demonstrate their qualifications is wrong. They do have to perform multiple dissections and surgeries before they can get hired anywhere.
> I do have data: I and a friend...
That's not data, that's personal experience.
Your "data" is proving my point that coding tests aren't causing a hiring problem.
Last chance, third time I'm asking: what would you do to fix it?
Firstly, this is extremely condecending.
But Third time I am answering: have trustworthy qualifications.
But you ignore most of what i write, and provide no evidence to support your position that this charade of hiring actually achieves what you think it does, so at this point I no longer believe you are debating in good faith
But Third time I am answering: have trustworthy qualifications.
But you ignore most of what i write, and provide no evidence to support your position that this charade of hiring actually achieves what you think it does, so at this point I no longer believe you are debating in good faith
You didn't start this argument in good faith, you started with a straw man and then added several more. You have also misquoted me and ignored most of what I wrote.
> have trustworthy qualifications.
This doesn't answer the question. People already have trustworthy qualifications, so this solves nothing. Your own point proves your answer wrong: people will game the system. You're contradicting yourself.
Doctors and structural engineers achieve trustworthy qualifications by doing a 4 year residency / apprenticeship program after their degree. Is that what you want in software? You would prefer years of supervised and low paid work rather than spend a total of a few days of your life doing easy coding tests?
> have trustworthy qualifications.
This doesn't answer the question. People already have trustworthy qualifications, so this solves nothing. Your own point proves your answer wrong: people will game the system. You're contradicting yourself.
Doctors and structural engineers achieve trustworthy qualifications by doing a 4 year residency / apprenticeship program after their degree. Is that what you want in software? You would prefer years of supervised and low paid work rather than spend a total of a few days of your life doing easy coding tests?
So trustworthy qualifications solve nothing and we already have them, except we don't and they do solve the issue for structural engineers and doctors as you pointed out exactly they are more trustworthy than anything we have in software.
For the fourth time, yes, thats exactly what I want. Having two years of supervised work would dramatically improve this industry
But i am the one arguing in bad faith.
For the fourth time, yes, thats exactly what I want. Having two years of supervised work would dramatically improve this industry
But i am the one arguing in bad faith.
[deleted]
> I understand why interviewers do this. They want to get a sense for what candidates actually know versus what they’ve copy/pasted from the web.
Bad assumption. If you really did understand why , you wouldn’t say it’s stupid (unless for clickbait).
Bad assumption. If you really did understand why , you wouldn’t say it’s stupid (unless for clickbait).
I think that they want to hear is thinking. So every time I am during technical interview, I just talk out loud what I see in front of my eyes. That is great because they get to know my thinking process and it works as a rubber ducky for myself.
This is something a great interviewee does. I don't try and trick interviewees or make things unnecessarily hard. I really just want to see "can you think like a programmer". When an interviewee gets stuck and isn't saying what they are thinking about, it makes it really hard for me as an interviewer to give good prompts.
Obviously, I can't give away a solution, but at the same time some people will get stuck on things like "Can I use X" or "can I use Y". And for me, that answer is almost always "Yes, absolutely use X!". Because that's what I WANT from someone I'm working with. I don't want you to write a sort from scratch, I want you to call `Collections.sort` or even have the wherewithal to ask someone/google "Hey, I know Java has some inbuilt sorts, how do I invoke those?".
The only time I'll maybe mark you lower is if you start the interview by saying "Yes, java is my favorite language to program in and I use it all the time!" and then struggle to do `new ArrayList<>()` (which has happened on multiple interviews). I always try to get someone to use whatever language they are most comfortable with. Because, to me, languages are stupidly easy to pick up and which one you are most familiar matters far less than how you use it.
Obviously, I can't give away a solution, but at the same time some people will get stuck on things like "Can I use X" or "can I use Y". And for me, that answer is almost always "Yes, absolutely use X!". Because that's what I WANT from someone I'm working with. I don't want you to write a sort from scratch, I want you to call `Collections.sort` or even have the wherewithal to ask someone/google "Hey, I know Java has some inbuilt sorts, how do I invoke those?".
The only time I'll maybe mark you lower is if you start the interview by saying "Yes, java is my favorite language to program in and I use it all the time!" and then struggle to do `new ArrayList<>()` (which has happened on multiple interviews). I always try to get someone to use whatever language they are most comfortable with. Because, to me, languages are stupidly easy to pick up and which one you are most familiar matters far less than how you use it.
By that logic, no societal system can ever be broken.
One thing I rarely see talked about is that the people doing tech interviews just plain suck at interviewing. Everyone assumes intent; that interviews go a certain way because someone has decided they should go that way. While that is quite possibly true at FANG, in my experience it's not the case elsewhere.
Interviews at most companies amount to "hey, who wants to/can interview someone next week?" Some people volunteer, and then they wing it. They might do a FANG-style interview because that's what they think they're supposed to do. They might parrot whatever the interview process was when they were hired. They usually just wing it, making up questions as they go.
It's not normally intentional or nefarious. Good engineers don't often make good interviewers. Sometimes they don't want to be doing the interview, and are simply trying to bumble through it. Usually, they have the best intentions, but they don't know what they're doing. No one's prepared them; they haven't prepared themselves (and likely don't know how to). Everyone usually has good intentions. They want to give the candidate a fair shake; they don't want to make the wrong decision; they want to find a good coworker; they want to do their employer a solid by finding the best employee.
I know I suck at interviewing, despite doing it dozens of times. I've had a few regrets about people I've hired, but not many. Whatever failures my process has, I don't think they're too egregious. I've been on my own to figure out what works and what doesn't.
Interviews at most companies amount to "hey, who wants to/can interview someone next week?" Some people volunteer, and then they wing it. They might do a FANG-style interview because that's what they think they're supposed to do. They might parrot whatever the interview process was when they were hired. They usually just wing it, making up questions as they go.
It's not normally intentional or nefarious. Good engineers don't often make good interviewers. Sometimes they don't want to be doing the interview, and are simply trying to bumble through it. Usually, they have the best intentions, but they don't know what they're doing. No one's prepared them; they haven't prepared themselves (and likely don't know how to). Everyone usually has good intentions. They want to give the candidate a fair shake; they don't want to make the wrong decision; they want to find a good coworker; they want to do their employer a solid by finding the best employee.
I know I suck at interviewing, despite doing it dozens of times. I've had a few regrets about people I've hired, but not many. Whatever failures my process has, I don't think they're too egregious. I've been on my own to figure out what works and what doesn't.
I worked in a company which was hiring tech folks without coding interviews.
I am glad I left.
I am glad I left.
I think part of the problem is that places with coding interviews usually pay better than those without, and the ones with the really painful, idiotic, grueling interviews that everyone hates offer the top pay, so the system ends up sorting all the worst candidates to the companies that don't do "normal" coding interviews.
In fact, I think that effect is exactly why the top-paying places do it, because it makes their hiring easier. At this point, if any of them unilaterally drops the hellish, silly, hazing-interview process, they'll be inundated with good candidates they've been excluding plus a shitload of bad ones who'd already been filtered out by the fact that their process is hellish and silly. So none of them will.
In fact, I think that effect is exactly why the top-paying places do it, because it makes their hiring easier. At this point, if any of them unilaterally drops the hellish, silly, hazing-interview process, they'll be inundated with good candidates they've been excluding plus a shitload of bad ones who'd already been filtered out by the fact that their process is hellish and silly. So none of them will.
Someone wrote a really great article for how you should play this as a company if you can't pay the best.
Unfortunately, don't have the link.
The premise is: you should be aware that there's a large subset of good engineers who aren't particularly great at coding interviews - and you should set up your interview process to select for those candidates.
Unfortunately, don't have the link.
The premise is: you should be aware that there's a large subset of good engineers who aren't particularly great at coding interviews - and you should set up your interview process to select for those candidates.
Hahaha, like Moneyball for programmers. I like it.
The company didn't use the three or six month probation period?
In my experience, getting a bad hire is worse than not hiring at all. It takes a huge amount of resources to properly onboard a new team member, and then if they are not performing well the work they were doing ends up becoming a liability instead of an asset to the team.
Yes, sure but if it's a bad hire. You would dismiss within the probation period, right?
I also worked at two companies like this. Both rank as #1 and #2 best places to work. A bunch of very competent people.
Just ask them to articulate technical concepts, so you can know whether they are full of sh*r not.
People who can hold targeted low-level technical conversations are generally well equipped to do the work you are expecting him/her to do.
What's wrong with this approach?
People who can hold targeted low-level technical conversations are generally well equipped to do the work you are expecting him/her to do.
What's wrong with this approach?
That's my preferred method as well. Talk about the stuff, throw in words they should have heard of. See if they run out of intelligent things to say.
A good question for ppl in the coding challenge camp.
What new information do you neeed from a coding challenge that you don't get talking shop with the individual?
I don't think low level technical stuff is something you can BS your way through an interview. If it works on you, then you as an interviewer has major gaps
What new information do you neeed from a coding challenge that you don't get talking shop with the individual?
I don't think low level technical stuff is something you can BS your way through an interview. If it works on you, then you as an interviewer has major gaps
I'm lead dev on a rather large frontend project, and I still feel like I'd struggle in a frontend-focused interview today. Underappreciated by many I think is the ability to solve problems. I solve them (almost) every day, but sometimes it's easy, and sometimes it's really tough. But eventually, if you know how to ask the right questions, you'll get to a workable answer. That kind of real-world experience doesn't seem to surface well in a typical interviewer/interviewee type setting.
And once you are a lead you are doing less of the stuff day to day that would make you shine in a coding interview. Especially the ones where “we need a c# engineer so let’s dust off those weird exception handling edge cases again and test on it” when in real life you look it up or run a unit test over it (better!)
I see posts like this all the time on HN and I really wonder if there's an alternate world of really stupid interview practices out there that I've just been lucky enough to avoid.
In this case: I've never heard of, participated in or administered a programming interview where the candidate couldn't Google things.
I remember when I interviewed at Braintree, I spent a good chunk of the pairing interview searching for Linux command syntax because I was new to using it. I got the job and spent 7 years there.
In this case: I've never heard of, participated in or administered a programming interview where the candidate couldn't Google things.
I remember when I interviewed at Braintree, I spent a good chunk of the pairing interview searching for Linux command syntax because I was new to using it. I got the job and spent 7 years there.
When I interviewed for Facebook, the interviewer berated me for the coding pad losing focus with an alt-tab for a second ("What are you doing?").
I wasn't aware they were tracking this, of course, nor that it wouldn't be kosher, and I specifically asked about whether I could look up Java API at the start and did not get a "no" ("We don't necessarily require working code here" is pretty ambiguous).
I passed the technical round, and then they ghosted me after asking what my technical level at Google was (which I didn't have to tell, as it's hidden even within Google).
I wasn't aware they were tracking this, of course, nor that it wouldn't be kosher, and I specifically asked about whether I could look up Java API at the start and did not get a "no" ("We don't necessarily require working code here" is pretty ambiguous).
I passed the technical round, and then they ghosted me after asking what my technical level at Google was (which I didn't have to tell, as it's hidden even within Google).
Are you allowed to Google things in a Google interview? I don’t think I was allowed to!
How would that even work? You’d just Google for the answer to the brain teaser and find it because there’s only so many brain teasers out there.
How would that even work? You’d just Google for the answer to the brain teaser and find it because there’s only so many brain teasers out there.
I'm referring to a programming interview, where you are writing code with the candidate -- not a brain teaser.
I'm not sure there's much of a clear line between the two. Google asked me to solve what was really a brain teaser, but solve it using code. Don't think I was allowed to Google anything.
That's fair, in that situation there's probably a valid case for disallowing use of a search engine. Whether or not that's an effective interview is another story :)
I failed an interview once because I knew pacman but they were looking for apt. Big green GPU company.
Were you the only candidate? I mean maybe they interviewed a few people, who already knew apt. So it’s a choice between hiring someone who already knows the tech stack they use, and someone that doesn’t.
To take this even further, it is possible that they had this same situation before, went with the one that didn't know the stack, and regretted it terribly.
Once bitten twice shy and all that.
Once bitten twice shy and all that.
Yeah I guess I was just thinking that this kind of thing seemed totally irrelevant to the job description which was for a CUDA application engineer, but maybe it was a lot more important for their people to be facile with ubuntu cli in front of customers.
As others have mentioned, this is more a sign of bad interviewers than a bad format. Of the companies I've worked at, at least one has allowed googling during the interview, and most de-emphasize all the stuff you have to look up in order to focus on the problem-break-down part of the question.
I think there's too much imitation without understanding in interviewing. Candidates hear questions, and then go on to repeat them when it's their turn to be interviewers, without really understanding what's behind the question. These new interviewers then approach the interview as if they were administering a coding test, whereas what they ought to be doing is facilitating a conversation using the question as a starting prompt.
The other problem that I see sometimes with interviewing is interviewers asking obscure questions. Really good candidates aren't solving the problem on the spot - they've come across problems like the one the interviewer asks, and are using approaches they've already validated in their life experience. When an interviewer asks an obscure question, they are not helping to facilitate a conversation as to whether the candidate can handle the typical things that would come his or her way.
I think there's too much imitation without understanding in interviewing. Candidates hear questions, and then go on to repeat them when it's their turn to be interviewers, without really understanding what's behind the question. These new interviewers then approach the interview as if they were administering a coding test, whereas what they ought to be doing is facilitating a conversation using the question as a starting prompt.
The other problem that I see sometimes with interviewing is interviewers asking obscure questions. Really good candidates aren't solving the problem on the spot - they've come across problems like the one the interviewer asks, and are using approaches they've already validated in their life experience. When an interviewer asks an obscure question, they are not helping to facilitate a conversation as to whether the candidate can handle the typical things that would come his or her way.
I find that pair programming with the candidate on their laptop, using their preferred tools works well.
It takes time and in many ways it requires more energy from me than if I were to just sit and observe. However, it gives a better feel for how the person goes about solving problems from the abstract, architectural stuff to the nitty gritty. I love it when pairing with a fellow vim-user as I will inevitably learn some new trick.
It takes time and in many ways it requires more energy from me than if I were to just sit and observe. However, it gives a better feel for how the person goes about solving problems from the abstract, architectural stuff to the nitty gritty. I love it when pairing with a fellow vim-user as I will inevitably learn some new trick.
Totally! On their laptop being the key. I interviewed at Pivotal around ~6 years ago and the person had some customized VIM setup which was a PITA to work with. He had a very good workflow with it and I honestly looked dumb trying to achieve the same things as him in his computer...
Yeah, or when they assume you have an English keyboard...
After convincing the boss to fire a guy who laughed at me using vim saying it was really old and then proceeded to push broken code to production at 4am, trying to blame it on me when the logs were really not in his favour, I ended up in the position of doing the hiring for a guy to replace this yahoo. I live in a country where people's resumes fit in a damn manila envelope because there's just SO MUCH emphasis on degrees and stuff like that, and while the education here is pretty good (A lot of focus on hardware in the compsci curriculum here actually), really all I wanted after rejecting a bunch of prospects for technobabbling bullshit at me with their huge, multi page resumes i barely read was someone to tell me humbly enough that he'd just have to google the solution to the really damn easy fizzbuzz i was presenting them. You know how hard it was to find that in a braindrain situation? My boss thought I was being too mean, but really I just can't work with dishonest people. It's the most important thing to me that I actually like my coworkers for being good people. A coding interview of the MOST BASIC kind can act as a sort of filter for this.
Like sheesh, I'm not looking for this:
https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
All I want is to know if you know how to work with a modulo operators or not in an honest way, if you know basic third grade level integer division with remainders. If you give me bullshit instead of that, chances are you're not going to get hired. A probationary period for a couple of weeks after on the job after that to see how good you actually are, paid, with the knowledge that if you're just a terrible programmer you don't make it past that.
Like sheesh, I'm not looking for this:
https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
All I want is to know if you know how to work with a modulo operators or not in an honest way, if you know basic third grade level integer division with remainders. If you give me bullshit instead of that, chances are you're not going to get hired. A probationary period for a couple of weeks after on the job after that to see how good you actually are, paid, with the knowledge that if you're just a terrible programmer you don't make it past that.
One of the companies I interviewed with did what he described and it was nice. I thought the cool thing was that in the second interview they wanted me to walk them through some of my thought process for why I did/searched for certain things.
I didn't end up taking the job, but boy was it a breath of fresh air compared to some of the other interviews I've done.
I didn't end up taking the job, but boy was it a breath of fresh air compared to some of the other interviews I've done.
Point of the interview should be to asses knowledge and thinking process, and many other things. Definitely, it is not easy, as in one hour someone needs to assess that person is not faking his entire working history, and that has acceptable skills for the certain role.
One of the good interviews I had was with few people DB/Architecture/Testing/Coding at the end they had combined overview. Usually when I am interviewing I would allow everything, as I am more interested in thinking process of attempting to solve something than actually giving me some clever solution in extra limited time.
For me it is interesting that many companies have decoupled skills vs. money paid for those skills expectation, and often they would like to have star workaholic developers but only if they could pay them as McDonald employees. What I am trying to say, don't have an interview like you trying to employee in Google if you are are not prepared to pay those people like the Google.
One of the good interviews I had was with few people DB/Architecture/Testing/Coding at the end they had combined overview. Usually when I am interviewing I would allow everything, as I am more interested in thinking process of attempting to solve something than actually giving me some clever solution in extra limited time.
For me it is interesting that many companies have decoupled skills vs. money paid for those skills expectation, and often they would like to have star workaholic developers but only if they could pay them as McDonald employees. What I am trying to say, don't have an interview like you trying to employee in Google if you are are not prepared to pay those people like the Google.
Looking up things isn't solving problems. Sometimes you need to look up things to solve problems but they are fundamentally unrelated. Coding interviews generally require 0 background (other than base CS principles) so you shouldn't need to look up anything. I've never had someone ask me for correct syntax in an interview setting.
As a job searcher, I want to be able to spend as little time as possible on the interview process for one company. I don't mind researching the company or talking about their specific process, but I want to minimize the amount of time I have to take to prove my competence to them, because that's something that should transfer well between the different roles I'm pursuing.
I understand that not everyone has an OSS portfolio, but it's better than having 5 different companies give me a unique take-home project, and then having another 3 companies give me an algorithms assessment. If everything was "let's take a look at one repo you've spent <10 hours on" or "let's do leetcode problems" that would be preferable to the current state of affairs, because as it stands I have the worst of both worlds.
I understand that not everyone has an OSS portfolio, but it's better than having 5 different companies give me a unique take-home project, and then having another 3 companies give me an algorithms assessment. If everything was "let's take a look at one repo you've spent <10 hours on" or "let's do leetcode problems" that would be preferable to the current state of affairs, because as it stands I have the worst of both worlds.
The best is when they give you a take-home problem and an agreement: "we own all the code you write, dont show this to anyone"
Our tech interviews focus on watching you work the way you actually work: Google allowed, your IDE of choice, your boilerplate project templates, etc. Yes, you still have to show progress against a problem in 30-60m -- but sometimes rapid prototyping is important to the job!
They are missing the point.
Coding interview has very little to do with Code!
I hire a lot of developers and the code interview is a pivotal point in the process. This is where I find out how you collaborate, take new ideas and incorporate them, deal with changing requirements, handle criticism etc.
Of course I am validating you actually know how to code but I am really looking for the rest other skills because this is what will make you a valuable integral part of the development environment here.
Trying to throw a 'stump the jock' type of question at a developer is worthless and only proves to show how well they memorize things.
Throwing them a difficult problem that requires them to research, ask questions, collaborate will show how well they will grow and take on new challenges they are uncomfortable with.
Coding interview has very little to do with Code!
I hire a lot of developers and the code interview is a pivotal point in the process. This is where I find out how you collaborate, take new ideas and incorporate them, deal with changing requirements, handle criticism etc.
Of course I am validating you actually know how to code but I am really looking for the rest other skills because this is what will make you a valuable integral part of the development environment here.
Trying to throw a 'stump the jock' type of question at a developer is worthless and only proves to show how well they memorize things.
Throwing them a difficult problem that requires them to research, ask questions, collaborate will show how well they will grow and take on new challenges they are uncomfortable with.
I disagree that coding challenges are stupid, I actually find them having some of the best signal-to-noise ratio of any step in the interview. Maybe it depends a lot on what type of coding problem it is?
One important aspect of doing the coding interview well is to make it generic enough and not look for a specific exact solution, so things like "write a function for a movie theater ticket dispenser" vs "write the most efficient string reversal" function.
I am also totally fine if a candidate needs to look something up, but I don't think it is unreasonable to expect candidates to be able to work though a general coding exercise without needing to stop and google things frequently. Especially if they can pick their own preferred language!
One important aspect of doing the coding interview well is to make it generic enough and not look for a specific exact solution, so things like "write a function for a movie theater ticket dispenser" vs "write the most efficient string reversal" function.
I am also totally fine if a candidate needs to look something up, but I don't think it is unreasonable to expect candidates to be able to work though a general coding exercise without needing to stop and google things frequently. Especially if they can pick their own preferred language!
It's largely in the execution. Coding exercises are probably the best thing you can do, but many interviewers use them to ask bizarre trivia and look down on people who don't have it memorized.
Some context, first: I work on infrastructure software. I'll try to define it so it's not so vague: software that is core to the company's platform that has strict performance, stability, and FMEA guarantees. Often the skillset we look for is niche, you need systems skills (broad spectrum, both OS and network) as well as software skills (preferred polyglot; "languages are tools" kind of people).
On one hand, I deplore the runaway game that software has let algorithmic knowledge play on our minds. People destroy themselves preparing for interviews at large firms on exciting teams, only to get blown away because someone threw them a curveball in algorithm or systems design knowledge. On the other hand, I've watched developers copy solutions on one screen and transpose them on another. I've hired engineers only to find out they only know the kind of information we would ask in an interview and how to bullshit.
College will never provide my team the guarantees we need to make a hire, that much is well-accepted. Being from a "top tier" company is no guarantee either. What I do think is worthwhile is investing in two things simultaneously:
1. An apprenticeship program. I entered software (and mathematics in general) without a degree. These subjects can be learned with self-discipline and having a career that gradually increases in complexity is a perfect aid to this learning process. In my path, it produced a generalist software engineer with a specialty in systems engineering. In another, it could produce a web backend software engineer, a frontend engineer, an infrastructure engineer, etc...
2. A certification exam. I mean something orthogonal to Professional Engineering or the Bar exam. Something entirely divorced from colleges that can prove you can negotiate algorithms and data structures. I believe people who are keen enough to figure this out once can figure it out again when the time is needed, however, in most cases your brains will be put to more abstract talents that are non-numerical in nature. Reserving your sanity for then, rather than depleting it all throughout the interviewing process over the course of your career seems to me to be the optimal choice.
On one hand, I deplore the runaway game that software has let algorithmic knowledge play on our minds. People destroy themselves preparing for interviews at large firms on exciting teams, only to get blown away because someone threw them a curveball in algorithm or systems design knowledge. On the other hand, I've watched developers copy solutions on one screen and transpose them on another. I've hired engineers only to find out they only know the kind of information we would ask in an interview and how to bullshit.
College will never provide my team the guarantees we need to make a hire, that much is well-accepted. Being from a "top tier" company is no guarantee either. What I do think is worthwhile is investing in two things simultaneously:
1. An apprenticeship program. I entered software (and mathematics in general) without a degree. These subjects can be learned with self-discipline and having a career that gradually increases in complexity is a perfect aid to this learning process. In my path, it produced a generalist software engineer with a specialty in systems engineering. In another, it could produce a web backend software engineer, a frontend engineer, an infrastructure engineer, etc...
2. A certification exam. I mean something orthogonal to Professional Engineering or the Bar exam. Something entirely divorced from colleges that can prove you can negotiate algorithms and data structures. I believe people who are keen enough to figure this out once can figure it out again when the time is needed, however, in most cases your brains will be put to more abstract talents that are non-numerical in nature. Reserving your sanity for then, rather than depleting it all throughout the interviewing process over the course of your career seems to me to be the optimal choice.
What a way to add nothing to a conversation horse that's been beaten to death as is.
The problem result is not the goal of the exercise, I don't care if you solve it, but rather how you approach it.
Note that I don't use puzzle problems, but more open-ended ones that elicit discussion and architectural approaches rather than simple coding.
When I'm interviewing I'm google for all intents and purposes.
I want to see how you interact, what you're thinking and how it would be like working with you as a team member.
I look for potential and red flags. I try to find out two main things:
- will you be good for the team?
- will the team be good for you?
It's a two way relationship we're trying to build here.
Interviewing is a lot like speed dating, we're trying to find a long term partner in a very small amount of time.
Note that I don't use puzzle problems, but more open-ended ones that elicit discussion and architectural approaches rather than simple coding.
When I'm interviewing I'm google for all intents and purposes.
I want to see how you interact, what you're thinking and how it would be like working with you as a team member.
I look for potential and red flags. I try to find out two main things:
- will you be good for the team?
- will the team be good for you?
It's a two way relationship we're trying to build here.
Interviewing is a lot like speed dating, we're trying to find a long term partner in a very small amount of time.
[deleted]
This is a bad take. Coding interviews help you see how someone solves a problem. And good coding interviews let you search for whatever you want except for "how to do this problem in x language."
I've done more interviews over the last two years than I'd like, and honestly - I have no problem with folks googling "how do I solve X in language Y".
Frankly, we tell folks up front - "This is not a memory test, please feel free to ask questions, use google/stack overflow/man-pages/etc."
I'm in the interview to give you a problem that looks as close as possible to the work we're going to ask you to do, and then discuss with you how you solved that problem.
If you copy from stack overflow and it works, great! Now we can talk about possible edge cases, performance issues, testing, add a new requirement, etc.
I don't really care how you get the code into the file, or where it comes from - I care that you understand it, and can think through how that code will perform in different situations. Most importantly of all - I care that you can tell me that information in a coherent fashion.
Frankly, we tell folks up front - "This is not a memory test, please feel free to ask questions, use google/stack overflow/man-pages/etc."
I'm in the interview to give you a problem that looks as close as possible to the work we're going to ask you to do, and then discuss with you how you solved that problem.
If you copy from stack overflow and it works, great! Now we can talk about possible edge cases, performance issues, testing, add a new requirement, etc.
I don't really care how you get the code into the file, or where it comes from - I care that you understand it, and can think through how that code will perform in different situations. Most importantly of all - I care that you can tell me that information in a coherent fashion.
What is your process for evaluating "how someone solves a problem" and how does that inform your hiring decision? Does it simply come down to "does this person solve the problem the way I would"? How do you address confounding issues like performance anxiety induced by constraints not present in a normal work scenario?
My observation is that many companies don't address those problems well.
My observation is that many companies don't address those problems well.
I like the idea of letting the candidate do whatever they would if they were actually coding. I even think it could be interesting (in the initial part of the process) to ask the candidate what question they would like to be asked, and let them implement that.
Few things to point out though:
1. you can ask the interviewer questions
2. the problems are usually simple enough that you don't need to google them, and should have prepared for them in the first place
3. you should be somewhat efficient (being able to do basic stuff without googling) in at least one language, since you can choose the language for the interview do that
Few things to point out though:
1. you can ask the interviewer questions
2. the problems are usually simple enough that you don't need to google them, and should have prepared for them in the first place
3. you should be somewhat efficient (being able to do basic stuff without googling) in at least one language, since you can choose the language for the interview do that
When I interview using coding problems, you just need to know very basic Java or python to solve the problem.
We also nudge interviewees in the right direction to make sure they don't get stuck, which would be in no ones interest.
There is also no hard requirement that you need to solve the problem completely. If you talk to someone for 45 minutes about a coding problem you will get a good sense of their performance, even if they don't solve the problem.
Coding interviews are great (except making the intervieww very nervous), but you have to be mindful as an interviewer.
We also nudge interviewees in the right direction to make sure they don't get stuck, which would be in no ones interest.
There is also no hard requirement that you need to solve the problem completely. If you talk to someone for 45 minutes about a coding problem you will get a good sense of their performance, even if they don't solve the problem.
Coding interviews are great (except making the intervieww very nervous), but you have to be mindful as an interviewer.
There is a limit to the proposed approach. Most interviews I conduct (algorithms) include puzzle-like problems. So if you just Google them - you would find a complete answer. In that case, what's left for the interviewee to solve?
(Whether algorithm questions should be a part of the interview cycle is a different matter.)
I interviewed at SkyScanner around 5 years ago and they gave me some problems that I already had seen since I was interviewing like crazy. I solved them very easily and the interviewee had nothing else to say. Then her feedback to the interview was negative because it looked like I had already seen the problem.. like... :shrugs:
Yes, Skyscanner is a horrible at hiring. Can't even bother to reject you or give feedback in the last round
Poorly written blog post. What an elaborate way to say "let me google". Except for hackerank type of interview, I've always been able to google or ask interviewers to help me out.
Just have some communication skill and communicate in the interviews, they want to test that in you too, rightfully so.
Just have some communication skill and communicate in the interviews, they want to test that in you too, rightfully so.
I much prefer interview styles where the candidate and the interviewer just talk about a technical solution they've solved in detail (challenges, architectural decisions, etc...). It'll become evident fairly quickly if the candidate doesn't know what they're talking about.
Except for me there is a half life of about 48 hours on how detailed I can talk about that solution.
This has me thinking that interviews should openly publish what types of question you’ll be asked so you can prepare.
Otherwise it’s prepare for anything, which is 20 or more hours work per interview, and requires me deciding what animal I am.
This has me thinking that interviews should openly publish what types of question you’ll be asked so you can prepare.
Otherwise it’s prepare for anything, which is 20 or more hours work per interview, and requires me deciding what animal I am.
Where are you interviewing that they don't let you google stuff? I've gotten to the point that I don't even ask anymore, I just open a new tab and explain what I'm looking for. If the interviewer takes issue with it that's a big red flag for me.
I find the 'take home exercise' to be tolerable. I hate the live coding things. Those are awful.
Even the take home things are not fantastic, though - if you already have a job, a family and other stuff going on, "just 4 hours" is kind of a PITA.
Even the take home things are not fantastic, though - if you already have a job, a family and other stuff going on, "just 4 hours" is kind of a PITA.
"Just 4 hours" for a chance at the job, and how high a chance depends on the employer (i.e. how early in the process they drop the take-home exercise on you, whether you've already come to terms on pay at that point, et c.)
In general I just wish places that weren't FAANG would be more open about their process. Do I need to brush up on leetcode shit before applying, or no? Are you gonna quiz me on obscure corners of a language that I reflexively avoid because they're terrible, so haven't thought about in a while? Gonna talk architecture? Will I be writing code with someone looking over my shoulder? Pairing? On what kind of thing, generally? Is your filter for the 200 candidates who apply to hit them with a take-home assignment or longish automated test first thing, so I know to just skip applying at all (screw burning that much time to apply for the privilege of getting an interview at just one place)? With FAANG you know what you're getting in to, so at least you know you're about to step in a big pile of shit and can put on your waders first. The blind roulette wheel of hiring outside FAANG is its own kind of trash-fire.
In general I just wish places that weren't FAANG would be more open about their process. Do I need to brush up on leetcode shit before applying, or no? Are you gonna quiz me on obscure corners of a language that I reflexively avoid because they're terrible, so haven't thought about in a while? Gonna talk architecture? Will I be writing code with someone looking over my shoulder? Pairing? On what kind of thing, generally? Is your filter for the 200 candidates who apply to hit them with a take-home assignment or longish automated test first thing, so I know to just skip applying at all (screw burning that much time to apply for the privilege of getting an interview at just one place)? With FAANG you know what you're getting in to, so at least you know you're about to step in a big pile of shit and can put on your waders first. The blind roulette wheel of hiring outside FAANG is its own kind of trash-fire.
I remember this with a newborn I spent 20 minutes on such a test. Recruiter tells me they said no because code quality so i gave it another shot spending 2 hours. Remember with multiple applications these times added up.
Got to the interview and they were so obsessed about this being written in some “perfect way” (see the enterprise fizz buzz someone linked to) that they knocked me back when I was explaining it at the interview.
It’s the coding equivalent of “ooh so you are not using a suse vide” at a mates bbq.
Got to the interview and they were so obsessed about this being written in some “perfect way” (see the enterprise fizz buzz someone linked to) that they knocked me back when I was explaining it at the interview.
It’s the coding equivalent of “ooh so you are not using a suse vide” at a mates bbq.
I want to take one of these interviews just to look the interviewer in the eyes and ask one question.
Since I'm not allowed to google stuff now, would that mean I won't be allowed to do that while working for your company as well?
Since I'm not allowed to google stuff now, would that mean I won't be allowed to do that while working for your company as well?
Ok, go ahead. Ask me a question. I've done > 400 interviews at FAANG companies, so I guess I'm due.
[deleted]
> I understand why interviewers do this. They want to get a sense for what candidates actually know versus what they’ve copy/pasted from the web. But this process is stupid for a variety of reasons.
That's not how I understand them. I always saw them as a way to see if the candidate is ready to invest a certain amount of time and energy. A bit like "if you can grind leetcode and pass the X interview, you can probably contribute to our codebase".
That's not how I understand them. I always saw them as a way to see if the candidate is ready to invest a certain amount of time and energy. A bit like "if you can grind leetcode and pass the X interview, you can probably contribute to our codebase".
> Coding is solving problems. A big part of that is looking stuff up. Like, seriously, the most important skill a developer can have it knowing how to research solutions to problems. Specifically…
Rather than have them Google things, there's an open dialog with the interviewer where the candidate thinks out loud and the interviewer gives the 'search results' for them to use now, later, or not.
The interview is not about 'if' the candidate can solve the problem. Even it they don't the result can be positive. It's about how they think, break things down, compose the parts into a solution. No one should care if you memorized the syntax of anything.