How to Interview Entry Level Software Engineers(technology.cloverhealth.com)
technology.cloverhealth.com
How to Interview Entry Level Software Engineers
https://technology.cloverhealth.com/how-to-interview-entry-level-software-engineers-daaecf9db97
129 comments
I think the bigger issue is that the interview-to-hire process is not fool-proof, and never will be.
It's a gamble, no matter how in depth or complicated your hiring process is (from personality interviews to whiteboarding to take homes). Being interviewed and doing well is a skill set and it's very different from a full time work-place skill set.
All of these interviewing processes that get refined and more lengthy just create a new class of people who are good at being interviewed.
I don't know what the solution is, but making candidates jump through 3-4 interview rounds, memorize algorithms just for the sake of it, and go through 4 to 8 hour on sites probably isn't it. Even if it happens to give you a better-than-average employee, what are you sacrificing (as a business) in terms of man-hours, productivity, and focus?
Personally I'd like to see more companies institute a temp-to-hire process. Bring in new candidates for a two or three week contract, see how they fit in, pay for the their time (at a discount), and then go from there. If it doesn't work out, you get some work done for your company, and they get some relevant experience for their resume.
Writing the perfect resume and re-reading CTCI for the 10th time won't tell you how well the candidate will perform, but seeing them actually work might.
As to your point about marketing - I agree. Not one of these approaches are really novel or helpful to anyone but the company themselves, and is just a glorified "look at how we took a chance" tactic.
It's a gamble, no matter how in depth or complicated your hiring process is (from personality interviews to whiteboarding to take homes). Being interviewed and doing well is a skill set and it's very different from a full time work-place skill set.
All of these interviewing processes that get refined and more lengthy just create a new class of people who are good at being interviewed.
I don't know what the solution is, but making candidates jump through 3-4 interview rounds, memorize algorithms just for the sake of it, and go through 4 to 8 hour on sites probably isn't it. Even if it happens to give you a better-than-average employee, what are you sacrificing (as a business) in terms of man-hours, productivity, and focus?
Personally I'd like to see more companies institute a temp-to-hire process. Bring in new candidates for a two or three week contract, see how they fit in, pay for the their time (at a discount), and then go from there. If it doesn't work out, you get some work done for your company, and they get some relevant experience for their resume.
Writing the perfect resume and re-reading CTCI for the 10th time won't tell you how well the candidate will perform, but seeing them actually work might.
As to your point about marketing - I agree. Not one of these approaches are really novel or helpful to anyone but the company themselves, and is just a glorified "look at how we took a chance" tactic.
This idea of temp-to-hire places a disproportionate amount of risk on the employee to be hired -- they will need to leave their current job (where they likely have decent job security) in order to attempt being hired at the new company.
In the real world, people generally don't put in their notice until they've accepted an offer at another company.
In the real world, people generally don't put in their notice until they've accepted an offer at another company.
It also abuses the system by not paying the going short term contractor rate some non tecnicaly savvy companies think that wanting some one to work a 4-6 month contract and only pay you as if you where an employee and not self employed rate.
In the real world there are a lot of people who don't currently have a job. Sure unemployment percentage is low (even if we discount accusations that the percentage is miscounted), but there are still a lot of people looking for a job. When someone doesn't have a job temp-to-hire isn't nearly as risky because if it doesn't work out you are back where you were - sometimes with renewed unemployment benefits.
Actually your downside is a side benefit to employers: you don't risk hard feeling by hiring somebody away from their old job if you latter need to work with them. (these shouldn't happen, but they do)
Actually your downside is a side benefit to employers: you don't risk hard feeling by hiring somebody away from their old job if you latter need to work with them. (these shouldn't happen, but they do)
Generally people who would make good employees are already employed (assuming there is demand for their labor). Hiring unemployed people is usually targeting the bottom of the market, or finding diamonds in the rough, which is risky.
I'm not sure if this is actually the case. I'm starting to meet more and more people who aren't that keen on amassing wealth, and they tend to spend decent stretches of time unemployed (traveling, working on side projects, bootstrapping companies etc)
Temp to hire wouldn't be so bad, if there was a guarantee from the company that they will either hire you, or replace you with another contractor with the intent to hire them. In that case it is about the same as having a 6-month probationary period.
I wonder if there are statistics on contract to hire positions that don't get hired, and out of those that don't, is it because the person really didn't work out or was there really no intent to hire to begin with?
As a point of reference, where I work they almost always to contract to hire. I've seen a few contractors leave (they found other work), some didn't get converted to full time (those weren't actually to-hire though, and specifically short-term, although in one case a contractor was brought back twice to fill in). But in almost every other case the person was hired eventually (even if it took a couple extra months to push it through HR).
I wonder if there are statistics on contract to hire positions that don't get hired, and out of those that don't, is it because the person really didn't work out or was there really no intent to hire to begin with?
As a point of reference, where I work they almost always to contract to hire. I've seen a few contractors leave (they found other work), some didn't get converted to full time (those weren't actually to-hire though, and specifically short-term, although in one case a contractor was brought back twice to fill in). But in almost every other case the person was hired eventually (even if it took a couple extra months to push it through HR).
We are talking about an entry level position with the understanding the candidate's skill set are not truly refined yet in the real world.
Maybe my view is warped because I don't work for an SV startup, but I would consider anyone with less than 5-10 years "entry level". I rarely see job ads for anything except interns/entry level and "senior engineers". Someone with 3-5 years of experience probably has some job security but definitely doesn't count as "senior" in my mind.
I don't think it's just SV, I've seen job postings in Boston for 3+ years experience and it was listed as a senior engineer. I think it's because many companies try to use titles as compensation. I've gotten the "senior engineer" title 3 times in my career. Two of those times it was given to me in the same conversation where I was told there was going to be no raises that year.
“We’re rewarding you this year by making it easier for you to find another job” sounds insane when you say it like that, but that’s what they did!
For an entry level position? Nah
Contrary to the analogies being drawn by sibling and child comments, work is different from dating or interviewing potential roommates, in that you typically have to give up a previous job when accepting a new one and if you don't make it through the trial run, you're unemployed.
It would be great if the only reason people could be let go were legitimate, but that's hardly the case.
It would be great if the only reason people could be let go were legitimate, but that's hardly the case.
It is impossible because it is always possible to game the system.
If you devise an interview framework or a hiring profile, no matter how many traits you want to evaluate(as the article points out), they will lump those together and measure in some scale of how likely they are to hire the candidate(sometimes that is even just binary), if a candidate can emulate their desired profile he gets a high "score" and gets an offer, the bigger the financial incentives to do this, more people will game the system(prepare for specific interviews), think, if you can play the part for a couple of 1-hour interviews, you can get a 6-figure salary at a tech startup, that is a high ROI.
Most companies can't stop that, there isn't limitless budget for interviewing so the interview process has limited resources to evaluate people, which leads to false positives. There are also a lot of talented people that don't practice their interviewing skills enough and end up as false negatives.
It's human nature, people need to evaluate things on a single criterion and you can game that. The same thing happens with school, you can work your way through school material to maximize your grades without learning much.
If you devise an interview framework or a hiring profile, no matter how many traits you want to evaluate(as the article points out), they will lump those together and measure in some scale of how likely they are to hire the candidate(sometimes that is even just binary), if a candidate can emulate their desired profile he gets a high "score" and gets an offer, the bigger the financial incentives to do this, more people will game the system(prepare for specific interviews), think, if you can play the part for a couple of 1-hour interviews, you can get a 6-figure salary at a tech startup, that is a high ROI.
Most companies can't stop that, there isn't limitless budget for interviewing so the interview process has limited resources to evaluate people, which leads to false positives. There are also a lot of talented people that don't practice their interviewing skills enough and end up as false negatives.
It's human nature, people need to evaluate things on a single criterion and you can game that. The same thing happens with school, you can work your way through school material to maximize your grades without learning much.
An interview is like dating and sometimes you think you found the one but maybe it doesn't work out in the end. Do you stop interviewing (or dating) people? No, you try again and hopefully take lessons from the last failed hire. A "failed hire" could actually be a failed process in your organization.
Also, every person you are interviewing has the option to interview with other companies. With that in mind, you should always be interviewing even if you aren't actually looking to hire at that moment. Sometimes you might find a really good candidate but, most importantly, you will have options yourself.
Last, I really like your idea of a paid temp position to assess fit. This will benefit everyone tremendously.
Also, every person you are interviewing has the option to interview with other companies. With that in mind, you should always be interviewing even if you aren't actually looking to hire at that moment. Sometimes you might find a really good candidate but, most importantly, you will have options yourself.
Last, I really like your idea of a paid temp position to assess fit. This will benefit everyone tremendously.
paid temp work, as has been said before, is a great excuse for companies to expand the gig economy. or better yet, an excuse to get some good quick expendable labor and then dump the employee when his work is done. I've seen it happen time and again.
I can't imagine a company spending the time and money marketing a position, going through an interview process, offering a temp-to-hire position and then letting the person go shortly thereafter. That is such a waste of resources.
If it was the company's goal to crank out gig workers then there are some really big problems happening below the surface at that company and it was in the candidate's best interest not to get a full time offer.
If it was the company's goal to crank out gig workers then there are some really big problems happening below the surface at that company and it was in the candidate's best interest not to get a full time offer.
man, let me introduce you to the world of consulting. it's pervasive.
Hilarious example to me, because while I’ve been a consultant or contractor seen as an exellent and valued member of a team (being the only contractor) they still insist (perhaps even on the hr level now) that I pass the interview process which is all of the first part that you describe. I’d say it’s their loss but I like it where I am. There is no recourse and everyone around me acknowledges that it’s broken, but nobody wants to change it.
Why though? Why is programming seemingly the only field where companies feel the need to put up these super interview processes with mutiple interview rounds, technical tests, white boarding, etc?
Is it the high salaries? In that case, do you see such testing in law, or finance?
Is it the technical difficulty? In a previous life (read: up to a few months ago) I worked as a mechanical engineer, in a pretty technical field (numerical simulation of structures). Additionally to demanding reasonably deep knowledge in a few areas, the job came with the responsibility of making sure that your designs do not kill or injure people.
I had never had a technical test in an interview before I interviewed for my current IT position. Nor have I heard of colleagues having such a test. Is it because most engineers in Canada are licensed (myself included), and most jobs require you be licensed? If that's the case, it's pretty funny, because at least in my province, the test you have to pass for the license is entirely about ethics and professionalism, nothing technical.
Part of the code of ethics though, is that it's on you as a professional to ensure you have the necessary competence to do the work.
Is it the high salaries? In that case, do you see such testing in law, or finance?
Is it the technical difficulty? In a previous life (read: up to a few months ago) I worked as a mechanical engineer, in a pretty technical field (numerical simulation of structures). Additionally to demanding reasonably deep knowledge in a few areas, the job came with the responsibility of making sure that your designs do not kill or injure people.
I had never had a technical test in an interview before I interviewed for my current IT position. Nor have I heard of colleagues having such a test. Is it because most engineers in Canada are licensed (myself included), and most jobs require you be licensed? If that's the case, it's pretty funny, because at least in my province, the test you have to pass for the license is entirely about ethics and professionalism, nothing technical.
Part of the code of ethics though, is that it's on you as a professional to ensure you have the necessary competence to do the work.
I've been both a professional pilot and software engineer. The process to become a pilot was more intense in time and cost than to become a software engineer. The interview process for a job as a pilot is much easier than the interview process for a job as a software engineer.
The barrier to entry for software engineer jobs is all at the interview. That's why it's so hard, tedious, and complicated.
The barrier to entry for software engineer jobs is all at the interview. That's why it's so hard, tedious, and complicated.
Imagine if this article was intended for pilots and it said:
"How to interview entry level pilots for A380. On the job training provided if you are fresh graduate"
I'm aware it's a regulated profession, but imagine if it wasn't? Whats there to stop management from actually fast tracking the backup pilots as that?
I'm aware it's a regulated profession, but imagine if it wasn't? Whats there to stop management from actually fast tracking the backup pilots as that?
That's kinda how it works in Asia. You sign a seven year contract, they send you to a flight school where you learn the basics, and they stick you as like fifth-in-command in an airliner. At some airlines you have to work for the company in some other capacity first.
On the job training provided if you are fresh graduate
That is, in fact, how every Air Force in the world recruits pilots.
That is, in fact, how every Air Force in the world recruits pilots.
>, or finance?
Tricky brainteaser questions are popular in quant finance interviewing. Example[1]. There are also discussion forums dedicated to it. Example[2].
Some electrical engineering firms will ask you to analyze complicated circuits in the interview. Some even have "take home tests" where the candidates design a circuit according to the interviewer's specification.
[1] http://www.streetofwalls.com/finance-training-courses/quanti...
[2] https://www.quantnet.com/forum/quant-interviews.41/
Tricky brainteaser questions are popular in quant finance interviewing. Example[1]. There are also discussion forums dedicated to it. Example[2].
Some electrical engineering firms will ask you to analyze complicated circuits in the interview. Some even have "take home tests" where the candidates design a circuit according to the interviewer's specification.
[1] http://www.streetofwalls.com/finance-training-courses/quanti...
[2] https://www.quantnet.com/forum/quant-interviews.41/
I think it's a multitude of factors. These jobs
1. Pay a lot. Barring trading, you'd be hard pressed to find a job that routinely pays 200K+ to 25-year olds
2. Are, at least most of them, not fundamentally difficult for a generally smart person, despite whatever people convince themselves of. I am not talking of the so-called 10x engineers here, who you are unlikely to find through standard interview techniques.
3. Pride themselves on NOT being credentialist (at least on the surface). You can't wake up tomorrow wanting to be a lawyer, read some books, and get a good job offer in a couple months. It's possible for CS jobs.
1. Pay a lot. Barring trading, you'd be hard pressed to find a job that routinely pays 200K+ to 25-year olds
2. Are, at least most of them, not fundamentally difficult for a generally smart person, despite whatever people convince themselves of. I am not talking of the so-called 10x engineers here, who you are unlikely to find through standard interview techniques.
3. Pride themselves on NOT being credentialist (at least on the surface). You can't wake up tomorrow wanting to be a lawyer, read some books, and get a good job offer in a couple months. It's possible for CS jobs.
>>Barring trading, you'd be hard pressed to find a job that routinely pays 200K+ to 25-year olds
Routinely? Hahaha.
Those salaries are pretty much exclusive to a few locations, primarily Bay Area and maybe NYC.
Routinely? Hahaha.
Those salaries are pretty much exclusive to a few locations, primarily Bay Area and maybe NYC.
Not sure why you are laughing. Bay area is the hub of the so-called CS job industry and the cult-ish hiring practices that are being discussed in this thread largely originate from there and are used there.
I'm not sure I understand this comment -- seems like you wholly agree with the sentiment you quoted? You might be hard-pressed outside of the cities you mentioned, but they are highly plausible in those and other cities (e.g. Seattle).
It might be because writing code is as close to magic as you can get. It does real world things, like banking and long distance communication, but it is invisible and in an arcane tongue.
If I see a bridge, I don't need to be an engineer to gauge if it is likely to support my weight. But if the bridge is invisible, I probably want significant assurance that no one is pulling a fast one here, so to speak.
If I see a bridge, I don't need to be an engineer to gauge if it is likely to support my weight. But if the bridge is invisible, I probably want significant assurance that no one is pulling a fast one here, so to speak.
A big part of it, which you allude to, is that many people are either self taught entirely, or having studied CS during undergrad, are self taught in practical software development.
In a field like mechanical engineering, I assume an employer can take for granted that a new grad has a certain set of skills (general physics, structures, fluids, etc.). That can't be assumed in quite the same way in software engineering.
In a field like mechanical engineering, I assume an employer can take for granted that a new grad has a certain set of skills (general physics, structures, fluids, etc.). That can't be assumed in quite the same way in software engineering.
Interesting thought, since I don't have a degree in CS, I'm not sure what it entails. Does it not include any "practical programming" at all?
In that case, should we conclude that universities are the failing piece here, and that is why even with a 4 year degree, employers feel the need to test candidates for their knowledge?
In that case, should we conclude that universities are the failing piece here, and that is why even with a 4 year degree, employers feel the need to test candidates for their knowledge?
No, CS degrees are designed to teach computer science. That's sort of vaguely related to "practical programming." Software engineering the way it's done in most companies is almost nothing like computer science and very much like a trade. The real problem is that programming trade schools are viewed as "below" a CS degree rather than a more directly applicable training/education.
The better analogy is to compare a theoretical physics degree to a mech civil or electrical engineering degree.
Not really, because unlike trade degrees, mech civil or elec engg degrees are prestigious, and often even more so than theoretical physics.
The better analogy is to compare an engineering degree to an AutoCAD training.
Are your sure it's just software? I work in the aerospace field, where we hire pretty much ever kind of engineer there is. The hiring process looks pretty similar to me for all engineering disciplines.
You're kidding, right? That other industries don't have high entry barriers? Just some I can name off the top of my head...
Medicine, law, pharmacy, actuarial science, accounting...
For most upper class jobs it's the norm.
Medicine, law, pharmacy, actuarial science, accounting...
For most upper class jobs it's the norm.
I edited my post to reflect my meaning better. I didn't mean high barriers generally, such as a degree, because that definitely isn't specific to software.
I was thinking of the complicated interview process, which seems rather exclusive to software, at least as far as I know.
Medicine, pharmacy, actuarial are all licensed professions. A lot (most?) law and accounting practitioners are also licensed.
So, is the difference for software that it doesn't require a professional license, so employers are forced to sort of invent their own "licensing" in the form of these very difficult interviews?
Not that I'm advocating for licensing by the way, it comes with its own set of problems (notably rent seeking behavior). Just trying to understand why software as a field seems to stand out here.
I was thinking of the complicated interview process, which seems rather exclusive to software, at least as far as I know.
Medicine, pharmacy, actuarial are all licensed professions. A lot (most?) law and accounting practitioners are also licensed.
So, is the difference for software that it doesn't require a professional license, so employers are forced to sort of invent their own "licensing" in the form of these very difficult interviews?
Not that I'm advocating for licensing by the way, it comes with its own set of problems (notably rent seeking behavior). Just trying to understand why software as a field seems to stand out here.
You answered your question. A degree is also a kind of 'license' (albeit a weaker form than a so-called professional license) but people frown at using them because it sounds credentialist.
This industry hates credentials because it doesn't trust credentials. Which makes it ironic when it's members come here and say other disciplines don't have the same issues as soon us because credentials.
When you have a job that pays a lot of money and requires no formal credentials whatsoever, people can and will try to get into that job even though they’re not qualified for it.
This is why there are programming candidates failing fizzbuzz.
It’s the equivalent of someone applying for a pharmacist position and not knowing what antibiotics are, but it happens, because anyone can call themselves a programmer.
This is why there are programming candidates failing fizzbuzz.
It’s the equivalent of someone applying for a pharmacist position and not knowing what antibiotics are, but it happens, because anyone can call themselves a programmer.
Yea I think it is precisely that.
When I interview at AmaGoogFaceSoft, they can't take my word for it that I know how to code my way out of a cardboard box. So my interviews tend to be fairly technical, making me prove that I know what I know, and might actually be a useful asset to the company.
When my wife (a CPA) interviews, they know that she knows a whole bunch of stuff, by virtue of being able to go the accreditation board. So when she interviews, it tends to be a super mile high sanity check (have you done accounts receivable/payable before, yes? no? If no that's ok too...)
I can't speak for medical, but I have family in law and it tends to lean a lot more towards the accounting style for interviewing (barring maybe some patent law or other specializations). E.g. one had an interview for a corporate banking regulatory position that went roughly "Oh you have no experience in banking or regulatory law? Well that's ok, welcome aboard anyways."
When I interview at AmaGoogFaceSoft, they can't take my word for it that I know how to code my way out of a cardboard box. So my interviews tend to be fairly technical, making me prove that I know what I know, and might actually be a useful asset to the company.
When my wife (a CPA) interviews, they know that she knows a whole bunch of stuff, by virtue of being able to go the accreditation board. So when she interviews, it tends to be a super mile high sanity check (have you done accounts receivable/payable before, yes? no? If no that's ok too...)
I can't speak for medical, but I have family in law and it tends to lean a lot more towards the accounting style for interviewing (barring maybe some patent law or other specializations). E.g. one had an interview for a corporate banking regulatory position that went roughly "Oh you have no experience in banking or regulatory law? Well that's ok, welcome aboard anyways."
> When my wife (a CPA) interviews, they know that she knows a whole bunch of stuff, by virtue of being able to go the accreditation board. So when she interviews, it tends to be a super mile high sanity check (have you done accounts receivable/payable before, yes? no? If no that's ok too...)
All the accreditation board knows is that she knew enough at one time to pass the tests. It says nothing about whether she still knows that stuff today.
It's the same thing with a PE license. In fact, with a PE the advice is to take the test as soon after graduation as you are able, because the further out of school you are the harder it is to pass. So, it says you knew that stuff once. It cannot be used as a measure of current competency.
All the accreditation board knows is that she knew enough at one time to pass the tests. It says nothing about whether she still knows that stuff today.
It's the same thing with a PE license. In fact, with a PE the advice is to take the test as soon after graduation as you are able, because the further out of school you are the harder it is to pass. So, it says you knew that stuff once. It cannot be used as a measure of current competency.
Well for CPA at least, there's continuing education requirements you have to meet, which varies by state. I dunno how it works for a PE.
PEs have continuing education requirements as well. They are, largely, a joke compared to what software engineers are supposed to do to prepare for a single FAANG interview. For example, Texas (where I would have been licensed had I pursued it) requires 15 hours per year, of which at least one must be ethics[1]. I went to a one-day power supply seminar sponsored by TI (just because I wanted to) that would have covered half that requirement. Are two day-long seminars really enough to keep current in anything technical?
[1] http://engineers.texas.gov/cep_info.htm
[1] http://engineers.texas.gov/cep_info.htm
Engineering licensing also varies a lot by jurisdiction (by state in the US).
In my (canadian) province, the actual license test has no technical component, but there is a continuing education requirement, plus the deontology/ethics requirement I mentioned in the GP.
In my (canadian) province, the actual license test has no technical component, but there is a continuing education requirement, plus the deontology/ethics requirement I mentioned in the GP.
Medicine, pharmacy, nursing, and optometry, at least, also require continuing education contact hours in order to maintain a professional license.
I've never done hiring, but I've chosen roommates, and I feel like it's the same. You make a checklist of stuff you're looking for and red flags, do a reference check or whatever, but ultimately, the decision always comes down to some gut impulse. My gut has always been right so far, but I guess the day it burns me is the day I start trying to find some optimal solution to protect myself from letting an asshole move in.
The problem with using gut instincts for these kinds of decisions is that it introduces bias and discrimination. For a roommate that might be positive as it will bias towards people that are similar to you, but for companies it's too easy to end up making rooms full of middle class white dudes.
Exactly. While it isn't ideal for your personal development as a human being to always hang out and live with people who confirm your unconscious biases, at least there it's somewhat understandable and isn't the end of the world. But allowing your unconscious biases to influence hiring decisions is disastrous for diversity.
Anecdotally, most if not all of the times I’ve been burned has been when I ignored my gut feeling.
I have the same anecdote :) the interview all checked out, everybody gave a thumb up, but my guts were telling me "it's not gonna work". I forced myself to ignore it, put it on the account of unfair bias or something and hired the candidates. After 6 to 8 months they were gone after a disastrous tenure.
It actually shook me a little after the 2nd time, because I thought I was doing the right thing not dismissing people just because of gut feeling.
It actually shook me a little after the 2nd time, because I thought I was doing the right thing not dismissing people just because of gut feeling.
The approach I prefer is have one of your tops talk to them about technology for an hour or less. Talk about their favorite projects, least favorite projects, favorite code, etc. The top will be able to tell if they have the interest and ability. The interest is what makes a entry level developer improve.
Wholeheartedly agree.
Make the process a conversation rather than an interview. Get to meet the person, have lunch with them, talk about what motivates them and what they're passionate about, discuss tech stuff, and at some point during that hour or two both you and them will have an idea whether you want to turn it into a steady relationship. :)
I like to add a code review stage to the mix, where we show them a small chunk of intentionally badly written code that might come up during a code review, which has both obvious and more obscure issues related to syntax, clarity, efficiency, etc., and you judge how they approach it and what issues they bring up. I've found this to be quite a useful metric that mostly replicates a real world scenario, but in a more comfortable setting than asking them to write code on a whiteboard.
And you try your hardest to block any biases for/against them you might have when making your decision.
Make the process a conversation rather than an interview. Get to meet the person, have lunch with them, talk about what motivates them and what they're passionate about, discuss tech stuff, and at some point during that hour or two both you and them will have an idea whether you want to turn it into a steady relationship. :)
I like to add a code review stage to the mix, where we show them a small chunk of intentionally badly written code that might come up during a code review, which has both obvious and more obscure issues related to syntax, clarity, efficiency, etc., and you judge how they approach it and what issues they bring up. I've found this to be quite a useful metric that mostly replicates a real world scenario, but in a more comfortable setting than asking them to write code on a whiteboard.
And you try your hardest to block any biases for/against them you might have when making your decision.
Here's an anecdote for a different approach that wasn't intential, but wound up giving the company a good sampling of my skills and personality, and me a good sampling of what it was like working for them. I volunteer some consulting and IT services for a local non-profit. They were having another company build some business specific software for them, which had been recommended to them by their board(client is part of a franchise structure of sorts). It was far past schedule, and from my perspective the client had misunderstanding of the scope and timeframe for this type project, which let to the client questioning me "should we just scrap it?".
By volunteering along side the company doing the software build(which was also volunteering, mostly) we were able to get the project on track, and update the client's expectations to a more realistic view of the timeframe and outcome.
By volunteering alongside them, for another org that we both were happy to support, I didn't feel like I was doing free work for them, and they weren't struggling to pay full salary and onboarding cost to find out the same.
Just a thought for an approach to get a sample of each other without one party doing free work for the other.
By volunteering along side the company doing the software build(which was also volunteering, mostly) we were able to get the project on track, and update the client's expectations to a more realistic view of the timeframe and outcome.
By volunteering alongside them, for another org that we both were happy to support, I didn't feel like I was doing free work for them, and they weren't struggling to pay full salary and onboarding cost to find out the same.
Just a thought for an approach to get a sample of each other without one party doing free work for the other.
I don't think the problem is about lack of innovative thought. The problem is a lack of a verification mechanism. Say I come to you with an amazing new black box. A candidate puts their hand in the box, and if a light on top of the box turns green, that will be a good employee and you should hire them. How do you verify whether my box works?
That's biased against the unhanded
For what it's worth, I'm one of the people who consistently shows up to rant about interviewing processes and what's wrong with them in HN threads that come anywhere near the topic.
I also work at the company this article is about, and the article was written by one of my colleagues. While no process is perfect (including ours -- both the one detailed in this article, and the one we use for general engineers), one of the things that I like about working where I do is the emphasis on putting thought into the interview process and questioning a lot of things that get assumed without a second glance by interview processes elsewhere.
I also work at the company this article is about, and the article was written by one of my colleagues. While no process is perfect (including ours -- both the one detailed in this article, and the one we use for general engineers), one of the things that I like about working where I do is the emphasis on putting thought into the interview process and questioning a lot of things that get assumed without a second glance by interview processes elsewhere.
You would probably enjoy http://n-gate.com .
[deleted]
Maybe if they hate the approach, the interview is a successful filter.
Well written, thanks for posting. This is my sentiment exactly.
How about employment by sortition: let the RNG decide.
Probably the only fair method.
> Also, we chose to completely scrap whiteboard coding and assigned homework instead.
For an entry level engineer, this removes a bunch of variance and also because it gets you another scenario which happens way often in actual work.
You show up with the code and it gets reviewed & discussed on how/why of the implementation.
The most talented entry level people I've worked with have struggled with criticism instead of grading (i.e "grade the code" through a bunch of test-cases is different from "why did you do this and explain").
For an entry level engineer, this removes a bunch of variance and also because it gets you another scenario which happens way often in actual work.
You show up with the code and it gets reviewed & discussed on how/why of the implementation.
The most talented entry level people I've worked with have struggled with criticism instead of grading (i.e "grade the code" through a bunch of test-cases is different from "why did you do this and explain").
I’m a senior engineer with an anxiety disorder that has trouble with white boarding. It’s honestly probably the most cruel environment possible for me because when I work I’m on the same team.
My social anxiety makes interviewing very difficult. I freeze up to a single-track-mind. Every spare bit of processing available is used up on guessing social situation around me.
I recently had an old boss try to bring me on to a new job with him, but wasn't able to get me through the interview phase, despite knowing I'm capable, because my interviews with the rest of the team went so bad.
I've been a software engineer over ten years. All of the positions that I have been offered have come from especially kind/empathetic teams, and the interviews usually focused more on conversation than quizzing.
It requires an unfortunate amount of rejection before coming across my more selective criteria, but I guess that is what it is.
I recently had an old boss try to bring me on to a new job with him, but wasn't able to get me through the interview phase, despite knowing I'm capable, because my interviews with the rest of the team went so bad.
I've been a software engineer over ten years. All of the positions that I have been offered have come from especially kind/empathetic teams, and the interviews usually focused more on conversation than quizzing.
It requires an unfortunate amount of rejection before coming across my more selective criteria, but I guess that is what it is.
I will put people on the white board, but never quiz them about random trivia, nor do I ask them to write pseudo-code. I have them draw the main components of some system they worked on and tell me a little about the role/technology on each component. Then I ask general, high level questions about some changes they claim to have made on some component. Whenever possible I have the recruiter inform the person that this will be the format of the discussion. This allows them to come prepared for the types of questions I'll ask and lets them guide the conversation towards something they feel comfortable talking about.
How do you think that would have been with regards to your anxiety?
How do you think that would have been with regards to your anxiety?
That sounds like relatively low-anxiety interview format.
Knowing the specific format, and subject if possible, ahead of time would be incredibly helpful for preparing mentally.
And speaking of projects that you've already done, vs speculating about something new, is a free confidence boost.
I've never understood the point of asking people questions about any topic they haven't specifically said they know something about. If you say you wrote a class/method to query a db, I might ask you high level questions like what is the communications mechanism between your software and the database (looking for anything that means network: socket, tcp/ip, etc.). Then if the language is java I might try to get you to tell me about using the try/catch/finally structure and why you'd do that. Basically, I don't care if you can regurgitate facts, I just want to dig deep enough to be convinced that you actually accomplished what you said you'd accomplished.
I definitely second this - coming in context free and having no real idea about how to prepare is a very difficult aspect when it comes to interview anxiety (the other is the social dynamic). Even a list of data structures and alg properties one would be expected to know to pass an interview would be helpful when the interview questions could be outside of previous domain scope (web developers implementing b trees from memory, for example). Proving that you can groc fundamental things should only be part of the interview and hopefully not even the most important part.
On the other hand, I’ve had take home assignments that were total wastes of time because all they really cared about was whether you coded in their particular OOP fashion. I made a functional demo which performed well but they neglected to mention that they were looking for OOP design paradigms. Mention should be given of current methods they use so that you don’t spend a week going down a path that won’t tell them how you will fit into the team.
On the other hand, I’ve had take home assignments that were total wastes of time because all they really cared about was whether you coded in their particular OOP fashion. I made a functional demo which performed well but they neglected to mention that they were looking for OOP design paradigms. Mention should be given of current methods they use so that you don’t spend a week going down a path that won’t tell them how you will fit into the team.
Our coding test is over Skype and a collaborative editor. Is that a scenario when you would freeze up? Genuinely curious as there were a few cases where it seemed that people were way nervous.
I personally would. Any situation where I'm being judged by strangers in real-time sets off social anxiety.
Some things that help me in that situation are when there's some light/friendly conversation preceding the test, or when there's a limited the us-vs-them size, like only one or two interviewers at a time.
Some things that help me in that situation are when there's some light/friendly conversation preceding the test, or when there's a limited the us-vs-them size, like only one or two interviewers at a time.
Yes very much so. The problem is that it’s such a high pressure environment, something that is the exact opposite of ideal thinking position. The focus is rarely on past accomplishments and a “bad day” of white boarding basically makes you look like an imposter. I feel like someone could get everything they needed to know about how well I code by drilling into specifics about what I’ve done.
I think there is a greater chance that I would freeze up in this situation than I would on the job with a new problem, or an executive breathing down my neck. I don't think I'd react that way consistently, but I know it's further down the discomfort continuum. In the test:
* I'm under pressure to impress, but also be my true self. I've had little contact with you so I'm not sure how to gauge all the feedback when I have increased sensitivity. * Sometimes don't have a real calibration for the level of the test. I've done the recommended prep, but it doesn't feel indicative of the test, or the interviewer inserts and additional level in before we've achieved the basics of the test. * What if I want to rough something out, but feel I'm exposing what I know would be a weakness from my pre-test assessment? I've just gone and made a hash of something I knew I needed to know.
I'd say "how you collaborate" would be the biggest factor. If you start out with some code and fix up something, have me fix up something, and progressively raise the difficulty, then I'm much less likely to freeze up. I don't believe I'm particularly unique, but I don't find many tests delivered in this manner. It's more a case of "here you go for your practical test, show me what you can do kiddo" which just demonstrates how little programmers know about assessment, interviewing, and psychology.
I'm going to state the following because I think the question was directed to one individual who identified as having a particular disorder. I don't believe I have any specific disorder that would cause me to freeze up. I'm an introvert, tested under Myers-Briggs as an INTP (if you consider that any value), and am OK in networking events. In jobs I've always tended to end up in regular conversations with skip-level managers etc.
* I'm under pressure to impress, but also be my true self. I've had little contact with you so I'm not sure how to gauge all the feedback when I have increased sensitivity. * Sometimes don't have a real calibration for the level of the test. I've done the recommended prep, but it doesn't feel indicative of the test, or the interviewer inserts and additional level in before we've achieved the basics of the test. * What if I want to rough something out, but feel I'm exposing what I know would be a weakness from my pre-test assessment? I've just gone and made a hash of something I knew I needed to know.
I'd say "how you collaborate" would be the biggest factor. If you start out with some code and fix up something, have me fix up something, and progressively raise the difficulty, then I'm much less likely to freeze up. I don't believe I'm particularly unique, but I don't find many tests delivered in this manner. It's more a case of "here you go for your practical test, show me what you can do kiddo" which just demonstrates how little programmers know about assessment, interviewing, and psychology.
I'm going to state the following because I think the question was directed to one individual who identified as having a particular disorder. I don't believe I have any specific disorder that would cause me to freeze up. I'm an introvert, tested under Myers-Briggs as an INTP (if you consider that any value), and am OK in networking events. In jobs I've always tended to end up in regular conversations with skip-level managers etc.
So here's a crazy idea. What if the "interviewer" doesn't know the answer, and the problem is designed to be big enough to require both of you to solve it? That puts you and them on an equal basis in a more realistic scenario. Plus it helps calibrate the results by generating data on how existing employees perform.
Problems: the existing employee ends with even more latitude to wreck the interview. You need to ensure that the interviewee is taking the lead. You probably also need to record the whole thing for objective review by someone else, which you'll have to disclose, which could make it awkward again.
Ed: I've also heard of processes where you put multiple interviewees on a task together. That has some similar potential benefits.
Problems: the existing employee ends with even more latitude to wreck the interview. You need to ensure that the interviewee is taking the lead. You probably also need to record the whole thing for objective review by someone else, which you'll have to disclose, which could make it awkward again.
Ed: I've also heard of processes where you put multiple interviewees on a task together. That has some similar potential benefits.
Sounds wonderful to me, but I think the “point” of the solo whiteboard experience is to reduce outside factors as much as possible. Of course that is really just a bit of a pipe dream since you’re hugely disadvantaging good coders with bad solo whiteboarding skills - pretending that interviews can be purely objective really does the process harm imo.
Sloppy handwriting and a not-100%-progressive coding style is my bane when it comes to whiteboard coding. For the last two senior level dev jobs I've gotten, I just brought a laptop with Visual Studio set up and told the interviewer I sucked at whiteboard coding and could I please do it in an IDE.
That’s definitely something I feel like I should try. I have failed interviews despite having optimal solutions because the way I went about it seemed haphazard (I was thinking in terms of 2 or 3 things at once and bouncing around).
This also opens space for people from different backgrounds (degrees).
In most cases you are not looking for the people with the most level of CS knowledge, but rather someone that can think logically, that learns/ramps fast and have at least some explanation, even if wrong, for his choices and is able to take the observations/review as a learning opportunity.
In most cases you are not looking for the people with the most level of CS knowledge, but rather someone that can think logically, that learns/ramps fast and have at least some explanation, even if wrong, for his choices and is able to take the observations/review as a learning opportunity.
How do you check whether they've done it themselves? Rely on the discussion?
Not OP, but we do home work assignment instead of white board coding as well.
While I can't be absolutely sure that they've done it themselves, a big part of the in person interview is based on the homework, asking them how they would extend it, add specific features, potential performance problems they see, etc.
Those answers are hard to fake if you didn't do it yourself
We had one case where a candidate used some open source code snippets in his solution, not declaring and removing the comments and license headers. Still it was very obvious in the difference of coding style.
While I can't be absolutely sure that they've done it themselves, a big part of the in person interview is based on the homework, asking them how they would extend it, add specific features, potential performance problems they see, etc.
Those answers are hard to fake if you didn't do it yourself
We had one case where a candidate used some open source code snippets in his solution, not declaring and removing the comments and license headers. Still it was very obvious in the difference of coding style.
I was pretty nervous at my interview; no test was mentioned until the day, at which time (after the rest of the interview) I was presented with two sheets of xml-style configuration code for the company's proprietary build system.
The task was super simple - basically just duplicating some of the configuration and renaming some values to create another job, but at the time I just froze and couldn't see it. I managed to get some sort of answer out, but wasn't happy with my response. They claimed that they were meant to give this to me in advance as a homework question and somewhere the ball got dropped, but I had a feeling they wanted it to be a more on the spot thing considering the problem was so simple.
Thankfully I had the wherewithal to ask if I could take the problem home with me after the interview. When I got home and looked at the problem with fresh eyes I facepalmed so hard. I wrote up the correct answer and emailed it over to the company recruiter, hoping she would pass it along to my interviewers, but didn't get my hopes up.
I guess she did, since I ended up getting the job. I still cringe a bit about how I just blanked on that stupid question. Thankfully I got ramped up really quickly once I actually started the job and had no problem figuring things out after that.
The task was super simple - basically just duplicating some of the configuration and renaming some values to create another job, but at the time I just froze and couldn't see it. I managed to get some sort of answer out, but wasn't happy with my response. They claimed that they were meant to give this to me in advance as a homework question and somewhere the ball got dropped, but I had a feeling they wanted it to be a more on the spot thing considering the problem was so simple.
Thankfully I had the wherewithal to ask if I could take the problem home with me after the interview. When I got home and looked at the problem with fresh eyes I facepalmed so hard. I wrote up the correct answer and emailed it over to the company recruiter, hoping she would pass it along to my interviewers, but didn't get my hopes up.
I guess she did, since I ended up getting the job. I still cringe a bit about how I just blanked on that stupid question. Thankfully I got ramped up really quickly once I actually started the job and had no problem figuring things out after that.
This is generally fine but “homework” is a terrible idea (trend?) because it isn’t looking from the perspective of the new hire:
1. Unemployment can be extremely stressful and there is a very good chance the person needs a job NOW. Every little thing you do to dilly-dally instead of making an offer is another hurdle that may send the applicant elsewhere. Act as soon as possible when you see a résumé you like (get on the phone, find out what you want to know, make a decision; if you bring them to interview and they’re struggling, give feedback immediately; don’t have a system where you wait 3 weeks to tell them yes/no).
2. Any effort you expect an applicant to spend will be greatly multiplied. That person is not talking only to you, they’re looking at LOTS of job postings!! If every potential employer starts assigning “homework”, it becomes a major hurdle and you can bet they’ll look a lot closer at an employer that might provide a paycheck in a couple weeks instead of “homework”.
3. You don’t need complex problems to understand if somebody can code. Let them use whatever language their résumé claims they know, give them something to do for a few minutes and see what happens. If they get stuck, give them a solid hint and again, see what happens. You’re trying to assess things like (a) if they appear to know what their résumé claims, (b) if they are the kind of person you can interact with to get somewhere on a problem.
1. Unemployment can be extremely stressful and there is a very good chance the person needs a job NOW. Every little thing you do to dilly-dally instead of making an offer is another hurdle that may send the applicant elsewhere. Act as soon as possible when you see a résumé you like (get on the phone, find out what you want to know, make a decision; if you bring them to interview and they’re struggling, give feedback immediately; don’t have a system where you wait 3 weeks to tell them yes/no).
2. Any effort you expect an applicant to spend will be greatly multiplied. That person is not talking only to you, they’re looking at LOTS of job postings!! If every potential employer starts assigning “homework”, it becomes a major hurdle and you can bet they’ll look a lot closer at an employer that might provide a paycheck in a couple weeks instead of “homework”.
3. You don’t need complex problems to understand if somebody can code. Let them use whatever language their résumé claims they know, give them something to do for a few minutes and see what happens. If they get stuck, give them a solid hint and again, see what happens. You’re trying to assess things like (a) if they appear to know what their résumé claims, (b) if they are the kind of person you can interact with to get somewhere on a problem.
I don't think you've thought this through carefully.
What do we think the median interval from "established intent to interview" to "offer" is at medium-to-large software companies in SFBA? If I had to guess, it's on the order of a month. Nobody's hiring homework process puts a dent in that timeline. On the other hand, when done well, "homework" makes it easier for engineering teams to make confident decisions, which expedites offers.
How much effort do you think it takes a typical candidate to get through an interview gauntlet at a tech company with more than (say) 50 employees? It's at least multiple full days worth of aggregate effort, almost always including one actual, on-site full day. To the extent that homework offsets on-site interviews, it reduces the amount of effort candidates need to put in.
What do we think the median interval from "established intent to interview" to "offer" is at medium-to-large software companies in SFBA? If I had to guess, it's on the order of a month. Nobody's hiring homework process puts a dent in that timeline. On the other hand, when done well, "homework" makes it easier for engineering teams to make confident decisions, which expedites offers.
How much effort do you think it takes a typical candidate to get through an interview gauntlet at a tech company with more than (say) 50 employees? It's at least multiple full days worth of aggregate effort, almost always including one actual, on-site full day. To the extent that homework offsets on-site interviews, it reduces the amount of effort candidates need to put in.
Is it your ignoring the fact that some good candidates will just look at the job add see the home work coding tests and think "fuck you pay me" and ignore the opportunity.
Who needs a full 1 day of interviews plus time for phone interviews for an entry level job.
Who needs a full 1 day of interviews plus time for phone interviews for an entry level job.
I probably don't need to be competing to hire the "fuck you I'm too good to do take-home problems" developers. Candidly: a significant fraction of those developers don't do take-home tests because they're extremely good at interviewing and... how shall we put this delicately... not so excellent at the skills required to solve straightforward programming problems. It's rational for them not to want to interview at take-home-test companies and rational for me not to want to hire them at all.
That's why we started doing take-homes: because we kept having the experience of interviewing people with impressive resumes and sterling references who did not deliver at the caliber of the rest of our team. But even though we started doing homework challenges as a way of screening out bozos who interviewed well, we quickly learned something much more important, which is that for every talented developer you'll talk to with a strong resume, there are many --- probably at least 10's --- of talented developers who are underemployed because they don't have strong resumes.
You also assume a job ad that basically says "We're the world's leading cat sharing company. Wanted: senior developer. Requirements: 15 years Python experience, must do homework problems". I don't blame you for assuming that because for some reason not a single recruiting team in SFBA seems to be able to write a decent job req, but that's not what our job reqs said, and we had zero problems.
Finally: what's a company that hires software developers at any rank with less than a day's worth of interviews? I think you'll have trouble naming one. Even if you can, I don't think you'll have rebutted my point about the amount of energy it takes to complete a typical interview-based process.
That's why we started doing take-homes: because we kept having the experience of interviewing people with impressive resumes and sterling references who did not deliver at the caliber of the rest of our team. But even though we started doing homework challenges as a way of screening out bozos who interviewed well, we quickly learned something much more important, which is that for every talented developer you'll talk to with a strong resume, there are many --- probably at least 10's --- of talented developers who are underemployed because they don't have strong resumes.
You also assume a job ad that basically says "We're the world's leading cat sharing company. Wanted: senior developer. Requirements: 15 years Python experience, must do homework problems". I don't blame you for assuming that because for some reason not a single recruiting team in SFBA seems to be able to write a decent job req, but that's not what our job reqs said, and we had zero problems.
Finally: what's a company that hires software developers at any rank with less than a day's worth of interviews? I think you'll have trouble naming one. Even if you can, I don't think you'll have rebutted my point about the amount of energy it takes to complete a typical interview-based process.
From what I was told when applying for an avowed job at HMGCC for a Job requiring DV (TS) clearance you seem to require as much if not more FTF interview time.
I'll go a bit further than you here. My experience with using homework exercises in interviewing is that they provide a better signal of both technical ability and technical level than anything else I've tried.
Also, I work for the company which published the article under discussion here, and did the homework exercise when I interviewed there. It was refreshing to be given something that was an actual open-ended real-world problem, to be solved in real-world conditions.
Also, I work for the company which published the article under discussion here, and did the homework exercise when I interviewed there. It was refreshing to be given something that was an actual open-ended real-world problem, to be solved in real-world conditions.
Why must a hiring position be so “confident” for an entry-level hire? Take a couple of weeks to decide, maybe even one.
You basically already know what an entry-level hire will know: they’ll remember most of what the last year of their degree taught them, and have a fuzzy memory of anything earlier. They’ll be about as good at problem-solving as you can see on a whiteboard. You’re not paying ultra-senior salaries out of the gate; you’re just getting what you pay for so bring in somebody promising, start them on a project and let them begin a career with you.
Like I said, put yourself in the “new hire” position. They should be treated better given their likely circumstances:
You have to survive, you need rent and food and whatever else. You have debts. You’re probably working some non-ideal job (or 2 jobs) in the meantime for enough cash, enjoying all the no-free-time-left that gives you. If you’re really lucky, only a handful of companies have given you some stupid assignment to do with your spare hours, and since it’s “homework” and you have “more time”, they probably dreamed up a more involved problem than the average whiteboard question would be. Then you feel extra pressure because you really want to get it right but in reality you probably ended up rushing the thing and making some silly mistakes that you won’t see.
Never mind that “homework” is just the latest example of unnecessarily-long-winded hiring practices. Once you add up every little thing that every company wants, you’ve spent a lot of time that you don’t really have. (Some want you to create a new web account and profile that redundantly re-specifies 90% of your résumé. Some require you to specify your résumé as a file, then they scan it and scatter it into the wrong fields and discard the rest, requiring you to fix it all. Some have other forms for you to fill out. Some want multiple phone interview half-hours of your time, multiple on-sites, etc. and still can’t get back to you in under 3 weeks. Now do that 10 or 20 more times and don’t miss your next shift at the coffee shop.)
You basically already know what an entry-level hire will know: they’ll remember most of what the last year of their degree taught them, and have a fuzzy memory of anything earlier. They’ll be about as good at problem-solving as you can see on a whiteboard. You’re not paying ultra-senior salaries out of the gate; you’re just getting what you pay for so bring in somebody promising, start them on a project and let them begin a career with you.
Like I said, put yourself in the “new hire” position. They should be treated better given their likely circumstances:
You have to survive, you need rent and food and whatever else. You have debts. You’re probably working some non-ideal job (or 2 jobs) in the meantime for enough cash, enjoying all the no-free-time-left that gives you. If you’re really lucky, only a handful of companies have given you some stupid assignment to do with your spare hours, and since it’s “homework” and you have “more time”, they probably dreamed up a more involved problem than the average whiteboard question would be. Then you feel extra pressure because you really want to get it right but in reality you probably ended up rushing the thing and making some silly mistakes that you won’t see.
Never mind that “homework” is just the latest example of unnecessarily-long-winded hiring practices. Once you add up every little thing that every company wants, you’ve spent a lot of time that you don’t really have. (Some want you to create a new web account and profile that redundantly re-specifies 90% of your résumé. Some require you to specify your résumé as a file, then they scan it and scatter it into the wrong fields and discard the rest, requiring you to fix it all. Some have other forms for you to fill out. Some want multiple phone interview half-hours of your time, multiple on-sites, etc. and still can’t get back to you in under 3 weeks. Now do that 10 or 20 more times and don’t miss your next shift at the coffee shop.)
It depends on how much effort we want to put into training entry level programmers. Different companies have different requirements. If we intend to train the programmer from the ground up, then we can probably look for programmers who are smart, resourceful and self-sufficient. We can train the rest, like good coding, etc.
If our expectations are higher, then what I look for is code sensibility. Do they cut and paste the code without thinking "hey, this is kind of dumb to do"? Do they see repeated code and think "this can be collapsed into something more succinct"? This is the hardest thing to train for, and the most time consuming from a mentor point of view, so if we have a high bar for entry level programmers, then code sensibility is one of the factors on top of the 3 above that I look for.
I've rejected interns that worked for me from a permanent position because after 4 months, they still had terrible code sensibility. I had to pour through their code with a fine tooth comb because it was riddled with tiny bugs, and it needed constant rewrites because they just didn't improve over the course of the internship. I've had other interns that just "got it" and wrote great code from the start, or only needed a few reviews and then "got it". Those are the ones that will be productive quickly and won't hamper the rest of the team with lengthy reviews.
If our expectations are higher, then what I look for is code sensibility. Do they cut and paste the code without thinking "hey, this is kind of dumb to do"? Do they see repeated code and think "this can be collapsed into something more succinct"? This is the hardest thing to train for, and the most time consuming from a mentor point of view, so if we have a high bar for entry level programmers, then code sensibility is one of the factors on top of the 3 above that I look for.
I've rejected interns that worked for me from a permanent position because after 4 months, they still had terrible code sensibility. I had to pour through their code with a fine tooth comb because it was riddled with tiny bugs, and it needed constant rewrites because they just didn't improve over the course of the internship. I've had other interns that just "got it" and wrote great code from the start, or only needed a few reviews and then "got it". Those are the ones that will be productive quickly and won't hamper the rest of the team with lengthy reviews.
We hire mostly junior developers. I give a simple whiteboard interview with 3 questions, and encourage them to solve it with mostly pseudo-code but have a few "extra points" things that I'm looking for. Absolutely avoided any kind of "trick" questions.
I like the homework idea. I don't know about "coming back" but maybe send it to them a couple days prior to the tech interview.
I like the homework idea. I don't know about "coming back" but maybe send it to them a couple days prior to the tech interview.
The one downside I've found with homework problems is that oftentimes the set of requirements (or "suggested features") doesn't line up with the amount of time you're expected to spend on it.
I've personally been in the situation of receiving a list of 10-12 required features, with a stack that the recruiters know I am unfamiliar with, and then been told to only spend four hours on it.
While this was an extreme case, I find that the unreasonable time expectation is not uncommon. That is quite frustrating as an applicant.
I've personally been in the situation of receiving a list of 10-12 required features, with a stack that the recruiters know I am unfamiliar with, and then been told to only spend four hours on it.
While this was an extreme case, I find that the unreasonable time expectation is not uncommon. That is quite frustrating as an applicant.
I love the "this shouldn't take more than an hour or two" assurances.
They might be looking for how good you are at time boxing
I have failed many time at interviewing at technical interviews. I have never failed any 'real' world job project/endeavor. I've built an entire org's codebase and this company is now selling products worldwide successfully. Design,coded,built,tested and deployed code in 4 languages , each element being critical to the company.
I am an experienced dev, but couldn't tell you my birthday in a technical interview. All the rest of the interview is fine, I only 'freeze' up at technical questions.
Sucks.
I am an experienced dev, but couldn't tell you my birthday in a technical interview. All the rest of the interview is fine, I only 'freeze' up at technical questions.
Sucks.
I cannot count how many interviews I've done where I've shown up with 10+ years of industry experience and been given an interview catered toward juniors. If anything there is too much emphasis on interviewing entry level software engineers. There are others out there with a proven track record.
I'm just underselling myself, shooting too low. At this point do I need to be applying for management?
I'm just underselling myself, shooting too low. At this point do I need to be applying for management?
It's tricky for interviewers because a lot of candidates claim to be "senior" yet can't even solve fizzbuzz on a whiteboard. Look for jobs with "Lead" or "Architect" in the title, but beyond that, expect that you'll have to go through some sort of stupid process as a bullshit screen.
Can confirm, have interviewed people for a senior position with a wonderful resume with 20+ years of experience, who seem to have never seen a computer before. Flip the bit endianess of a word was one of the problems, and I saw a guy who literally (supposedly) did embedded software for the space shuttle just grind for 45 minutes and somehow not make any progress. Like it was impressive. I would have expected a random walk to make more progress.
Do you mean a word by 8 bits? Or 16 or whatever the architecture defines as a word?
So 1000 gets flipped to 0001 ?
So 1000 gets flipped to 0001 ?
It was a uint32_t in this case (we do a bunch of ARM stuff mainly so without any other context, that's what 'word' means to me). And yeah, exactly, 1000 to 0001. We had an environment setup with a bunch of test cases and a stub function to fill in. When you come in, push the button, everything compiles, but most of the tests fail because the function is just
And like I said, this is for an experienced embedded software position. Ironically enough I care less about showing that you can code in junior positions. Those are more "does it seem like you can learn?"
uint32_t flip_bit_endianess(uint32_t word) {
return word;
}
And like I said, this is for an experienced embedded software position. Ironically enough I care less about showing that you can code in junior positions. Those are more "does it seem like you can learn?"
Is there an efficient solution to this? I'm relatively used to switching byte orders, but I've never encountered a situation where I needed to flip bit orders before. I could certainly do it with a bunch of shifts and ors, but that seems awfully inelegant for an interview question. (A quick search indicates that there isn't any instruction that makes this easier than bit twiddling, but I'm not sure if I'm missing something.)
Where would this be used in the real world?
Where would this be used in the real world?
> Is there an efficient solution to this?
To be real, I'm not even looking for a particularly efficient solution; just "can you make the tests all turn green with ors and shifts" is what I'm looking for.
There is a O(log(N)) N is the number of bits solution where you swap each pair of bits, and then swap pair of pairs, and then each pair of pair of pairs, and so on. I want to say that we clocked it to 15 cycles for a 32 bit word on a semimodern superscalar core that can do each half of each pair in parallel until you or them back together.
All of that being said, I'm really just looking for a for each bit, shift and or it into the right place, O(N) solution. It's mind boggling how many people with wonderful resumes can't do that in an interview time block. Hence why I don't have a lot of patience for the whole "well I'm senior and have a great resume, so you shouldn't have to make me do a stupid coding thing that's beneath me" mindset.
> Where would this be used in the real world?
Yeah the problem itself is a little contrived, but it's a good use of the sort of logic and putting the pieces in the right place at the right time sort of skills you use in embedded programming, and more so HDL, which we require of our software engineers too. And I have seen it in the real world a few times on a serial line that has the wrong bit endianess.
To be real, I'm not even looking for a particularly efficient solution; just "can you make the tests all turn green with ors and shifts" is what I'm looking for.
There is a O(log(N)) N is the number of bits solution where you swap each pair of bits, and then swap pair of pairs, and then each pair of pair of pairs, and so on. I want to say that we clocked it to 15 cycles for a 32 bit word on a semimodern superscalar core that can do each half of each pair in parallel until you or them back together.
All of that being said, I'm really just looking for a for each bit, shift and or it into the right place, O(N) solution. It's mind boggling how many people with wonderful resumes can't do that in an interview time block. Hence why I don't have a lot of patience for the whole "well I'm senior and have a great resume, so you shouldn't have to make me do a stupid coding thing that's beneath me" mindset.
> Where would this be used in the real world?
Yeah the problem itself is a little contrived, but it's a good use of the sort of logic and putting the pieces in the right place at the right time sort of skills you use in embedded programming, and more so HDL, which we require of our software engineers too. And I have seen it in the real world a few times on a serial line that has the wrong bit endianess.
I haven't used bitwise operators since college (approaching 8 years ago) but wanted to try this as a fun exercise during the NFL playoff game. Here's what I came up with -- mostly untested.
[edit] Note this is obviously for 16-bit integers.
int flip_endian_helper(int num, int i) {
int flip_endian(int num) {
[edit] Note this is obviously for 16-bit integers.
int flip_endian_helper(int num, int i) {
if (i == 16) {
return 0;
}
int shift = 15 - 2*i;
int workingBit = num & (1 << i);
int partial = shift > 0 ? workingBit << shift : workingBit >> (shift * (-1));
return partial | flip_endian(num, i + 1);int flip_endian(int num) {
return flip_endian_helper(num, 0);
}What do you mean catered towards juniors? I interview people a lot and part of the interview, junior or not, involves a short coding test. The reason it is done is because some people can skate along and make an impressive resume but there is nothing to validate that resume. How do I know someone did the work that they are claiming? Not everyone has a github, and even then it can take time to validate someone's work.
One thing I'd be interested in, but feel like I never see, is a post-post-mortem somewhere down the line, say a year or two later. This gets a little tricky because you don't want to air your company's dirty laundry or say negative things that could be easily traced to point to a specific person, but...
In theory, I like the approach they've taken to hiring associates, but at the end of the day, what really matters is: are the people they end up hiring successful at their jobs? It might take a year or more to really answer that question.
I don't feel like we're very good at this. As 'kinkrtyavimoodh points out in another comment, we see a lot of "how to interview engineer" type posts. But we never find out if, in the end, they were effective. This article claims that everything went really well because they had (or believed they had) such high-quality candidates that it was difficult to choose at the end, but they don't actually know that. They won't find this out after evaluating actual on-the-job performance. It might have been hard to pick from their candidate pool because their process failed to tease out flaws that are only going to become apparent later. We just don't know, and likely will never know.
In theory, I like the approach they've taken to hiring associates, but at the end of the day, what really matters is: are the people they end up hiring successful at their jobs? It might take a year or more to really answer that question.
I don't feel like we're very good at this. As 'kinkrtyavimoodh points out in another comment, we see a lot of "how to interview engineer" type posts. But we never find out if, in the end, they were effective. This article claims that everything went really well because they had (or believed they had) such high-quality candidates that it was difficult to choose at the end, but they don't actually know that. They won't find this out after evaluating actual on-the-job performance. It might have been hard to pick from their candidate pool because their process failed to tease out flaws that are only going to become apparent later. We just don't know, and likely will never know.
> Notably, “potential” is nowhere on the scale. Many posts talk about hiring entry level engineers on their “potential,” but the term is seldom defined. Most people use “potential” to describe a sense that, even though a candidate currently cannot be an autonomous engineer, someday in the future they could be. We did not attempt to predict the future for any of our candidates but rather evaluated them on what they presented through the interviews.
Well...
> Some of the qualities we looked for in our associates in particular were:
> Resilience: Learning on the job is hard and we assumed that the associates would make mistakes and struggle through difficult concepts. We needed people who could endure these struggles and bounce back ready for the next challenge.
> Willingness to learn and the initiative to do so: Clover would assist the Associates in their growth, and provide teachers and mentors to help along the way, but any incoming Associate would need to be responsible for their own growth.
> Humility: This is an important trait for all Clover engineers, but we paid special attention to it in our Associates. They would have to learn from those around them, be respectful of others, and be able to take difficult feedback with grace.
> ...
> Deciding the technical skills to evaluate was a long process. We expected to teach our Associates most of the technical skills they would need to do their job, but we couldn’t accept candidates that were completely blank slates.
It really sounds like this is measuring the candidate's "potential".
I would argue they defined "potential" as "resilience, willingness to learn, and humility", which they feel is an atypical definition.
(Personally, I think that's a decent set of criteria to look for in entry-level engineers.)
Well...
> Some of the qualities we looked for in our associates in particular were:
> Resilience: Learning on the job is hard and we assumed that the associates would make mistakes and struggle through difficult concepts. We needed people who could endure these struggles and bounce back ready for the next challenge.
> Willingness to learn and the initiative to do so: Clover would assist the Associates in their growth, and provide teachers and mentors to help along the way, but any incoming Associate would need to be responsible for their own growth.
> Humility: This is an important trait for all Clover engineers, but we paid special attention to it in our Associates. They would have to learn from those around them, be respectful of others, and be able to take difficult feedback with grace.
> ...
> Deciding the technical skills to evaluate was a long process. We expected to teach our Associates most of the technical skills they would need to do their job, but we couldn’t accept candidates that were completely blank slates.
It really sounds like this is measuring the candidate's "potential".
I would argue they defined "potential" as "resilience, willingness to learn, and humility", which they feel is an atypical definition.
(Personally, I think that's a decent set of criteria to look for in entry-level engineers.)
Resilience, willingness to learn, and humility can be the potential, but I would also say that they are the candidates attitude. Yes, I would say that attitude makes a big difference.
We give homework and it has been a successful approach. Even if it's a trivial assignment one can gain a lot of insight on the candidate based on the overall professionalism of their submission.
entry level is bar far the easiest to interview for.
with experienced people they always think they are at or near the top of the scale, but dont understand that you have a different scale entirely.
with a junior person, ask them to describe a project they worked on. if they 'i dunno, school stuff', then you're done.
if you can engage with them meaningfully about their school work, or if they've done personal projects, have developed any kind of insight, show a spark of excitement or creativity then you're done - hired.
just keep an eye on them for a while to make sure they're engaged and absorbing culture. invest a good amount of time in their success (mentorship, pair debugging, whiteboard talks). if they dont start contributing more than they are costing in time after a few months, find them another role or fire them.
with experienced people they always think they are at or near the top of the scale, but dont understand that you have a different scale entirely.
with a junior person, ask them to describe a project they worked on. if they 'i dunno, school stuff', then you're done.
if you can engage with them meaningfully about their school work, or if they've done personal projects, have developed any kind of insight, show a spark of excitement or creativity then you're done - hired.
just keep an eye on them for a while to make sure they're engaged and absorbing culture. invest a good amount of time in their success (mentorship, pair debugging, whiteboard talks). if they dont start contributing more than they are costing in time after a few months, find them another role or fire them.
I love your attitude, but when I got out of school, nobody gave a flying eff about personal projects. This was before Google Analytics, so I had to track IPs myself, but I think something like .1 percent of companies looked at my website--and that might have been a false positive.
Even the company that eventually hired me didn't look at my website.
Even the company that eventually hired me didn't look at my website.
if you've spent years programming and dont have anything to say about it, there isn't anything for me to go on. you obviously dont have any real interest in it, and you're below the bar where i'm willing to invest the time to see if you're teachable. what would you suggest instead? gpa? top school? test? those dont seem particularly relevant.
edit: oh, maybe i wasn't clear. school projects are fine grist, you just have to be able to say something interesting about them
edit: oh, maybe i wasn't clear. school projects are fine grist, you just have to be able to say something interesting about them
I'm not arguing with you. Just saying the attitude is (or was) rare in my experience.
I saw the title to this piece on the HN front page, and thought, "Wow, I'd like to see what a random tech company does with entry level engineers." Too bad this isn't a piece of writing that really discusses that in detail. There is no real actionable content here; it reads more like an HR post or marketing for Clover. A shame.
I would say, try to test the fundamentals. Do they understands how memory is accessed, have sense what kind of algorithm is used (though it would be too much to code in 45 minutes), ambitious, have general idea to solve a problem, can they see issues if things are not performant... Having a conversation, comfortable conversation is the key, imo.
For a general developer role that seems excessive not sure what you mean by algorithem here?
I would assume you'd be asking about how the address and data busses work which varies by cpu and architecture (von Neumann vs Harvard) presumably you'd be happy if some one explained how PIC memory access works ?
I would assume you'd be asking about how the address and data busses work which varies by cpu and architecture (von Neumann vs Harvard) presumably you'd be happy if some one explained how PIC memory access works ?
With algorithms, I mean if the problem demands some optimization, be it network flow, string compression or some kind of resource allocation (something like register allocation in compilers), one should have the vague idea which direction to pursue or approach. This requires somehow rigorous course-work in school. Assuming everyone takes standard CS machine architecture courses, having the breadth in multiple domain is, essential, imo for a well rounded engineer. Now knowing(or having no idea) how memory access, hierarchy etc works, they won't know when things just get stuck.
It'll be interesting to see how it works out. Without some kind of real-time test of facility with abstract symbolic reasoning, I would expect them to get burned. It's way too easy for people to fool themselves and others about that kind of capability.
If they asked the subjects to modify the homework slightly during the debriefing, that might be enough.
If they asked the subjects to modify the homework slightly during the debriefing, that might be enough.
You with your sensible guidelines, your calibrated metrics, your well-defined endpoints. Where's all the VOODOO!??! Where's the WITCHCRAFT!??!?!
[deleted]
I have only one modest proposal: No. More. Whiteboard interviews!!
This is then done for every possible combination of approaches (homework questions, whiteboard interview, pair programming, fizzbuzz questions, algorithmic questions, work-history questions, career-progression questions, mini-project, magical incantations, and so on...)
It may sound cynical but at this point it feels like no one has any new ideas about it and the same content keeps getting recycled in the form of yet another article and yet another comments section.
Half of these articles are from company blogs which exist largely as just a marketing tool.
Since there is no actual new thought, everyone just cargo-cults the Google approach.