Refactoring Back End Engineering Hiring at Slack(slack.engineering)
slack.engineering
Refactoring Back End Engineering Hiring at Slack
https://slack.engineering/refactoring-backend-engineering-hiring-at-slack-b53b1e0e7a3c
54 comments
There is one thing mentioned there that I've never considered for some reason (and it's quite obvious in retrospect): Giving a candidate a code review exercise instead of (or in addition to) a code writing one. A code review can reveal a lot of how a candidate pays attention to details, looks at (and deals with) other people's code, reads code etc.
Agreed, if hiring processes involved less questions about CS/algorithms, and more day-to-day skills like "use git to do <x>", "how would you unit test this?", or "given these use-cases, how would you design the API?", a lot of places would be hiring better engineers and building better products.
I don't think you build better products if you hire someone who knows how to use git. It doesn't fit in with the rest of the skills you're listing.
I agree that testing for knowledge of specific tools is probably not what you want, but I find that among people who do have significant experience with git, ability to use it effectively seems to correlate pretty strongly with overall programming ability and productivity.
I mainly have experience with git, but I assume this generalizes to other version control systems. I also think that version control is a pretty fundamental skill for software engineers (comparable to unit testing), so I think substituting that for "use a version control system to do x" would fit in with with other skills.
I mainly have experience with git, but I assume this generalizes to other version control systems. I also think that version control is a pretty fundamental skill for software engineers (comparable to unit testing), so I think substituting that for "use a version control system to do x" would fit in with with other skills.
I would disagree when it's a clear requirement of the work to be done (and often tossed haphazardly onto a resume). There can be some real time sinks that are not towards making someone productive in a code base as you have to stop and teach them how to think about Git. There's also the fun of having to use advanced Git magic to untangle their PR into something sensible because they've done odd merges along the way where they should have rebased their commits. People that still are from centralized source control only world still exist and getting them to work Git well is a skill that's worth interviewing/filtering over if it's a day 0 expectation.
https://news.ycombinator.com/item?id=12018965
Generated a few comments but not many. I put in by thoughts on why code reviews are so much more better at gauging expertise. I have since adopted that and found that it led to much more productive interviews. Even candidates that do not make it thru' appreciate the fact that they were treated as peers rather than being 'questioned'.
I'd like to also add something else that I've adopted -- the solve-at-home question, if we do go with one, is never just a lazy spec. document. It is an unfinished bit of code with tests etc. This way the candidate is not intimidated/overwhelmed/have analysis-paralysis and it feels more natural. Again, insights from submission to those are better than from-scratch assignments IMO.
Generated a few comments but not many. I put in by thoughts on why code reviews are so much more better at gauging expertise. I have since adopted that and found that it led to much more productive interviews. Even candidates that do not make it thru' appreciate the fact that they were treated as peers rather than being 'questioned'.
I'd like to also add something else that I've adopted -- the solve-at-home question, if we do go with one, is never just a lazy spec. document. It is an unfinished bit of code with tests etc. This way the candidate is not intimidated/overwhelmed/have analysis-paralysis and it feels more natural. Again, insights from submission to those are better than from-scratch assignments IMO.
Can vouch for its efficacy as an indicator for the traits you might want.
I use a small application that demonstrates some anti patterns / questionable design choices and ask candidates to talk me through their thoughts.
I use a small application that demonstrates some anti patterns / questionable design choices and ask candidates to talk me through their thoughts.
There's been a lot of threads like this recently that discuss changing the hiring process to more closely resemble day to day work and working with an actual codebase.
My question is how does this work for new grads and people that haven't worked in the industry before? Are they using these new techniques to hire recent graduates/people that haven't worked as a software dev before? Does this kind of process adversely effect those who haven't worked as a software dev before?
I'm not trying to say that algorithmic/leetcode type problems are a better solution, but rather questioning whether this form of interviewing has some kind of bias.
My question is how does this work for new grads and people that haven't worked in the industry before? Are they using these new techniques to hire recent graduates/people that haven't worked as a software dev before? Does this kind of process adversely effect those who haven't worked as a software dev before?
I'm not trying to say that algorithmic/leetcode type problems are a better solution, but rather questioning whether this form of interviewing has some kind of bias.
Currently, the existing form also has a bias - towards CS and top CS universities. Yet I keep working with great self-taught engineers from a variety of disciplines, some of whom don't even have degrees. Maybe a shift in hiring wouldn't be a bad thing.
As for how, you could focus more on logical thinking, user experience considerations, technical writing, or other, softer skills.
For a lot of grads, we already have to re-teach them the language, as universities don't seem to give marks for consistent style/using linters/unittesting/best practices/comments/writing documentation etc. In the time it takes to break these bad habits, you can definitely teach somebody who's technical and done a basic programming course a lot of things.
As for how, you could focus more on logical thinking, user experience considerations, technical writing, or other, softer skills.
For a lot of grads, we already have to re-teach them the language, as universities don't seem to give marks for consistent style/using linters/unittesting/best practices/comments/writing documentation etc. In the time it takes to break these bad habits, you can definitely teach somebody who's technical and done a basic programming course a lot of things.
> questioning whether this form of interviewing has some kind of bias.
In the case above—the bias is toward experience. I don't think that's a bad thing.
Context is important, however. I haven't read any Slack postings, but if they're asking for tangible experience this kind of gatekeeping makes more sense than textbook questions. If they're hiring for an entry-level role then a different approach might be merited.
In the case above—the bias is toward experience. I don't think that's a bad thing.
Context is important, however. I haven't read any Slack postings, but if they're asking for tangible experience this kind of gatekeeping makes more sense than textbook questions. If they're hiring for an entry-level role then a different approach might be merited.
Code review assignments are so much better than coding assignments that for me it's a great sign that your hiring sucks if you haven't switched to them.
If you do a large coding assignment you have to leave the time frame rather open ended to assist candidates that have limited free time. Instead this forces every candidate to spend as much time as possible to get noticed, having an opposite effect. There is no obvious solution to that apart from doing short timed assignments.
But what is even more important, a code review can be tailored to provide the same information on technical expertise, has a more natural effort limit and more importantly, it actually tells you about how that person will act as a coworker.
Will they even explain their work? Will they provide reasonable feedback, and will their way of explaining their thoughts in written word be detailed enough to guide people through?
Not only are code reviews, for myself, essentially part of the documentation of the code base itself, but they are also a very important communication channel. And how one communicates with others through it is a great proxy about how they will handle the rest of their communication.
I've never worked with an engineer that was considerate, kind and detailed (with a focus on being understood) in their review comments or presentation, who turned out to be an asshole when interacting with through other channels.
If you do a large coding assignment you have to leave the time frame rather open ended to assist candidates that have limited free time. Instead this forces every candidate to spend as much time as possible to get noticed, having an opposite effect. There is no obvious solution to that apart from doing short timed assignments.
But what is even more important, a code review can be tailored to provide the same information on technical expertise, has a more natural effort limit and more importantly, it actually tells you about how that person will act as a coworker.
Will they even explain their work? Will they provide reasonable feedback, and will their way of explaining their thoughts in written word be detailed enough to guide people through?
Not only are code reviews, for myself, essentially part of the documentation of the code base itself, but they are also a very important communication channel. And how one communicates with others through it is a great proxy about how they will handle the rest of their communication.
I've never worked with an engineer that was considerate, kind and detailed (with a focus on being understood) in their review comments or presentation, who turned out to be an asshole when interacting with through other channels.
I thought this was a job offer for an engineer whose main job would be refactoring back-end code.
Turns out it's just another article about how to interview people.
Turns out it's just another article about how to interview people.
That would be an interesting idea, having a dedicated refactoring person on a team. I wonder how well that would help with tech debt...
I'm working through reading the DevOps Handbook (the companion successor to The Phoenix Project), and in it is an entire chapter dedicated to the necessity of dedicating either the time (measured in 20%) or dedicating a small group of people who's express and specific charge is refactoring and paying down technical debt.
They pointed to Nordstrom as an example of an enterprise organization who did this as a part of their internal devops transformation, and it worked wonders at the time when it was needed.
This team didn't tackle new user stories for feature additions or enhancements, they didn't work on existing tickets. Their role was singularly focused on identifying and making payments on technical debt and operational enhancements.
Not every team can afford to focus an entire group of people to the task of paying down technical debt, but what I did take from it is that every team can't afford to not dedicate some time out of the week or month making critical fixes where necessary.
They pointed to Nordstrom as an example of an enterprise organization who did this as a part of their internal devops transformation, and it worked wonders at the time when it was needed.
This team didn't tackle new user stories for feature additions or enhancements, they didn't work on existing tickets. Their role was singularly focused on identifying and making payments on technical debt and operational enhancements.
Not every team can afford to focus an entire group of people to the task of paying down technical debt, but what I did take from it is that every team can't afford to not dedicate some time out of the week or month making critical fixes where necessary.
lol, I thought the exact same thing
chief among them is the high cost of hiring the wrong person, or missing out on the right one.
As a hiring manager, I've never cargo culted this particular view. In fact I think it's completely misguided. I always work in at-will states, it's trivially easy to let someone go. I also personally have no problem doing it.
I generally will hire based on conversation alone (no coding tests). If you ask someone detailed questions about work a candidate has done you can get a really good idea if they know what they're talking about. I know people say that they've interviewed people that can talk a good game but can't code a for loop once in the door, but I've interviewed hundreds of people and I can't say this is often the case.
Easy in, easy out. You know what though? There's very rarely ever the out part. If I can look at your Github, like what I see and you can back that up in a conversation, you're hired. That has worked out really well for me and reduces a lot of friction and wasted time (mine and other engineers') during the hiring process.
The moral of my story is that just because Google or some other tech giant espouses a theory like "false positives are way more costly than false negatives", it doesn't mean it works for all organizations or that it's even true at all.
As a hiring manager, I've never cargo culted this particular view. In fact I think it's completely misguided. I always work in at-will states, it's trivially easy to let someone go. I also personally have no problem doing it.
I generally will hire based on conversation alone (no coding tests). If you ask someone detailed questions about work a candidate has done you can get a really good idea if they know what they're talking about. I know people say that they've interviewed people that can talk a good game but can't code a for loop once in the door, but I've interviewed hundreds of people and I can't say this is often the case.
Easy in, easy out. You know what though? There's very rarely ever the out part. If I can look at your Github, like what I see and you can back that up in a conversation, you're hired. That has worked out really well for me and reduces a lot of friction and wasted time (mine and other engineers') during the hiring process.
The moral of my story is that just because Google or some other tech giant espouses a theory like "false positives are way more costly than false negatives", it doesn't mean it works for all organizations or that it's even true at all.
I've worked at places where the managers hem and haw over firing someone even going so far as to use passive-aggression to make them leave on their own. In environments like that, the cost of hiring the wrong person is huge. It could take a couple of quarters to shake the dead weight and the entire team suffers.
I've also worked with managers who have no problem being the hatchet man and it's great. They sniff out the losers, put them on a plan, and they're either out the door soon after or stick around to get fired.
I would rather managers teach each other to wield the hatchet than try to get seven steps ahead of a dud hire during the interview process. "Release early, release often."
I've also worked with managers who have no problem being the hatchet man and it's great. They sniff out the losers, put them on a plan, and they're either out the door soon after or stick around to get fired.
I would rather managers teach each other to wield the hatchet than try to get seven steps ahead of a dud hire during the interview process. "Release early, release often."
I totally agree. I think so much of the "it's really hard to get rid of bad people" story comes from weak management. Firing people is part of the job, you need to be comfortable doing it.
Interviewing people is extremely disruptive to an engineer's time and in a large company it could potentially happen a lot. I think that is the metric that should be optimized for, not minimizing firing work for the hiring manager.
Interviewing people is extremely disruptive to an engineer's time and in a large company it could potentially happen a lot. I think that is the metric that should be optimized for, not minimizing firing work for the hiring manager.
I wonder about this - am I doing a candidate a disservice by hiring them without knowing that there is a non-trivial chance that they'll be fired within 1-2 months? If someone doesn't already have a job, that's fine, but someone might quit a situation where they are suited for one in which they aren't. Or, if someone is a junior developer, they've just spent 2 months before being let go, and has to explain the situation to the next place.
I care about the people I hire as humans and don't want to put them in a bad situation.
I care about the people I hire as humans and don't want to put them in a bad situation.
I wouldn't say there's a non-trivial chance that they'll be fired in 1-2 months. Being able to effectively screen people is another technical managerial skill. There is a very, very small percentage of candidates that make it through the interview and later turn out to be duds.
I think there are two issues at play:
1. Ineffective managers are terrified of firing people.
2. Ineffective managers rightly or wrongly think they can't screen people without code tests, white boarding...
Like I said in my original comment I look at their Github or any other work portfolio they provide and have a deep discussion about it with them.
I think there are two issues at play:
1. Ineffective managers are terrified of firing people.
2. Ineffective managers rightly or wrongly think they can't screen people without code tests, white boarding...
Like I said in my original comment I look at their Github or any other work portfolio they provide and have a deep discussion about it with them.
What about the impact on the person for the false positive? Taking a new position can be hugely disruptive for someone, so making sure they are a fit (both directions) seems like the respectful thing to do.
Like I said, it rarely actually happens but when it does it's because the person really was incompetent (I don't worry too much about "culture fit" and other vaporous qualities).
As a manager, it's my responsibility to do what's best for the business.
As a manager, it's my responsibility to do what's best for the business.
> I always work in at-will states, it's trivially easy to let someone go. I also personally have no problem doing it.
Most managers are not as cold-blooded as you are claiming to be.
Most managers are not as cold-blooded as you are claiming to be.
It's not about being cold blooded. It's a function of the job and what's best for the team. Better to get them out early than let them bring everyone down then let them go.
I don't disagree with your rationale. I once saw the statement that in order for a company to succeed, they either have to figure out how to hire well or how to fire well. If you do one well enough, you don't have to worry as much about optimizing the other. Most companies just prefer to focus more on hiring well because firing people is unpleasant.
And what about what’s best for the person being fired? Or are they just a cog in the machine, unworthy of empathy or decent treatment?
If someone can't perform the job, the only option is to let them go. As I said in my original post, it rarely happens because it's actually not that difficult to vet someone without forcing them to write tons of code.
In order of importance a manager should put the company first, then team, then individual employees, then themselves. Poor performers hurt everyone on that list.
In order of importance a manager should put the company first, then team, then individual employees, then themselves. Poor performers hurt everyone on that list.
If you hire someone who is currently employed and then fire them, you are doing very real damage to their finances and career. Empathy for employees is a valid reason to err on the side of avoiding bad hires over easy in/easy out.
I disagree. By optimizing to protect poor performers, you jeopardize all employees. It's more important that the company succeed and that people who are good at their job get to work unencumbered.
Framing the decision to fire someone as a trolley problem (cf. https://en.wikipedia.org/wiki/Trolley_problem) doesn't make you look any less cold-blooded :)
Have you ever been a manager? Again, it's not being cold blooded, it's one of the main responsibilities of the job. Being a leader isn't easy and you often have to make tough decisions and sacrifice in the name of the team. This has been well known throughout the history of leadership.
It shouldn't be looked at that way and it's not black and white. Ideally they're treated like people, not cogs, and the manager and company does what it can to either get the person the skills they need to succeed, move them to a role they can succeed in, or give them a path out in a reasonable amount of time.
Most of the time the under-performer will be happier after any of these scenarios than they would knowing they're under-performing in their current role. It just takes a bit of time and maturity to realize.
Most of the time the under-performer will be happier after any of these scenarios than they would knowing they're under-performing in their current role. It just takes a bit of time and maturity to realize.
It sounds like you're working in some specific (and enviable) circumstance where people can get productive right away, and where job success is fully defined by meeting your expectations. False positives get more expensive when there is training or provisioning effort required to get someone onboarded, or when you expect your teams to satisfy multiple stakeholders or to grow beyond initial job requirements.
When you start working with someone day to day you can very quickly find out if they are good or not. You don't have to fully get them up to speed and have them learn all of the domain knowledge of your company. Start small and evaluate quickly. Certainly you can learn more when they start working than you can in the interview process.
> when you expect your teams to satisfy multiple stakeholders or to grow beyond initial job requirements
This honestly sounds like bad management.
> when you expect your teams to satisfy multiple stakeholders or to grow beyond initial job requirements
This honestly sounds like bad management.
Wat? Bad management to consider how your devs will interact with other roles (PMs, customer support, other dev teams)?
I can't figure out if your devs just don't do that, or if you think you can speak for everyone's standards after one conversation with a candidate.
I can't figure out if your devs just don't do that, or if you think you can speak for everyone's standards after one conversation with a candidate.
Those people should of course be part of the interview process if the developer will be interacting with them. I don't see how that makes a bad hire any more expensive though. Maybe I'm misunderstanding what you're trying to say.
Nah, looks like I misunderstood your "hire based on conversation alone". I thought you meant conversation with just you. My bad.
I've offered to give many interviewers access to my private github repos so they could review my code and see how I've worked. I have ~5 relatively large projects, full tested, where I was the sole, full-stack dev. 1 out of 20+ took me up on the offer and I work there now.
>> I generally will hire based on conversation alone (no coding tests)
>> If I can look at your Github, like what I see and you can back that up in a conversation, you're hired.
Wish more hiring managers followed this line of thinking
>> If I can look at your Github, like what I see and you can back that up in a conversation, you're hired.
Wish more hiring managers followed this line of thinking
So true. This argument is simply bullshit in the US. It's just another justification for ridiculous hiring tests and other stupidity that weeds out a lot of good candidates. You can hire someone today and fire them next week. It happens all the time and there isn't a high cost other than the compensation for a week. You can, although a really shitty move, even delay someone's benefits for a few weeks or even months while they go through a trial period, often known as probation. It's shitty, but if costs are that important, it can be easily done. If you can't tell if a candidate is good after a few weeks or months of them working with you, that is a problem with your company not the candidate.
I'm not taking a job that delays my benefits for any period, sorry, that's a deal breaker. I think most good candidates would feel the same way.
It would be interesting to see a follow up a year or so from now about how candidates that went through this vs. the previous process are performing.
> Our take-home exercise, while loved by many, was also time-consuming. Its open-ended qualities meant that candidates, wanting to show off their best work, could end up spending many more hours of their own time to complete it than we had expected.
> This was often a barrier for candidates who couldn’t afford to invest the time needed to complete the exercise to our desired level of quality.
Isn't there a conflict here? Why are you setting the bar for your own desired level of quality so high that it isn't possible to meet it in the amount of time you expect people to take?
> This was often a barrier for candidates who couldn’t afford to invest the time needed to complete the exercise to our desired level of quality.
Isn't there a conflict here? Why are you setting the bar for your own desired level of quality so high that it isn't possible to meet it in the amount of time you expect people to take?
Isn't that their point? The article is acknowledging that they had set their expectations too high and so candidates were failing to meet one of the two goals, which is part of why they redesigned the process.
But that problem doesn't require a process redesign. You can just lower your expectations.
that github api is really interesting. does github never run GC on repos or do they flag the commits you create from the API so they are never GC'd? I guess in a normal workflow and normal git gc options commits that are unrooted would not be deleted because git avoids deleting recent data during GC. however, there doesn't seem to be any warning in the API that your commits might disappear if you don't root them.
It gratifys me in some weird way to know that companies stress out about the recruitment process at least as much as I do
Would love to work at one of these magical companies that HN posters work for that have zero internal domain knowledge required, zero runway time to the engineers being maximally productive, and zero friction teams where adding or removing any given engineer has zero cost for anyone.
The term "signal" is ill-defined in this piece. Are there other posts where they go into this in detail?
The decrease in the hard time-to-hire metric was great to see. This is a real problem with some top-tier shops.
The decrease in the hard time-to-hire metric was great to see. This is a real problem with some top-tier shops.
"a high degree of craftsmanship"? I know what this means but still, how exactly do you define that and not be biased by how the candidate presents him/herself.
Anyone know what kind of review questions work well ?
Any rule of thumbs, best practices to create these questions?
Why do they work better ?
Why do they work better ?
I was waiting for links to all the awesome [open source] code Slack talks about in this post. I was sad.
> it would have taken a year to fill our existing open headcount
Maybe open an office in a different location or hire remote?
Maybe open an office in a different location or hire remote?