How to Hire Great Engineers(autoiterative.com)
autoiterative.com
How to Hire Great Engineers
https://autoiterative.com/blog/posts/how-to-hire-great-engineers/
172 comments
Everyone wants something for nothing. IE hire great engineer who got all their experience and made their mistakes somewhere else.
Why not roll your own? I am constantly amazed by companies moaning "We have a skills shortage! We have to get people from overseas."
Sure, get people from overseas if that works. But bloody train people to do what you need! For the FAANG, train hundreds of them!
Sure, you may want penalties for people leaving early after training or bonus for hanging around after training (like you get bonus stock 1-3 years after the training) but HR people, get creative and do your damn job! Your job isn't just hire&fire.
Why not roll your own? I am constantly amazed by companies moaning "We have a skills shortage! We have to get people from overseas."
Sure, get people from overseas if that works. But bloody train people to do what you need! For the FAANG, train hundreds of them!
Sure, you may want penalties for people leaving early after training or bonus for hanging around after training (like you get bonus stock 1-3 years after the training) but HR people, get creative and do your damn job! Your job isn't just hire&fire.
I completely agree. The approach of hiring young and training them is particularly overlooked these days but is the best approach as you end up with a blend of experience over time.
Is it really overlooked? Pretty much every big tech company I’m aware of does a good amount of hiring through the intern->new grad pipeline
Big companies have the resources to do that sort of thing. I work at a small startup, we need some experienced people because its just extra overhead getting juniors up to speed and we are already under resourced. I don't think anyone has anything against training juniors in principle, but it takes time and effort.
In order to train people, you need people who are qualified to train them :)
Hire people who use boring technology and build systems that do things simply and directly.
This is great engineering though. Isn't the trick to find engineers who don't build nests for themselves and create strategic technology problems for your company, especially if your core business ins't tech.
This is great engineering though. Isn't the trick to find engineers who don't build nests for themselves and create strategic technology problems for your company, especially if your core business ins't tech.
I disagree. Great engineering is when you propose and implement (or lead a team implementing) a complex, novel and valuable piece of engineering. Linus is a great engineer, key people working on database systems are great engineers etc. What you're describing is "just" competent engineering.
Most engineering problems aren't up the engineers to invent or propose. What you're describing sound more like product design. A lot of companies face a problem such as "We need a bridge an we have X money for it". The engineering is in meeting spec with those constraints. The great engineering is doing that and not baking in a hidden cost of support, maintenance etc.
In civil engineering world, great engineering would be proposing a novel design of a bridge (for example, a first large bridge built out of steel) and delivering it. Of course, most situations don't require novel designs, you just need competent implementation of something that is an industry standard.
In software world, you can also satisfy 99.9% of cases with existing approaches. You mostly don't need great engineers for that.
In software world, you can also satisfy 99.9% of cases with existing approaches. You mostly don't need great engineers for that.
> In software world, you can also satisfy 99.9% of cases with existing approaches. You mostly don't need great engineers for that.
Unlike other engineering disciplines, software has a constant pressure for pseudo novelty and fad adoption. One dimension of "greatness" is resistance to those temptations and pragmatism in the face of constant change. In that regard, lower quality engineering talent often correlates with increased tech churn and defects, even given a simple business domain.
Tl;Dr engineering excellence doesn't only imply innovativeness, it also implies discipline.
Unlike other engineering disciplines, software has a constant pressure for pseudo novelty and fad adoption. One dimension of "greatness" is resistance to those temptations and pragmatism in the face of constant change. In that regard, lower quality engineering talent often correlates with increased tech churn and defects, even given a simple business domain.
Tl;Dr engineering excellence doesn't only imply innovativeness, it also implies discipline.
In other words: Systems that do things simply and directly are great, and often exciting, technology.
I don't like this negativity. Pay and position doesn't necessarily reflect competence. I don't buy that the smartest engineers in the world are all working for big companies.
That's because people are obsessed with this idea of a one-size-fits-all great engineer.
A generally great engineer to a startup is somebody who can ship fast, make good enough decisions and trade offs, can think on their feet, and is a jack of all trades.
A generally great engineer to a big company is somebody who can work well with others, can follow process, can work within huge code bases and abstractions, and follow architectural best practices.
These two do not necessarily overlap.
A generally great engineer to a startup is somebody who can ship fast, make good enough decisions and trade offs, can think on their feet, and is a jack of all trades.
A generally great engineer to a big company is somebody who can work well with others, can follow process, can work within huge code bases and abstractions, and follow architectural best practices.
These two do not necessarily overlap.
They don't, but...
Google has brand recognition, and an enormous recruiting pipeline so they can hire one in 50 or 100 candidates.
They are better able to select for great candidates because they have more data and much more sophisticated system for identifying talent.
They pay very high salaries so they can retain top talent.
The average candidate that applies to Google is going to be a better than the average candidate that applies to a small/medium shop.
A small medium shop that tried this would be chronically and severely understaffed with a high turn over and would probably still fail at finding the best candidates.
If you are a small or medium sized shop and really need top level talent the best thing to do is probably just hire googlers and netflixers and pay them a market rate of 400-500k a year.
But for most products it's not worth it because the end product wouldn't be materially different enough from an average developer building it to justify the cost.
Also most small to medium shops could improve engineering quality by taking the Netflix approach which is getting better at firing people, and compensate them for the lack of job security with cash.
Google has brand recognition, and an enormous recruiting pipeline so they can hire one in 50 or 100 candidates.
They are better able to select for great candidates because they have more data and much more sophisticated system for identifying talent.
They pay very high salaries so they can retain top talent.
The average candidate that applies to Google is going to be a better than the average candidate that applies to a small/medium shop.
A small medium shop that tried this would be chronically and severely understaffed with a high turn over and would probably still fail at finding the best candidates.
If you are a small or medium sized shop and really need top level talent the best thing to do is probably just hire googlers and netflixers and pay them a market rate of 400-500k a year.
But for most products it's not worth it because the end product wouldn't be materially different enough from an average developer building it to justify the cost.
Also most small to medium shops could improve engineering quality by taking the Netflix approach which is getting better at firing people, and compensate them for the lack of job security with cash.
Optimizing for the rarest of scenarios - that you're going to hire the bestest engineers in all the worlds - is silly and foolish.
It's not negative. It's pragmatic. If you even happen to just encounter one of those people (they're rare by definition) then do what you can to hire them, but the truth is that the vast majority of companies producing software wouldn't even know what to do with them if they could attract them.
Stable, predictable, performance has a ton of value too - and you get a lot of that from 'regular' developers.
It's not negative. It's pragmatic. If you even happen to just encounter one of those people (they're rare by definition) then do what you can to hire them, but the truth is that the vast majority of companies producing software wouldn't even know what to do with them if they could attract them.
Stable, predictable, performance has a ton of value too - and you get a lot of that from 'regular' developers.
I worked in unicorns and small companies over a couple of of dozen of years.
Best people do thing that matter.
There are fewer things that matter in big companies.
Best people do thing that matter.
There are fewer things that matter in big companies.
I think most of this usually boils down to "I feel that I am a great engineer, so all great engineers must have the same career path that I have"
I keep hearing this trope of big companies vs small companies and it's flat out wrong.
Best engineers are working at high growth organizations. Pay and position do reflect competence if you are looking at that demographic.
Best engineers are working at high growth organizations. Pay and position do reflect competence if you are looking at that demographic.
I now work for a 100 person org. I came from a place with high standards and 1000+ people. I’m not happy. Broken window theory applies.
We need to make money but it’s my responsibility to ensure we do it safely. Incredibly difficult with adderall addicted developers who shine bright in politics.
We need to make money but it’s my responsibility to ensure we do it safely. Incredibly difficult with adderall addicted developers who shine bright in politics.
That’s what most big companies already do and why it burns out experiences developers. The degree of settling for competence is a sliding scale that sets horrid expectations. That’s so many developers can’t tell the difference between expert beginners and actual experts.
I don't know about this. Most experienced developers I know of have family, and they generally value their personal time. They don't mind doing boring software. And even if they are bored, I don't think it burns them out.
What I do know is that company processes burns out all sorts of people - experienced or not. From stack ranking, agile micromanagement, unrealistic deadlines, office politics, bad bosses... These things are prominent in nearly every company, especially in top companies like Google.
I agree with the parent post, even if you by chance somehow hired fantastic 10x developers to your team, the environment in which those developers will be working in will probably burn them out quicker than just working on 'boring' software.
What I do know is that company processes burns out all sorts of people - experienced or not. From stack ranking, agile micromanagement, unrealistic deadlines, office politics, bad bosses... These things are prominent in nearly every company, especially in top companies like Google.
I agree with the parent post, even if you by chance somehow hired fantastic 10x developers to your team, the environment in which those developers will be working in will probably burn them out quicker than just working on 'boring' software.
I see what you're getting at with the first point but what about the second? It seems to me the "quality of engineering culture" at a company doesn't need to correlate with company size.
A great engineer in a small/medium company is not the same thing as a great engineer in a big company.
Agreed. I think that being a great engineer in a big company has more to do with being at least moderately competent and being able to build political capital and drive change in a bureaucracy.
Many of the higher-ranking engineers I've seen at big companies weren't necessarily brilliant, but they were all well regarded.
Many of the higher-ranking engineers I've seen at big companies weren't necessarily brilliant, but they were all well regarded.
You can be a genius savant engineer and still crash and burn at large companies because you can't politic. I've seen it.
That's how small/medium companies stay small/medium for the most part though. I don't think the solution to this problem is "give up" =/
Most companies will be small / medium. There isn't any shame in that. For many people the end goal of being employed / building a company is "keep a roof over my head and put food on the table". The desire to "change the world" is really pretty limited in the broader population.
I don't know why this would be true of only small/medium companies. I would say it is true of most companies of any size. Without the right engineering culture you will not be able to enable building anything interesting.
Boring technologies seems like a good idea until you realize the market place for software people rewards people with cutting edge experience.
Boring technologies seems like a good idea until you realize the market place for software people rewards people with cutting edge experience.
> Instead of looking for "greatness", most companies in this position should simply settle for competence. Hire people who use boring technology and build systems that do things simply and directly.
Great engineers use boring technology and build simple and direct systems.
Great engineers use boring technology and build simple and direct systems.
That's what I would do, but no one seems to value that when you are jobseeking. It's all about having numerous shiny technologies on your CV.
There are plenty of "good enough" engineers, most jobs are not a lot more than "intellectual grunt work while being distracted" as someone on here said.
I really like the idea of "chopping the onion". I've personally had a handful of devs who passed our interview process with flying colors, but couldn't actually deliver anything once they got in the door. They could recite any leetcode problem from memory, but didn't know how to contribute value to their team or company.
> The grading is automatic. I think is ok as _one_ metric, but I wouldn't want it to the be the _only_ metric that is being looked at. ICs aren't really ICs. You can be a JIRA champion, knocking tickets into the done column at the speed of light, but Great Engineers do more than just that.
Great Engineers build up their team, and help others grow. They know how to push back on unrealistic asks. They can incite help when they get stuck. They can translate complex issues into digestible bites. Writing good code is the bare minimum quality of being a Great Engineer.
The main barrier I see to hiring Great Engineers is that you really need to hire Great People. But a lot of hiring practices are taking all the human aspects out of hiring (except for the "behavioral" questions, but even these can be gamed like system design or leetcode). Great People worthy of hiring are driven, passionate (not "spent all my waking time coding", but "truly believe in the mission/goals of the company/project"), empathetic, intelligent, and trust worthy. None of those traits are easily discernible from random logic puzzles.
I don't know what the solution is. At some point there has to be a "can actually write code" litmus test, but I don't believe any of the current methods work well to prove that. I think take home tests are closer to the "chopping onions" idea, but only require commitment from the applicant, not the employer, which leads to a lot of wasted time.
Personally, I think temp-to-hire or work-for-a-day type models could work, if there was some standards to prevent abuse. I would rather sacrifice a few days to a week to see if I actually like working somewhere, and to prove to them that I can do the job, then look at CTCI ever again.
> The grading is automatic. I think is ok as _one_ metric, but I wouldn't want it to the be the _only_ metric that is being looked at. ICs aren't really ICs. You can be a JIRA champion, knocking tickets into the done column at the speed of light, but Great Engineers do more than just that.
Great Engineers build up their team, and help others grow. They know how to push back on unrealistic asks. They can incite help when they get stuck. They can translate complex issues into digestible bites. Writing good code is the bare minimum quality of being a Great Engineer.
The main barrier I see to hiring Great Engineers is that you really need to hire Great People. But a lot of hiring practices are taking all the human aspects out of hiring (except for the "behavioral" questions, but even these can be gamed like system design or leetcode). Great People worthy of hiring are driven, passionate (not "spent all my waking time coding", but "truly believe in the mission/goals of the company/project"), empathetic, intelligent, and trust worthy. None of those traits are easily discernible from random logic puzzles.
I don't know what the solution is. At some point there has to be a "can actually write code" litmus test, but I don't believe any of the current methods work well to prove that. I think take home tests are closer to the "chopping onions" idea, but only require commitment from the applicant, not the employer, which leads to a lot of wasted time.
Personally, I think temp-to-hire or work-for-a-day type models could work, if there was some standards to prevent abuse. I would rather sacrifice a few days to a week to see if I actually like working somewhere, and to prove to them that I can do the job, then look at CTCI ever again.
> "truly believe in the mission/goals of the company/project"
Well, first make sure there actually is some "mission" there to "believe in". In 99.9% of the cases people work on perhaps technically exciting but impactwise boring industry projects, whose main mission is to make money. Requiring passion is a euphemism for being over-workable. That you'll work outside working hours, you can be called during vacation etc, because of course this job is the mission of your life, right? Otherwise, you're not passionate...
In practice, these "passionate" people are often young and naive or just really ambitious and work their ass off to build their resume and will be on their way soon enough.
A good engineer contributes high value and knows this. And therefore will make the company pay up, and will not take any bullshit. And being that good, they will be picking where to work and you as the company compete to get them. In these cases the interview is about the engineer judging the company and looking for exactly things like the requirement to "believe in the mission" or whether the conversation is on a professional, adult level.
Well, first make sure there actually is some "mission" there to "believe in". In 99.9% of the cases people work on perhaps technically exciting but impactwise boring industry projects, whose main mission is to make money. Requiring passion is a euphemism for being over-workable. That you'll work outside working hours, you can be called during vacation etc, because of course this job is the mission of your life, right? Otherwise, you're not passionate...
In practice, these "passionate" people are often young and naive or just really ambitious and work their ass off to build their resume and will be on their way soon enough.
A good engineer contributes high value and knows this. And therefore will make the company pay up, and will not take any bullshit. And being that good, they will be picking where to work and you as the company compete to get them. In these cases the interview is about the engineer judging the company and looking for exactly things like the requirement to "believe in the mission" or whether the conversation is on a professional, adult level.
> Requiring passion is a euphemism for being over-workable.
That's definitely true in the general case. Having passion means wanting to do the best overall thing because you care about the thing going well. The best overall thing is working hard to prevent long hours which lead to burnouts. If you have managers and senior leaders who share the engineers passion, this problem is self-correcting.
Source: I frequently lecture our (junior esp) new hires about working too many hours. Especially during incidents. Yes this is exciting. Now go sleep and see your family. Come back when you are rested and ready to play at full speed.
I also have uncomfortable talks with people who frequently say "meh, close enough" about whether they actually want to be here. My team performs at a consistently high level.
That's definitely true in the general case. Having passion means wanting to do the best overall thing because you care about the thing going well. The best overall thing is working hard to prevent long hours which lead to burnouts. If you have managers and senior leaders who share the engineers passion, this problem is self-correcting.
Source: I frequently lecture our (junior esp) new hires about working too many hours. Especially during incidents. Yes this is exciting. Now go sleep and see your family. Come back when you are rested and ready to play at full speed.
I also have uncomfortable talks with people who frequently say "meh, close enough" about whether they actually want to be here. My team performs at a consistently high level.
Not to mention that most companies don't need great engineers. Average engineers for average companies is perfectly fine. Of course, then the company leadership has to be realistic about their place in the universe, but that's a separate problem, plus hiring authorities can have a realistic view of the big picture and sell their hiring selections as "great engineers for a great company" to management.
It's kind of mutinous, but it's not like there's anything to be gained from loyalty to management who aren't loyal to employees.
It's kind of mutinous, but it's not like there's anything to be gained from loyalty to management who aren't loyal to employees.
> It's kind of mutinous, but it's not like there's anything to be gained from loyalty to management who aren't loyal to employees.
It goes both ways. The current equilibrium, at least in the US, is that (SW/IT) engineers leave after ~2 years, and it's causing quite a bit of friction. But it's hard to move out of the equilibrium.
It goes both ways. The current equilibrium, at least in the US, is that (SW/IT) engineers leave after ~2 years, and it's causing quite a bit of friction. But it's hard to move out of the equilibrium.
>But it's hard to move out of the equilibrium.
I don't think it's so much an equilibrium so much as a battle, denoted by "friction," but I'm sure it would come down to each saying the other one defected first. I think we can figure out where the disloyalty started, calculated from the first principles of capitalism.
I don't think it's so much an equilibrium so much as a battle, denoted by "friction," but I'm sure it would come down to each saying the other one defected first. I think we can figure out where the disloyalty started, calculated from the first principles of capitalism.
By friction I meant all the onboarding costs and time, and the lost knowledge evaporating with the leaving employees. There's tons of waste there.
> In 99.9% of the cases people work on perhaps technically exciting but impactwise boring industry projects, whose main mission is to make money.
Passion, imo, is an attitude - and is reflected in quality of work and teamwork. I do think it is often used to trick people into getting less than they deserve, but doesn't have to be the case.
When I say I want to work with passionate people, I don't mean people who are willing to work 80 hours a week for 10 dollars an hour because they love coding or want to solve world hunger. I mean I want to work with people who give their best, put the user/experience first, and are genuinely motivated to do good work. This doesn't mean you have to fake smile, or get taken advantage of, it just means you have bring some positivity to the table.
Passion, imo, is an attitude - and is reflected in quality of work and teamwork. I do think it is often used to trick people into getting less than they deserve, but doesn't have to be the case.
When I say I want to work with passionate people, I don't mean people who are willing to work 80 hours a week for 10 dollars an hour because they love coding or want to solve world hunger. I mean I want to work with people who give their best, put the user/experience first, and are genuinely motivated to do good work. This doesn't mean you have to fake smile, or get taken advantage of, it just means you have bring some positivity to the table.
Some people give their best, but sometimes it isn't good enough. In those cases, the company will just look for someone else. Then that passion counts for little.
This is the issue with signalling passion - it causes people to drive their expectations of you higher. You have a higher chance of not living up to someone's standard.
This is the issue with signalling passion - it causes people to drive their expectations of you higher. You have a higher chance of not living up to someone's standard.
Requiring passion is a euphemism for being over-workable.
Yes. Makes me think of Zappos, the shoe store. They're big on demanding employee enthusiasm.[1] But in the end, they're just a shoe store.
[1] https://www.zapposinsights.com/about/core-values
Yes. Makes me think of Zappos, the shoe store. They're big on demanding employee enthusiasm.[1] But in the end, they're just a shoe store.
[1] https://www.zapposinsights.com/about/core-values
> Personally, I think temp-to-hire or work-for-a-day type models could work
I've seen the contract-to-hire stuff come back up recently. There are a couple big flaws with that model, beyond possible abuse, though.
1) You basically are excluding yourself from any candidate that still has a full time job. It would be crazy for them to leave their current gig for a place that might not actually bring them on full time.
2) It doesn't scale, bringing on new team members properly takes time and effort. Not only are you potentially wasting their time, you are also wasting your team's time, and you can only do this kind of evaluation so many times.
I do agree though that the standard interview model doesn't make sense. Memorizing leetcode problems doesn't map to good engineering, and take home problems filter out a lot of good engineers that won't bother spending the extra time.
I've seen the contract-to-hire stuff come back up recently. There are a couple big flaws with that model, beyond possible abuse, though.
1) You basically are excluding yourself from any candidate that still has a full time job. It would be crazy for them to leave their current gig for a place that might not actually bring them on full time.
2) It doesn't scale, bringing on new team members properly takes time and effort. Not only are you potentially wasting their time, you are also wasting your team's time, and you can only do this kind of evaluation so many times.
I do agree though that the standard interview model doesn't make sense. Memorizing leetcode problems doesn't map to good engineering, and take home problems filter out a lot of good engineers that won't bother spending the extra time.
Hey, thanks for insightful feedback (blog post author here)!
I fully agree with you about temp-to-hire or work-for-a-day models. And I think if the company has such a process in place, it is the sign of a well-developed culture. It is exactly the point of the blogpost -- evaluate using the right metric for the job. However, this is rare in practice, and I would say for the same reason the well-established culture is rare: it requires a lot of work to be done by the very same great people in advance. It is a chicken and egg problem. I saw countless times in multiple companies the signs of immaturity that prevent them from giving people access to their production: secrets in git, shared accounts, lack of established onboarding and offboarding processes, no processes for credential rotation, absence of audit logs, legal issues, you name it. The majority of companies can not afford a work-for-a-day option.
Maintaining the dedicated challenge is another option for them, but it has its downsides, too. In our experience, the main one is that you either invest in creating the grading infrastructure upfront or spend a lot of time manually grading the submissions. Checking the correctness of the code is a hard task unless one can actually run it and see how it performs under a barrage of tests. I'm not saying there should not be a code review, of course. I am saying that the automated assessment of the code should be done before you start spending human time. Machines are cheap, your engineers are not. And at this point, one doesn't need AutoIterative, but has effectively reinvented it :)
We walked this path ourselves over the years. We did fizzbuzz style interviews, that was suboptimal. We graded challenges manually for a long time, which was tedious and error-prone. We made all the mistakes and tried to fix them. We were using the wrong metric.
Qualitative improvement was when we started using the right metric, and automating its assessment was the next logical step.
I fully agree with you about temp-to-hire or work-for-a-day models. And I think if the company has such a process in place, it is the sign of a well-developed culture. It is exactly the point of the blogpost -- evaluate using the right metric for the job. However, this is rare in practice, and I would say for the same reason the well-established culture is rare: it requires a lot of work to be done by the very same great people in advance. It is a chicken and egg problem. I saw countless times in multiple companies the signs of immaturity that prevent them from giving people access to their production: secrets in git, shared accounts, lack of established onboarding and offboarding processes, no processes for credential rotation, absence of audit logs, legal issues, you name it. The majority of companies can not afford a work-for-a-day option.
Maintaining the dedicated challenge is another option for them, but it has its downsides, too. In our experience, the main one is that you either invest in creating the grading infrastructure upfront or spend a lot of time manually grading the submissions. Checking the correctness of the code is a hard task unless one can actually run it and see how it performs under a barrage of tests. I'm not saying there should not be a code review, of course. I am saying that the automated assessment of the code should be done before you start spending human time. Machines are cheap, your engineers are not. And at this point, one doesn't need AutoIterative, but has effectively reinvented it :)
We walked this path ourselves over the years. We did fizzbuzz style interviews, that was suboptimal. We graded challenges manually for a long time, which was tedious and error-prone. We made all the mistakes and tried to fix them. We were using the wrong metric.
Qualitative improvement was when we started using the right metric, and automating its assessment was the next logical step.
> They could recite any leetcode problem from memory,
There is your problem. You are testing for something unrelated to the job.
There is your problem. You are testing for something unrelated to the job.
> handful of devs who passed our interview process with flying colors, but couldn't actually deliver anything once they got in the door. They could recite any leetcode problem from memory, but didn't know how to contribute value to their team or company.
Interesting, why do you think was this the case?
Interesting, why do you think was this the case?
Coding is not the only thing required to be an excellent developer and contributor to your team and company. Sometimes you must gather requirements from users / customers. Sometimes you have to work with other teams or products to integrate them into your product.
There’s also the aspect of working with the rest of your team on whatever problems you are solving and not siloing yourself to just the problems you’re working on.
That and when you’re hit with new and interesting problems that have never been solved before, you cannot look those up in the leetcode manual and memorize them to solve. You have to actually create something on your own and apply whatever you’ve learned from programming in the past.
It’s like playing a survival video game with the detailed how-to always at your fingertips versus being in the wilderness having to survive on your own wits without a guide.
There’s also the aspect of working with the rest of your team on whatever problems you are solving and not siloing yourself to just the problems you’re working on.
That and when you’re hit with new and interesting problems that have never been solved before, you cannot look those up in the leetcode manual and memorize them to solve. You have to actually create something on your own and apply whatever you’ve learned from programming in the past.
It’s like playing a survival video game with the detailed how-to always at your fingertips versus being in the wilderness having to survive on your own wits without a guide.
It does seem to be a thing with younger developers, who want to stay wfh and just work on jira tickets.
Any sensible developer learns early on that mostly the coding is the easy stuff - what Is the problem I am solving here?
Any sensible developer learns early on that mostly the coding is the easy stuff - what Is the problem I am solving here?
I see this time and again from observation over 20 years. Also many great engineers don't have cs backgrounds and don't do well on whiteboard/leetcode
Not the person you're responding to, but my perspective is that there is often a false equivalency between academic understanding (solving hard/clever problems) and productive development (meeting deadlines, building/fixing projects). Many of the stereotypical developer interviews assume "good developers" who are good at one are good at the other. My anecdotal experience is that most are not, they excel in one or the other.
I don’t think they are exclusive, you can be a great engineer that knows all about the algorithms in CS books and keeps up on the latest research in parts of the academic field. They aren’t even different skill sets, just different places to find the knowledge. Game development and cryptography bear this out well. The top engineers are the ones that read all the papers and that can build solid systems based on them.
I don't think I framed it as exclusive, but that the intersection of both sets of skills is uncommon (I attest they are two different sets). Sure, you can aim to hire only people who excel in both, but the typical software interview certainly doesn't assess the latter.
[deleted]
some common engineering things that leetcode doesn't test for
- using http apis and transforming the resulting data
- using database queries and transforming the resulting data
- navigating a large codebase
- figuring out what tasks are important to accomplish
- researching things you don't know
- debugging a complex issue
- evaluating tradeoffs
- general engineering intuitionDoer/implementation vs abstract/big idea thinker.
thanks, can be useful tips for my future staff recruitment.
thanks, can be useful tips for my future staff recruitment.
It's like memorizing the answers to an IQ test. A relative of mind knows all the answers, as he conducted the test on many occasions, but he doesn't have the max IQ.
Ye. Making up quick sort sort on the fly without ever seeing the solution would be really impressive, but reciting it from memory doesn't say much. If someone would implement "max sort" on a interview at-least you would know he didn't practice for the "IQ test".
>>They could recite any leetcode problem from memory, but didn't know how to contribute value to their team or company.
That's understandable any one with that much proficiency on leetcode would be wasting their whole day on that site than building real things.
That's understandable any one with that much proficiency on leetcode would be wasting their whole day on that site than building real things.
> Second—people who had spent the last month memorizing the “Cracking The Coding Interview.” They will pass as well, but you will have no assessment of whether they can apply their knowledge. That is, until their first deployment to your production environment.
Everyone criticizes this, including me before I decided to capitulate to the system and do it myself. But it's actually kind of fair? Everyone knows the game. Some people will choose to train hard enough to play, some people will not. That is a signal in of itself as to whether the candidate will work well in the big tech company environment.
Everyone criticizes this, including me before I decided to capitulate to the system and do it myself. But it's actually kind of fair? Everyone knows the game. Some people will choose to train hard enough to play, some people will not. That is a signal in of itself as to whether the candidate will work well in the big tech company environment.
I didn't read it as criticism of the engineers reading “Cracking The Coding Interview.” - if you want to pass algorithmic interviews you have to read it or something equivalent - or be fresh out of college.
But just as this post says, as a hiring manager, you'll get very little insight into how good the candidate is until after you've hired them. This does serve as a weak signal for dedication, but:
1. You don't want to make your hiring decision only based on dedication.
2. There are other, better signals for dedication.
But just as this post says, as a hiring manager, you'll get very little insight into how good the candidate is until after you've hired them. This does serve as a weak signal for dedication, but:
1. You don't want to make your hiring decision only based on dedication.
2. There are other, better signals for dedication.
>>very little insight into how good the candidate is until after you've hired them.
The fact that one depends on a proxy measurement which has little correlation with actual results is the core of the problem here.
The fact that one depends on a proxy measurement which has little correlation with actual results is the core of the problem here.
Hey! To be fair, we don't criticize it (blog post author here), nor do we discourage such training, of course. It is perfectly valid and could be a good ROI, as you mentioned. What we found out is that using algorithmic knowledge as a metric for hiring was suboptimal. As other people mentioned, there's plenty of people w/o formal CS background who can deliver, and there's plenty of people who know algorithms by heart but have trouble shipping things. There's some correlation, of course, but I would say those are different dimensions.
And then it looks like this: when hiring someone, the company measures them by one dimension and then expects them to perform on another one. I did get some backlash for comparing engineers to cooks, heh, but the main point is and always was about choosing the right metric.
And then it looks like this: when hiring someone, the company measures them by one dimension and then expects them to perform on another one. I did get some backlash for comparing engineers to cooks, heh, but the main point is and always was about choosing the right metric.
You are just testing willingness to jump through hoops in order to get the job (and, hopefully, to keep it). Which could be all that is needed for a megacorp job. In a workplace where you are very likely have more than enough brainpower to solve any problem just because of its size, teamwork and loyalty are much more important than expertise.
In a smaller company this is not true though. If you only have 10 engineers there is a good chance that none of them is qualified enough to solve the current problem so even if they all are extremely loyal and cooperative - the problem will remain unsolved. Even with a 100 you might end with just ~10 experts who are going to be overextended, burn out and quit. So, while loyalty and teamwork are still important, you really, really want to test for the actual expertise in a small-medium company. Unless you don't foresee any hard problems, then hiring for loyalty is the best you can do.
In a smaller company this is not true though. If you only have 10 engineers there is a good chance that none of them is qualified enough to solve the current problem so even if they all are extremely loyal and cooperative - the problem will remain unsolved. Even with a 100 you might end with just ~10 experts who are going to be overextended, burn out and quit. So, while loyalty and teamwork are still important, you really, really want to test for the actual expertise in a small-medium company. Unless you don't foresee any hard problems, then hiring for loyalty is the best you can do.
In a workplace where you are very likely have more than enough brainpower to solve any problem just because of its size
Except that a company that's 10 times as big is working on 10 times as many things. The idea that a huge company can afford to hire just anyone because they're so huge and they only need a few smart people is ludicrous. If that were the case, these companies wouldn't feel the need to pay so much.
Except that a company that's 10 times as big is working on 10 times as many things. The idea that a huge company can afford to hire just anyone because they're so huge and they only need a few smart people is ludicrous. If that were the case, these companies wouldn't feel the need to pay so much.
It does not matter as long as 99.9% of those things are a mix of busywork and trivial problems.
The scenario I have in mind is something like this: you make an app, your team manages UI, backend, whatever, just fine - it's simple and is 99% of the job. You release your app and a million people download it, next day you are flooded with crash reports, it crashes for 25% of the users!
Megacorp: calls a special team of greybeards who spend 24 hours digging and find it's a bug in a driver for some popular chipset. They write a fix in another hour, file a bug with the popular chipset manufacturer and return back to playing WoW.
Small business: panic, panic some more, post questions on stackoverflow, really panic, find a bogus solution, release a patch in a week which makes the app crash even more because it's rushed and is actually breaking things by design, go back to panicking. The app gets bad rep, even people who had no problems are dropping it, its reputation is destroyed. The small business tries to pivot and shuts down in 6 months.
Problems like this do not happen often but when they happen it may end the business so you don't need a lot of resources to solve them but if you have 0 then you are not going to last.
The scenario I have in mind is something like this: you make an app, your team manages UI, backend, whatever, just fine - it's simple and is 99% of the job. You release your app and a million people download it, next day you are flooded with crash reports, it crashes for 25% of the users!
Megacorp: calls a special team of greybeards who spend 24 hours digging and find it's a bug in a driver for some popular chipset. They write a fix in another hour, file a bug with the popular chipset manufacturer and return back to playing WoW.
Small business: panic, panic some more, post questions on stackoverflow, really panic, find a bogus solution, release a patch in a week which makes the app crash even more because it's rushed and is actually breaking things by design, go back to panicking. The app gets bad rep, even people who had no problems are dropping it, its reputation is destroyed. The small business tries to pivot and shuts down in 6 months.
Problems like this do not happen often but when they happen it may end the business so you don't need a lot of resources to solve them but if you have 0 then you are not going to last.
Except the problem didn't happen after the bug was revealed. The problem happened before, during development. Better engineering, better development practices, better engineers, is what you need to avoid problems like that.
And you don't get better engineers by hiring just anyone who comes along.
And you don't get better engineers by hiring just anyone who comes along.
Yes, better engineers would have caught it before release, maybe. This is why I am saying that testing for loyalty makes no sense for smaller companies as a loyal engineer may not be any good. I can even speculate that the best engineers are unlikely to be very loyal e.g. I doubt Google makes its "stars" to jump through the hoops.
Don't you think that requiring candidates to spend a month studying makes it so that only candidates that can afford to do so are hired? Some people have other responsibilities outside of work that mean they simply cannot spend time memorizing a book. It adds insult to injury that for the vast majority of the time, you won't use anything from that book. And the interview doesn't test for skills that you actually do use at your job.
The best whiteboard/leetcode engineer I was involved in hiring and worked with was absolutely terrible at designing and coding real software. The signal of dedication is a pretty poor proxy for real capability.
Just wondering, was it any good? Would you say it played a big part in passing whiteboard coding and landed your current job? Asking because I'm thinking of doing it myself..
Absolutely. I went to the local library every Saturday for a month or two and did practice problems without distractions. Total prep time was probably ~40 hours.
My compensation increased by at least $150,000 a year.
Best ROI I've gotten on anything I've done in my life.
I tell that stories to people considering doing it all the time, you'd be amazed how many people hear my experience and say, yeah, naw I'm not going to spend that time.
¯\_(ツ)_/¯
My compensation increased by at least $150,000 a year.
Best ROI I've gotten on anything I've done in my life.
I tell that stories to people considering doing it all the time, you'd be amazed how many people hear my experience and say, yeah, naw I'm not going to spend that time.
¯\_(ツ)_/¯
Totally agree. I spent a lot longer (months!) on a very challenging real-world side-project, hoping that would be the ticket to a job.
At the end of the day, it was the ~40-60 hours I put into Cracking the Code Interview that got me the FAANG job, the side project was barely a blip.
At the end of the day, it was the ~40-60 hours I put into Cracking the Code Interview that got me the FAANG job, the side project was barely a blip.
40 hours worth of work is a lot.
That's a weeks wage on what is essentially a lottery ticket, albeit with good odds. (Having done some freelance work for a friend in my Spare time, finding 40 hours free is going to take me weeks).
That's a weeks wage on what is essentially a lottery ticket, albeit with good odds. (Having done some freelance work for a friend in my Spare time, finding 40 hours free is going to take me weeks).
Meanwhile, I’ve put hundreds of hours of prep in and never received an offer from FAANG.
It really is a lottery ticket. Luck plays a ridiculously large part of the interview.
It really is a lottery ticket. Luck plays a ridiculously large part of the interview.
I had a phone / screenshare interview with Google one day. Asked me to solve a Soduko problem (I have never done a Sudoko until that point). Didn't get it.
Half an hour later wandering done the street a very elegant solution came to me. I wouldn't say I am bad algorithmic stuff, but as I never practice it's pretty random whether I will get during an interview. "Luck" as you say.
Half an hour later wandering done the street a very elegant solution came to me. I wouldn't say I am bad algorithmic stuff, but as I never practice it's pretty random whether I will get during an interview. "Luck" as you say.
How many problems did you practice?
What was your question bank.
What was your question bank.
I did all the medium & hard ones in there and was getting through them pretty consistently. I also don't have a CS degree so prior to that book I took the online Stanford algorithms class, I'm not including that in the 40 hours of work because I think most people reading this probably have more formal education in the field than I have.
Engineering is vastly more complex than chopping onions. I strive for the ability to on-board in less than a day for projects I work on but it's not realistic in many cases. Companies need to put effort into having small side projects with dedicated github issues only meant for these candidates (anecdote: the few companies I've applied to that do this type of thing did not do a good job: their tickets were vague, lacking context, and mostly just led me wanting to talk to the PO).
Many engineers are also looking for jobs while they currently have one, making a half/whole day "trial" very difficult or impossible. Asking people to commit their weekend or free time is also discriminating against those with families, social lives, and those who just like leaving programming behind when office hours are over.
The article only talks about algorithmic interviews. There are plenty of other interview styles that delve into more pragmatic problem solving that still aren't perfect, but work much better.
Many engineers are also looking for jobs while they currently have one, making a half/whole day "trial" very difficult or impossible. Asking people to commit their weekend or free time is also discriminating against those with families, social lives, and those who just like leaving programming behind when office hours are over.
The article only talks about algorithmic interviews. There are plenty of other interview styles that delve into more pragmatic problem solving that still aren't perfect, but work much better.
Contract work would work similarly for engineers. Do it on your own time if you have a job.
They can ask the current team questions via chat. Get some real code into PRs and prove that they have skills.
Personally I don't even apply to jobs anymore. I am tired of 6+ hour code reviews that I don't get paid for, and no explanation when I get passed up.
I would much rather be a contract worker and get paid at least something while doing actual work, over building a search engine for star wars characters.
I'm exactly like what you describe. When I leave the office, I want to camp, rock climb, and prep for marathons. I don't want to spend more time coding. That's exactly what makes me absolutely disgusted with applying for new opportunities. Eight interviews with various people, just to get ghosted. Time I could have been improving at other passions. Instead I'm left with no time, no indication of what I can improve on, and no job.
They can ask the current team questions via chat. Get some real code into PRs and prove that they have skills.
Personally I don't even apply to jobs anymore. I am tired of 6+ hour code reviews that I don't get paid for, and no explanation when I get passed up.
I would much rather be a contract worker and get paid at least something while doing actual work, over building a search engine for star wars characters.
I'm exactly like what you describe. When I leave the office, I want to camp, rock climb, and prep for marathons. I don't want to spend more time coding. That's exactly what makes me absolutely disgusted with applying for new opportunities. Eight interviews with various people, just to get ghosted. Time I could have been improving at other passions. Instead I'm left with no time, no indication of what I can improve on, and no job.
I see sentiments like this expressed sometimes, and I'm usually a bit confused. Are you currently employed, and if so is it in a permanent role or as a contractor? What do you plan to do the next time you want to change jobs?
Genuinely curious here, I only know one model and it works well enough for me, but you seem to have a different one and I'd like to know more about it.
Genuinely curious here, I only know one model and it works well enough for me, but you seem to have a different one and I'd like to know more about it.
I have been fully employed for years. Though I have sought a less abusive relationship a number of times.
Obviously, should my current security evaporate, I would resolve to deal with the process.
I'm not entirely sure what you're confused about. Nor, what model you refer to.
Obviously, should my current security evaporate, I would resolve to deal with the process.
I'm not entirely sure what you're confused about. Nor, what model you refer to.
> also looking for jobs while they currently have one
Thanks for mentioning this. Once you have a family at home it gets unrealistic to not work while looking for work. Especially in countries where your contract can usually require months of notice and preclude unemployment in case you quit.
Thanks for mentioning this. Once you have a family at home it gets unrealistic to not work while looking for work. Especially in countries where your contract can usually require months of notice and preclude unemployment in case you quit.
"The restaurant industry figured the hiring way better than the software engineering industry! They test both professional skills and cultural alignment! In other words, they test your ability to deliver. They do not ask you to explain a recipe by heart or describe how you would chop the onion. They ask you to do it. Live."
It's ironic that the article points out the error in blindly imitating others and then we get this entirely apples and oranges 'recipe for effective interviews'.
What are the fixed costs of having a 'live' chop the onions and shoot the breeze on slack for software companies? For a restaurant, they simply need a bit of tabletop for the 'new workstation'.
- All onions are pretty much the same in one industry; in our industry, we famously have 'a new way to slice onions' every other day! Let's not even discuss the variety of knives ..
- What 'secret sauce' info is disclosed during the 'live' getting to know each other? Can we just put candidates in company slack and check out how it works? Seems problematic.
It's ironic that the article points out the error in blindly imitating others and then we get this entirely apples and oranges 'recipe for effective interviews'.
What are the fixed costs of having a 'live' chop the onions and shoot the breeze on slack for software companies? For a restaurant, they simply need a bit of tabletop for the 'new workstation'.
- All onions are pretty much the same in one industry; in our industry, we famously have 'a new way to slice onions' every other day! Let's not even discuss the variety of knives ..
- What 'secret sauce' info is disclosed during the 'live' getting to know each other? Can we just put candidates in company slack and check out how it works? Seems problematic.
So much nope to this.
> They would put you to the test for like 2-3 days to see how you perform
And pay you. And give you some level of feedback along the way, not just "here make some salads" and then "nah nevermind" after 3 days of hard work. And you get to work with them and decide if you like working there too. It's not a one-way street like these bullshit homework interview pipelines.
Unlike in food, you're not going to be productive with real code for a few months. Unlike in food, just doing the coding is only 40-60% of the job. Unlike in food, it's hard to improvise code in a language, framework, test-style, knowledge-domain, or other toolset you may not be familiar with. Hiring for restuarants is not a relevant comparison.
This just seems like the typical unpaid homework assignment--which is usually a couple softball algorithm questions mixed with some arbitrary glue-code riddles--with some CI-flavored slop on top.
> They would put you to the test for like 2-3 days to see how you perform
And pay you. And give you some level of feedback along the way, not just "here make some salads" and then "nah nevermind" after 3 days of hard work. And you get to work with them and decide if you like working there too. It's not a one-way street like these bullshit homework interview pipelines.
Unlike in food, you're not going to be productive with real code for a few months. Unlike in food, just doing the coding is only 40-60% of the job. Unlike in food, it's hard to improvise code in a language, framework, test-style, knowledge-domain, or other toolset you may not be familiar with. Hiring for restuarants is not a relevant comparison.
This just seems like the typical unpaid homework assignment--which is usually a couple softball algorithm questions mixed with some arbitrary glue-code riddles--with some CI-flavored slop on top.
I actually like the homework assignments.
As an interviewer, I think they are the best at assessing the coding aspects of job performance. Not perfect, but better than high-pressure in-person whiteboard/coding tests.
As an interviewee, I like the flexibility to work on it as I have time (rather than using up PTO).
The point of using algorithms isn't to test your algorithm skills (though using job relevant algorithms is a good idea if you can) but to test your ability to translate requirements into code by eliminating the quality of requirement specification as a variable. That is, because algorithms have an unambiguous specification, failing the coding assignment due to poorly specified requirements should impossible (or much less likely).
For the non-coding part of the job, I find a (relatively short) structured interview works best.
As an interviewer, I think they are the best at assessing the coding aspects of job performance. Not perfect, but better than high-pressure in-person whiteboard/coding tests.
As an interviewee, I like the flexibility to work on it as I have time (rather than using up PTO).
The point of using algorithms isn't to test your algorithm skills (though using job relevant algorithms is a good idea if you can) but to test your ability to translate requirements into code by eliminating the quality of requirement specification as a variable. That is, because algorithms have an unambiguous specification, failing the coding assignment due to poorly specified requirements should impossible (or much less likely).
For the non-coding part of the job, I find a (relatively short) structured interview works best.
As an interviewer homework loops are easier for you. Of course. It requires significantly less of your time time. You don't have to actually confront a human being in front of you, understand the thought-process they went through in doing the work, answer questions about yourself, projects, or the company. You can say no by typing a bit.
You've taken all of the leverage and one-sided information, given nothing to the candidate, and you expect them to be okay with this without payment. Good candidates, myself included, will not jump through arbitrary hoops or suffer such foolery.
I have ample github history, 15 years of senior-level experience with good companies, and the ability to discuss solutions at every level in a tech-stack. I will not coderpad on my own or do unpaid homework. I will pair-program for an hour with an agreeable potential-colleague, but not with somebody who's hiring for other teams.
When you say jump, I say jump with me or I'll see ya later.
You've taken all of the leverage and one-sided information, given nothing to the candidate, and you expect them to be okay with this without payment. Good candidates, myself included, will not jump through arbitrary hoops or suffer such foolery.
I have ample github history, 15 years of senior-level experience with good companies, and the ability to discuss solutions at every level in a tech-stack. I will not coderpad on my own or do unpaid homework. I will pair-program for an hour with an agreeable potential-colleague, but not with somebody who's hiring for other teams.
When you say jump, I say jump with me or I'll see ya later.
> with an agreeable potential-colleague
This is a huge part of it to me. In a whiteboard/pair-programming interview, you get the ability to judge the company and potential colleagues. In a take home, you get absolutely nothing out of it.
This is a huge part of it to me. In a whiteboard/pair-programming interview, you get the ability to judge the company and potential colleagues. In a take home, you get absolutely nothing out of it.
> As an interviewer homework loops are easier for you.
I have been on both the interviewee and interviewer side of this interview style, and I speak from both perspectives when I say that like this style.
> You don't have to actually confront a human being in front of you, understand the thought-process they went through in doing the work, answer questions about yourself, projects, or the company. You can say no by typing a bit.
Are you assuming that the homework is the only part of the interview process? That's absurd. Once they complete the assignment (which should be an objective thing to assess with practically unambiguous criteria) they are brought in to an interview. If they did a good job on the homework, they'll likely get the job. I would rather not ask them to take time out of their workday for their current employer until I think there's a good chance we will extend them an offer.
The interview is exactly what you describe: a chance for them to evaluate me, for us to have a discussion about their thought process regarding their code, about their prior experience, etc.
> You've taken all of the leverage and one-sided information, given nothing to the candidate
Not sure what you are mean exactly here.
> and you expect them to be okay with this without payment.
I also expect them to send me a resume without payment, and people can easily take as much time editing and formatting their resume as they do on the coding assignment.
I'm not categorically opposed to the idea of paying candidates who submit a (correct) solution to the homework assignment. That seems like a nice gesture. It just wasn't an idea that ever came up in discussion, and in retrospect doesn't really seem like it was necessary.
Honest question: If you were to get paid for a completed homework assignment, would all of your objections go away?
> Good candidates, myself included, will not jump through arbitrary hoops or suffer such foolery.
We had some really good candidates (and hires) using this process, but nice No True Scotsman.
> I have ample github history, 15 years of senior-level experience with good companies, and the ability to discuss solutions at every level in a tech-stack.
Good for you.
For every candidate, I would always assess the homework assignment before looking at their resume, or even seeing their name. I wanted to have it be as much of a "blind audition" as possible. We brought in people with weak homework assignments and strong resumes, and people with strong homework assignments and weak resumes (and other combinations). While not a huge sample size, the experience of myself and others on my team was that a strong homework assignment was a better predictor of a good interview and job performance than a strong resume.
> I will not coderpad on my own or do unpaid homework. I will pair-program for an hour with an agreeable potential-colleague, but not with somebody who's hiring for other teams.
If that's your preference, that's fine. As professionals in the same industry who both in good faith want to improve the state of interviewing, we can discuss the trade-offs between homework and pair-programming.
But let's not pretend there are no trade-offs. Time-boxed live programming interviews favor people who are good at working under pressure and who are already familiar the the technical stack. They filter out people who may be great engineers but are more deliberative or come from a different tech stack.
I have been on both the interviewee and interviewer side of this interview style, and I speak from both perspectives when I say that like this style.
> You don't have to actually confront a human being in front of you, understand the thought-process they went through in doing the work, answer questions about yourself, projects, or the company. You can say no by typing a bit.
Are you assuming that the homework is the only part of the interview process? That's absurd. Once they complete the assignment (which should be an objective thing to assess with practically unambiguous criteria) they are brought in to an interview. If they did a good job on the homework, they'll likely get the job. I would rather not ask them to take time out of their workday for their current employer until I think there's a good chance we will extend them an offer.
The interview is exactly what you describe: a chance for them to evaluate me, for us to have a discussion about their thought process regarding their code, about their prior experience, etc.
> You've taken all of the leverage and one-sided information, given nothing to the candidate
Not sure what you are mean exactly here.
> and you expect them to be okay with this without payment.
I also expect them to send me a resume without payment, and people can easily take as much time editing and formatting their resume as they do on the coding assignment.
I'm not categorically opposed to the idea of paying candidates who submit a (correct) solution to the homework assignment. That seems like a nice gesture. It just wasn't an idea that ever came up in discussion, and in retrospect doesn't really seem like it was necessary.
Honest question: If you were to get paid for a completed homework assignment, would all of your objections go away?
> Good candidates, myself included, will not jump through arbitrary hoops or suffer such foolery.
We had some really good candidates (and hires) using this process, but nice No True Scotsman.
> I have ample github history, 15 years of senior-level experience with good companies, and the ability to discuss solutions at every level in a tech-stack.
Good for you.
For every candidate, I would always assess the homework assignment before looking at their resume, or even seeing their name. I wanted to have it be as much of a "blind audition" as possible. We brought in people with weak homework assignments and strong resumes, and people with strong homework assignments and weak resumes (and other combinations). While not a huge sample size, the experience of myself and others on my team was that a strong homework assignment was a better predictor of a good interview and job performance than a strong resume.
> I will not coderpad on my own or do unpaid homework. I will pair-program for an hour with an agreeable potential-colleague, but not with somebody who's hiring for other teams.
If that's your preference, that's fine. As professionals in the same industry who both in good faith want to improve the state of interviewing, we can discuss the trade-offs between homework and pair-programming.
But let's not pretend there are no trade-offs. Time-boxed live programming interviews favor people who are good at working under pressure and who are already familiar the the technical stack. They filter out people who may be great engineers but are more deliberative or come from a different tech stack.
The issue with homework is that the top percentage of engineers really won't have any time to complete a multi-hour homework assessment that might not even be graded by a human. For spammy applications or bad colleges sure. But that's not the target demographic.
How will said engineers have an entire day to do an onsite interview loop, if they can't carve out a couple of hours to do a homework assignment?
My real problem with take home assignments is that while they take away the stress of being put on the spot in front of a whiteboard, often, the reward for doing well on them is... being put on the spot in front of a whiteboard. If anything, it might get one past the phone screen, but phone screens are not the biggest problem in SWE hiring.
My real problem with take home assignments is that while they take away the stress of being put on the spot in front of a whiteboard, often, the reward for doing well on them is... being put on the spot in front of a whiteboard. If anything, it might get one past the phone screen, but phone screens are not the biggest problem in SWE hiring.
The homework assignment is low effort on the employer's side but high effort on the candidate's side. Especially if it's through some SaaS where even the grading is automated. If you are Google then sure, candidates will take the challenge. But the reward is a job at Google. I'm not against take homes, I've said it here[0], they are great when you are swarmed with low quality resumes from places that have a bad application to hire ratio.
To me an on-site is more serious and shows commitment. You are asking the candidate to dedicate an entire day but asking the same to your engineering team. From my experience, engineers with multiple competing offers will go to an on-sites with certain companies that they would have ignored had they sent an online homework.
[0] https://news.ycombinator.com/item?id=24001495
To me an on-site is more serious and shows commitment. You are asking the candidate to dedicate an entire day but asking the same to your engineering team. From my experience, engineers with multiple competing offers will go to an on-sites with certain companies that they would have ignored had they sent an online homework.
[0] https://news.ycombinator.com/item?id=24001495
As an interviewee, a full-day onsite is much more of a turn-off for me. I don't want to use up PTO for my current employer for such a long, stressful interview process, especially without any indication of the likelihood that I will get an offer.
> How will said engineers have an entire day to do an onsite interview loop
Right. This practice is not better than an onsite. It's actually worse. Not only do you have to put in arbitrary amounts of time, you're not able to do any actual assessment of a prospective employer. You have nothing to go on other than a random problem statement which the company could have perfected over hundreds of hours and interviews. It's giving the employer all the leverage while also doing work for them for free.
Right. This practice is not better than an onsite. It's actually worse. Not only do you have to put in arbitrary amounts of time, you're not able to do any actual assessment of a prospective employer. You have nothing to go on other than a random problem statement which the company could have perfected over hundreds of hours and interviews. It's giving the employer all the leverage while also doing work for them for free.
> often, the reward for doing well on them is... being put on the spot in front of a whiteboard.
Ha! That can be true. So for me personally, I had candidates do a little bit of whiteboarding, but all I do is ask them to walk me through the execution of the algorithm they just implemented using minimal input parameters. Really what I'm looking for is the ability to explain how their code works (and to make sure they didn't copy someone else's work).
Ha! That can be true. So for me personally, I had candidates do a little bit of whiteboarding, but all I do is ask them to walk me through the execution of the algorithm they just implemented using minimal input parameters. Really what I'm looking for is the ability to explain how their code works (and to make sure they didn't copy someone else's work).
My company makes us interview with whiteboard problems. It's stupid but not my decision. My favorite spin on this is the reverse problem.
Ask the candidate to talk about, solve, and explain their favorite whiteboard-style problem. Ask them for something they were initially intimidated by. Maybe ask them a slight twist of it. We all know candidates study riddle problems, so let's both take advantage of that rather than giving an anxiety-test even if it's phrased as low-key.
Ask the candidate to talk about, solve, and explain their favorite whiteboard-style problem. Ask them for something they were initially intimidated by. Maybe ask them a slight twist of it. We all know candidates study riddle problems, so let's both take advantage of that rather than giving an anxiety-test even if it's phrased as low-key.
This actually measures communication skills, and it's often overlooked. If someone's able to clearly explain a design they did or an algorithm/data-structure that's non-trivial it's a good thing.
Not every company subjects you to 8 hours of interviews.
Geeze why is everyone so damn FAANG focused on this site?
Not every company, true. But, I've interviewed at least 20 or 30 times over the course of my career, and every one had an onsite loop of at least 4 hours in length. None of hte companies were FAANG, or even close. More than 90% were your typical whiteboard hazing style.
That's really not my experience. The places I've worked that have assignments like have managed to hire some pretty good senior engineering talent, and these people also often have pretty involved lives outside of work (families, communities, etc.).
You've gotten lucky. Imagine how many candidates have taken a look at your process and decided to value their own time and instead look at companies that don't jerk people around.
I don't think luck has anything to do with it.
If some people find this process too burdensome, that's their right. But plenty of skilled engineers that I've worked with don't see it as being jerked around. At the very least, I find it less burdensome that turning coding into some sort of live-performance (which an in-person coding test inevitably is).
Personally, I don't judge an hiring process primarily by how convenient it is for me. I judge it by how likely it is to be a good filter. If the company I'm applying to has a hiring process that is effective at selecting good candidates (along a number of different dimensions) then I'm more likely to enjoy working with the people there.
From this perspective, homework assignments are a good filter, and so companies that use that approach seem like better places to apply.
If some people find this process too burdensome, that's their right. But plenty of skilled engineers that I've worked with don't see it as being jerked around. At the very least, I find it less burdensome that turning coding into some sort of live-performance (which an in-person coding test inevitably is).
Personally, I don't judge an hiring process primarily by how convenient it is for me. I judge it by how likely it is to be a good filter. If the company I'm applying to has a hiring process that is effective at selecting good candidates (along a number of different dimensions) then I'm more likely to enjoy working with the people there.
From this perspective, homework assignments are a good filter, and so companies that use that approach seem like better places to apply.
> you're not going to be productive with real code for a few months
I've hired several engineers who get productive within the first week. Most manage to get in a PR and deploy to production by the end of the first day.
If it's taking a few months before the candidate can get productive, something has gone horrifically wrong.
I've hired several engineers who get productive within the first week. Most manage to get in a PR and deploy to production by the end of the first day.
If it's taking a few months before the candidate can get productive, something has gone horrifically wrong.
> If it's taking a few months before the candidate can get productive, something has gone horrifically wrong.
There's a big difference between "deployed their first PR" and "getting productive." The latter implies the engineer knows or has figured out how to use your source control, CI, and deployment processes. That probably is something most people can manage in a day, with proper instruction, provided the code change itself is simple enough. The latter means they're familiar with large parts of the codebase and are able to make changes, and have an idea how the major parts interact. This is what takes a few months.
There's a big difference between "deployed their first PR" and "getting productive." The latter implies the engineer knows or has figured out how to use your source control, CI, and deployment processes. That probably is something most people can manage in a day, with proper instruction, provided the code change itself is simple enough. The latter means they're familiar with large parts of the codebase and are able to make changes, and have an idea how the major parts interact. This is what takes a few months.
We've probably all seen or heard of places where a developer sits around for weeks, going to standups with their status update - "waiting on license XYZ" or waiting for access to the project (git, TFS, jira, vpns etc). It's a huge problem in many orgs.
It's also a good sniff test if you're interviewing. Slow and steady might be up your alley too. There's definitely a hierarchy of security and compliance needs depending on the org, but many orgs loose the forest for the trees.
It's also a good sniff test if you're interviewing. Slow and steady might be up your alley too. There's definitely a hierarchy of security and compliance needs depending on the org, but many orgs loose the forest for the trees.
Solving a maze problem or fixing a gimme CSS or unit-test bug is significantly different development from what even junior-level engineers do on a regular basis after a few months in. The kinds of code you get on any sort of interview or one-day homework assignment are significantly different from what you'll see in 3 months. If not, as you say, something has gone horrifically wrong.
> ... something has gone horrifically wrong
Agree 100%. And in most cases it probably isn't a problem with the candidate. It's a problem with the existing software and development process. It shouldn't take someone months to become useful on a new project.
Agree 100%. And in most cases it probably isn't a problem with the candidate. It's a problem with the existing software and development process. It shouldn't take someone months to become useful on a new project.
WordPress does the take home assignment but they pay you for your time, which is very cool.
And how many candidates do they think you can interview this way?
Let's say you hire 1 our of 10 candidates. Can you afford to give these types of assignments and manage them for 10 people to hire 1? Sounds completely implausible to me.
Let's say you hire 1 our of 10 candidates. Can you afford to give these types of assignments and manage them for 10 people to hire 1? Sounds completely implausible to me.
Conversely, let's say a candidate joins 1 out of 10 companies they apply for.
Can they afford to spend a few days coding or interviewing with each company? That's more than a solid month of work for the candidate.
Can they afford to spend a few days coding or interviewing with each company? That's more than a solid month of work for the candidate.
That's not my problem as an interviewee. Competence is expensive, yes, and compared to an actual salary a few days' work is nothing.
I've always thought that a good way to do a practical interview is to get the candidate to do a fizzbuzz. Then tell them that the requirements have changed, they need to add feature x. Then feature y, and so on. I reckon this would avoid ppl who just learn the test, and comes some way to emulating what actually happens in a real world job.
But anyways, I'm not a hiring manager, so someone go out and try this and tell me why it's a terrible idea.
But anyways, I'm not a hiring manager, so someone go out and try this and tell me why it's a terrible idea.
This is exactly how I do my interviews. I have them "whiteboard" (or use a real computer) to wrote a very simple function, then I change requirements when they are almost done to see what they do.
I do not agree with the premise that the best way to hire engineers is to follow the hire—chef model of restaurants. I don’t think it’s fair in any way to ask someone to work for free for few days, while you keep the candidate guessing whether you will give the job to the candidate or not. Also how can one plan to hire engineers who already have day jobs? Should the engineers take 3 days leave from their employer every-time they try to interview?
This kind of model is way too biased towards employers and gives them too much power to exploit, in an environment where employers anyways have a lot of power as compared to one employee (prospective candidate in fact, not even employee). This reasoning works well for companies whose job is to recruit or help recruit engineers. After all, they are being paid by the employer and not the candidate, so they want to make the process optimal for the employers. But as much as I like the doing the right thing for customer attitude, this seems like taking that to an extreme and exploiting everything that isn’t paying.
This kind of model is way too biased towards employers and gives them too much power to exploit, in an environment where employers anyways have a lot of power as compared to one employee (prospective candidate in fact, not even employee). This reasoning works well for companies whose job is to recruit or help recruit engineers. After all, they are being paid by the employer and not the candidate, so they want to make the process optimal for the employers. But as much as I like the doing the right thing for customer attitude, this seems like taking that to an extreme and exploiting everything that isn’t paying.
> This kind of model is way too biased towards employers and gives them too much power to exploit, in an environment where employers anyways have a lot of power as compared to one employee (prospective candidate in fact, not even employee).
And the issue with that is that great talent will simply lose interest. Unless of course the company is willing to be extremely aggressive salary-wise or is a leader in a very visible field (SpaceX). Because they typically have have multiple offers on the table.
And the issue with that is that great talent will simply lose interest. Unless of course the company is willing to be extremely aggressive salary-wise or is a leader in a very visible field (SpaceX). Because they typically have have multiple offers on the table.
I agree. Taking more than the requisite ~6 hours to do a full on-site for interview is a non-starter. I can do a take-home over the weekend, but my job has deadlines, and I don't want to be noticed.
Especially if I'm in full-on hunt-mode, I might be doing a couple interviews a week. This is totally infeasible.
I'm all for ditching the canned CTCI rigamarole, but please point to something that respects the candidate.
Especially if I'm in full-on hunt-mode, I might be doing a couple interviews a week. This is totally infeasible.
I'm all for ditching the canned CTCI rigamarole, but please point to something that respects the candidate.
Hiring is not a one size fits all problem. It's true that early stage startups could probably use more product-minded engineers (https://blog.pragmaticengineer.com/the-product-minded-engine...) and engineers who can ship code fast with the drawback being that it's not as scalable. Even then, different teams have different needs. Some need people who just want requirements and go. Some teams need junior developers, some need senior.
Half of the article is just an advertisement of their service.
The entire article is an advertisement of their service, and it is a pretty well executed advertisement.
By the time I got to the end, I realized it was an advertisement, but still felt I had learned something useful.
By the time I got to the end, I realized it was an advertisement, but still felt I had learned something useful.
Thanks! Full disclosure -- it is my first advertisement, ever, so I'm glad you liked it :)
I would challenge the assumption that this is purely advertisement, though. In my head, this is not "buy our shit™" type of post -- in fact, there's no mention of buying the subscription anywhere. All there is is a link to a demo, with a valid challenge (not a fizzbuzz/fibonacci, but one you can actually have a lot of fun with), which we provide for free to anyone, w/o even requiring valid email -- it is up to you if you don't wanna reset the password later. Instead, the post is more of a "use the right metric -- here's our experience and what has worked for us" type of post.
Would you agree that this is a more fair assessment?
I would challenge the assumption that this is purely advertisement, though. In my head, this is not "buy our shit™" type of post -- in fact, there's no mention of buying the subscription anywhere. All there is is a link to a demo, with a valid challenge (not a fizzbuzz/fibonacci, but one you can actually have a lot of fun with), which we provide for free to anyone, w/o even requiring valid email -- it is up to you if you don't wanna reset the password later. Instead, the post is more of a "use the right metric -- here's our experience and what has worked for us" type of post.
Would you agree that this is a more fair assessment?
I would argue that the whole article is an advertisement. The second half is just where they admit it. How does this kind of trash make the front page?
Are all advertisements trash?
The article introduces a problem and a solution that they came up with. Are you saying it's trash just because they have a chance of making money off it? It's ok to have a conflict of interest, imo - the work should stand for itself, and in this case I feel it was a good article.
The article introduces a problem and a solution that they came up with. Are you saying it's trash just because they have a chance of making money off it? It's ok to have a conflict of interest, imo - the work should stand for itself, and in this case I feel it was a good article.
The article introduces a problem as though the author is a neutral third-party, and that's slimey. I generally enjoy write-ups from companies that explain why they exist. Reading this one felt like being invited over for dinner and then finding out it's a timeshare pitch.
I’m not sure better coding exercises is the solution here. What you want to do is bring people in for a week and let them work with the team. But unless you’re the hottest thing in town, no one is going to spend a week with you, at least most likely not the people you want to hire.
I guess the argument is that the restaurant industry does it all the time - hell, hiring a designer will often involve assigning them a project, having them give a presentation on it, and asking questions about their portfolio.
We tend to put more trust in end product in that regard more than any specific process (like quizzing them on the process to create a specific animation in After Effects or on interaction cost or cognitive ergonomics).
We tend to put more trust in end product in that regard more than any specific process (like quizzing them on the process to create a specific animation in After Effects or on interaction cost or cognitive ergonomics).
So... restaurants hire people by giving them onions to chop and seeing if they can make good jokes. Every white collar job that's ever existed hires people by checking resumes, checking references, considering education and past work experience and a face-to-face introductory meeting. So, is programming (sorry, "engineering") more like cooking food or more like every other job where you sit in an office and work toward delivering business value?
Amazingly enough, restaurants don’t see if chefs can grow food and raise cattle, yet big tech does expect you to be able to reverse a binary tree on the whiteboard.
I know someone that was hired by Plex. He spent a couple of months as a contractor before they hired him.
I assume it worked out, because he's still there.
I was a hiring manager for a long time. I seemed to make good decisions, most of the time (a couple of notable exceptions). I kept fairly senior-level employees on board for decades.
I don't know if I ever developed a "system." It was always important for me to establish as much of a relationship with the prospective employees as possible, up front; which meant treating them with respect, and being open and emotionally available to them. I found that an attitude of trust and respect goes a long way to figuring out how people will deal with you and your team.
Tests were worthless. I loved portfolios, and discussing the details of projects they had done. I did a great deal of homework before each interview; carefully reviewing their resume, and finding out about as much as I could.
From all that, I have learned to be an "open book," myself. There's really not much about me that can't be figured out with some quick googling. That's on purpose.
But this seems to be more or less an advertisement for their company. It may be a good idea and a good company, but I can't really say.
I assume it worked out, because he's still there.
I was a hiring manager for a long time. I seemed to make good decisions, most of the time (a couple of notable exceptions). I kept fairly senior-level employees on board for decades.
I don't know if I ever developed a "system." It was always important for me to establish as much of a relationship with the prospective employees as possible, up front; which meant treating them with respect, and being open and emotionally available to them. I found that an attitude of trust and respect goes a long way to figuring out how people will deal with you and your team.
Tests were worthless. I loved portfolios, and discussing the details of projects they had done. I did a great deal of homework before each interview; carefully reviewing their resume, and finding out about as much as I could.
From all that, I have learned to be an "open book," myself. There's really not much about me that can't be figured out with some quick googling. That's on purpose.
But this seems to be more or less an advertisement for their company. It may be a good idea and a good company, but I can't really say.
[deleted]
"We automatically test not only that the solution complies with the problem definition, but that it works correctly, that it handles edge cases and errors, that it replies within latency tolerance, and that it keeps working when we throw a lot of load at it.
The candidate only has access to the initial test that gives them early feedback on how they are doing. The rest of the stages are visible only to the hiring manager."
I almost love this, except for the second paragraph. It sounds as if you can't see how well your code performed in latency, load tests, or edge cases. That does not seem like a realistic simulation of work, because normally one would need to iterate a little bit to optimize those things.
I mean I know I would prefer to just do an average job passing the core tests before I try to completely optimize it. Are the candidates even informed that the latency and load tests exist?
I think it could be okay if you just mention that is part of it and there is a straightforward way for them to do their own latency and load testing.. so hopefully that's the case?
The candidate only has access to the initial test that gives them early feedback on how they are doing. The rest of the stages are visible only to the hiring manager."
I almost love this, except for the second paragraph. It sounds as if you can't see how well your code performed in latency, load tests, or edge cases. That does not seem like a realistic simulation of work, because normally one would need to iterate a little bit to optimize those things.
I mean I know I would prefer to just do an average job passing the core tests before I try to completely optimize it. Are the candidates even informed that the latency and load tests exist?
I think it could be okay if you just mention that is part of it and there is a straightforward way for them to do their own latency and load testing.. so hopefully that's the case?
I don't agree with the author's (salesman's) conclusion. Most software engineers in a small company are going to do a lot of design. There won't be architects or principal engineers doing the design for you.
The kind of "chef" you interview by asking to chop an onion or cook for a few days is not the type of chef you are probably going to ask to design the menu. You're not making the menu, it's an almost pure execution role.
One of the things that can really set a great engineer apart is the ability to design elegant systems and manage complexity. You're likely not going to find that out coding with them for a few days unless you make sure to index for it.
Ironically, bigger companies where much of the design is centralized would benefit from this approach much more. They can get a lot more out of a "grunt" engineer who does journeyman work and is careful about testing.
The kind of "chef" you interview by asking to chop an onion or cook for a few days is not the type of chef you are probably going to ask to design the menu. You're not making the menu, it's an almost pure execution role.
One of the things that can really set a great engineer apart is the ability to design elegant systems and manage complexity. You're likely not going to find that out coding with them for a few days unless you make sure to index for it.
Ironically, bigger companies where much of the design is centralized would benefit from this approach much more. They can get a lot more out of a "grunt" engineer who does journeyman work and is careful about testing.
Yes, this is an ad for their service. But then, you are on HN, not on Lobsters.
I think that this is great idea that you're trying this. I hope you stick with it and succeed. Because it would be great to be able to hire and be hired, where you're not competing on how cheap you are or some other demeaning metric.
Godspeed.
I think that this is great idea that you're trying this. I hope you stick with it and succeed. Because it would be great to be able to hire and be hired, where you're not competing on how cheap you are or some other demeaning metric.
Godspeed.
> Small and medium companies pick the crumbs of those rituals and fall into cargo culting. They want to reap the same promised benefits of the obscure interview rituals. Yet they miss the reasons behind these rituals and do not fully understand what trained interviewers exactly seek.
Just my opinion, but I think this line comes across as a little too condescending or perhaps paternalistic. I work at a small (40-50 ppl) unknown to you company and all of us (so far) are senior in our fields and largely have FAMGAN experience on our CVs. We’re not just cargo culting, but rather taking things from the FAMGAN hiring processes that make sense and using them.
Just my opinion, but I think this line comes across as a little too condescending or perhaps paternalistic. I work at a small (40-50 ppl) unknown to you company and all of us (so far) are senior in our fields and largely have FAMGAN experience on our CVs. We’re not just cargo culting, but rather taking things from the FAMGAN hiring processes that make sense and using them.
Hey, post author here. Apologies if it sounded paternalistic, that was definitely not the intention.
I would argue that 40-50 senior people in one place would be rather an exception from the norm, and your company would be quite an outlier on the bell curve. If all of you are not just seniors, but also had first-hand experience participating (or even building) in hiring loops of the large companies, then you would have all the necessary knowledge of why those rituals were created, and why. The point of the blog post was that this is usually not the case, so the imitation is the only option.
Could you elaborate on your hiring loop? Which of the stages is the most effective and brings you the most value?
I would argue that 40-50 senior people in one place would be rather an exception from the norm, and your company would be quite an outlier on the bell curve. If all of you are not just seniors, but also had first-hand experience participating (or even building) in hiring loops of the large companies, then you would have all the necessary knowledge of why those rituals were created, and why. The point of the blog post was that this is usually not the case, so the imitation is the only option.
Could you elaborate on your hiring loop? Which of the stages is the most effective and brings you the most value?
...and if you share the same kind of jokes and values.
Cultural alignment No less important is the cultural alignment, the “Can I work with you?” question.
This screams all sorts of “isms” to me.
Cultural alignment No less important is the cultural alignment, the “Can I work with you?” question.
This screams all sorts of “isms” to me.
I love if I can share jokes with coworkers, but making that a prerequisite for hiring is kind of brain-dead.
In the real world that’s how it is, everyone has their own moral compasses
How do they prevent people sending the problem to a friend or a paid advisor and, thus, letting someone else solve it for you?
"The same happens with time-constrained problem-solving. That’s not how real life happens. People need time to think and reflect on the problems they face to build the right solution. Because of this, we do not artificially limit time for solving a challenge."
sounds very critical in that regard. If you have lots of time, that makes it much easier to get outside advice.
"The same happens with time-constrained problem-solving. That’s not how real life happens. People need time to think and reflect on the problems they face to build the right solution. Because of this, we do not artificially limit time for solving a challenge."
sounds very critical in that regard. If you have lots of time, that makes it much easier to get outside advice.
I was like: No. No. No. Up until I reached “Cultural alignment”! Then I was like: hell no! You know who does “Cultural alignment”? All the dictatorships in the world!
I once worked with a guy that was very difficult. People warned me ahead of time. I made sure to accommodate him and we got it done. Otherwise, he would not fit anywhere, he would be jobless and then what?
If you don’t respect the people’s right to work, you exclude them from society. And then what do you think it will happen?
I once worked with a guy that was very difficult. People warned me ahead of time. I made sure to accommodate him and we got it done. Otherwise, he would not fit anywhere, he would be jobless and then what?
If you don’t respect the people’s right to work, you exclude them from society. And then what do you think it will happen?
I wonder if other industries spend quite as much time focusing on how to hire exceptional people? Do we see articles in "The Stage" magazine about how to hire the next Lin-Manuel Miranda for your next production? Do we see Human Resources industry magazine publish a thousand pieces about how to spot the next Hannibal Lecter to really boost your HR teams performance?
Does anyone else feel like it should be illegal to make someone go through 3 or 4 interview stages, essentially making them work for free, only to not give them the job?
On another note, if your code base uses very niche software and there aren't too many people out there who know it, what's the process for hiring an engineer that will be able to learn it? What do you look for?
On another note, if your code base uses very niche software and there aren't too many people out there who know it, what's the process for hiring an engineer that will be able to learn it? What do you look for?
I don't think you can sensibly say that most interviews are making people "work" for free- you're not going to get any productive work product out of an interview unless you're literally making someone do a trial shift in a coffee shop, which I think people would agree is scummy. That doesn't really happen in Tech. I can pay you to do our coding test, but I don't think anyone can reasonably argue that 350 different solutions to the exact same toy programming problem has any value to my company. Also, what am I going to pay you? Minimum wage? £20 or so? We could do that, but I'm not sure it would make any difference to anyone but the most marginal candidate.
[deleted]
The difference is that you get paid at a restaurant for chopping onions, even if it is for three days only. A highly valued IC wouldn't have time to spend three days coding for free. On the other hand someone can outsource this to a skilled developer in India or a friend who is good at coding.
> A highly valued IC wouldn't have time to spend three days coding for free.
So pay them, even if it's just a token amount. At least shows some appreciation for the candidate sacrificing their time.
So pay them, even if it's just a token amount. At least shows some appreciation for the candidate sacrificing their time.
Anyone try their "demo"? Is it any good?
(Oh, autoiterative people? Where's your privacy policy? Where's your opt-out of tracking link? For that matter, where's your company? And why is your domain registered via WhoisGuard out of Panama?)
(Oh, autoiterative people? Where's your privacy policy? Where's your opt-out of tracking link? For that matter, where's your company? And why is your domain registered via WhoisGuard out of Panama?)
Hey John, how about you try it yourself? For the right mindset, I've been told that it is very fun and addictive :)
Thanks for the questions, let me elaborate:
Thanks for the questions, let me elaborate:
- I didn't have time to prepare privacy policy, but we'll indeed make one and post it. You're right, we should have it.
- There is no tracking. Never will be. We'll blog about why we're for 100% anonymity of the candidates in more detail, but for now: We don't collect names. We don't require valid emails to enroll in the demo. We don't use tracking cookies. We don't send HTML emails with tracking pixels, and we don't use any 3rd party service. We use cookies to authenticate you, and that's it. Javascript, social media buttons, or trackers or analytics of any sort on our pages are considered a bug. What we don't have, is a proper explicit privacy policy explaining all that.
- We're a European company.
- That's how Namecheap sets it up -- curious, why do you think it is a bad thing?I'll copy-paste and expand something I wrote earlier about interviewing [0]. I think you are spot-on that algorithmic interviews are a form of cargo culting and that they select for the wrong thing but that's not because they are algorithmic interview; it's because they are poorly conducted.
I'll play devil's advocate and defend whiteboard interviews as compared to take home or online assessments. Take homes are not very respectful of the candidate's time and, especially if they are not time bound, often leads to a candidate wasting a significant amount of time on these assignments. Plus, there's always the possibility that desperate candidates would cheat and have the assignment done by someone else.
I would only use take homes and HackerRank style quiz for applicants that are from institutions I don't fully trust with quality or applications that are very spammy in nature, i.e from a college/bootcamp that has an abysmal application-to-hire ratio [1].
For candidates from serious institutions I favor a two-step approach: a screening interview with a light coding question and a full whiteboard interview. The screening's purpose is really only to answer questions about the hiring process and check that the candidate can write very simple code by himself when given a trivial problem [2]. Something not more complicated than FizzBuzz really.
A lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course, it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can't make any assumptions on what the candidates know.
People, especially non-technical, over-emphasis on knowledge of specific languages and framework and disregard computer science fundamentals, and I've been doing the exact opposite. I'm not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.
Now, large tech companies optimize hiring at the college level because that's when everyone is on the market at the same time [3]. It's also why they have internship programs, to make sure that the best talent is already in the hiring pipelines a few years before even getting their diplomas. After that initial post-graduation job search phase, you'll see great engineers often spending several years at the same company and not even looking for a job. They are often the strongest candidates and totally invisible to recruiters.
Lastly, you can either really hope hard to find a diamond in the rough at whichever market rate you set or decide to compete with the companies that are actually attracting top talent. There's really no secret formula it's: compensation, interesting projects, technical career path and technical management. You can try to play games with CoL or market rate, but just keep in mind that if you do some of the top applicants won't even bother sending a resume in the first place.
[0] https://news.ycombinator.com/item?id=23989466
[1] https://economictimes.indiatimes.com/tech/ites/95-engineers-...
[2] https://blog.codinghorror.com/why-cant-programmers-program/
[3] https://www.inc.com/magazine/20070501/column-guest.html
I'll play devil's advocate and defend whiteboard interviews as compared to take home or online assessments. Take homes are not very respectful of the candidate's time and, especially if they are not time bound, often leads to a candidate wasting a significant amount of time on these assignments. Plus, there's always the possibility that desperate candidates would cheat and have the assignment done by someone else.
I would only use take homes and HackerRank style quiz for applicants that are from institutions I don't fully trust with quality or applications that are very spammy in nature, i.e from a college/bootcamp that has an abysmal application-to-hire ratio [1].
For candidates from serious institutions I favor a two-step approach: a screening interview with a light coding question and a full whiteboard interview. The screening's purpose is really only to answer questions about the hiring process and check that the candidate can write very simple code by himself when given a trivial problem [2]. Something not more complicated than FizzBuzz really.
A lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course, it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can't make any assumptions on what the candidates know.
People, especially non-technical, over-emphasis on knowledge of specific languages and framework and disregard computer science fundamentals, and I've been doing the exact opposite. I'm not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.
Now, large tech companies optimize hiring at the college level because that's when everyone is on the market at the same time [3]. It's also why they have internship programs, to make sure that the best talent is already in the hiring pipelines a few years before even getting their diplomas. After that initial post-graduation job search phase, you'll see great engineers often spending several years at the same company and not even looking for a job. They are often the strongest candidates and totally invisible to recruiters.
Lastly, you can either really hope hard to find a diamond in the rough at whichever market rate you set or decide to compete with the companies that are actually attracting top talent. There's really no secret formula it's: compensation, interesting projects, technical career path and technical management. You can try to play games with CoL or market rate, but just keep in mind that if you do some of the top applicants won't even bother sending a resume in the first place.
[0] https://news.ycombinator.com/item?id=23989466
[1] https://economictimes.indiatimes.com/tech/ites/95-engineers-...
[2] https://blog.codinghorror.com/why-cant-programmers-program/
[3] https://www.inc.com/magazine/20070501/column-guest.html
I feel like the crux here is requiring real work for 3-4 days. In addition to screening time and so on.
A few days of a truly great engineers time costs like 4K-5k. At least.
A few days of a truly great engineers time costs like 4K-5k. At least.
Actually if you think differently it’s better. This is like diverse team skills. If you are the same what difference does you bring to the team?
It's not as straightforward as 'thinking differently'. Diversity of knowledge, experience, and skill is extraordinarily valuable but if you...
* hoard knowledge
* refuse to elevate those around you
* are frequently combative
* lie about your knowledge/experience
* seek to be right instead of discover optimal solutions
... then all of that diversity is lost or wasted.
It's not about diversity for diversity's sake, it's about sharing in diversity and using all of the differing perspectives to reach optimal solutions that your organization would be blind to otherwise.
* hoard knowledge
* refuse to elevate those around you
* are frequently combative
* lie about your knowledge/experience
* seek to be right instead of discover optimal solutions
... then all of that diversity is lost or wasted.
It's not about diversity for diversity's sake, it's about sharing in diversity and using all of the differing perspectives to reach optimal solutions that your organization would be blind to otherwise.
[deleted]
[deleted]
What's the blogging engine? Looks familiar
I suspect Hugo: https://autoiterative.com/blog/index.xml
You're spot on. We use hugo with a custom theme, which is derived from etch (https://themes.gohugo.io/etch/).
[deleted]
Hey folks! co-founder here, I'm one of the people behind autoiterative.com. I did a ShowHN yesterday[1], but obviously, the blog post provokes much more engaged discussion, love it ;) I appreciate your feedback, please keep it coming and challenge our assumptions!
I will try to address the questions tomorrow, I already see some of them require a ton of thinking indeed.
[1]: https://news.ycombinator.com/item?id=23986002
I will try to address the questions tomorrow, I already see some of them require a ton of thinking indeed.
[1]: https://news.ycombinator.com/item?id=23986002
Hi Ilya, I saw in your linked post that this started as an internal tool to assess candidates which you then spun out. Could you talk more about that? How many people did you screen and hire with this tool?
Hi! Thanks for asking. We will blog about it in more detail, but I can give you some numbers right away. At the moment when we decided to start making it into a product, we've been using this approach for roughly 4 months. During this period, the data was:
- we processed roughly 600 candidates
- of which we hired about 50
- per-recruiter rate of processing people increased from 15/month to 100/month
- average invite-to-offer time reduced from two months to two weeks
- on top of that, the number of challenges to review by our engineers dropped by 60%. The explanation of why will be in our next blog post about the cost of engineering engagement.
At this point, we decided that this is too good to hold only for ourselves. Thus, AutoIterative was born, where we also significantly improved the parts that weren't polished well enough and added features we lacked.
At this point, we decided that this is too good to hold only for ourselves. Thus, AutoIterative was born, where we also significantly improved the parts that weren't polished well enough and added features we lacked.
This is very, very similar to something I've been working on. I'll email you to trade thoughts, if you're interested.
Hey juped, of course, please do reach out, I would love to hear your take on this problem because it is a very complex one indeed.
Some context: we're not claiming we have a silver bullet or fit-it-all solution as some people got from the post (and that's my fault because I'm responsible for that text, and blogging is hard, haha). And we're also not claiming we can replace _the whole_ of your hiring loop (that would be the silver bullet).
We claim that choosing the right metric can drastically improve the hiring loop, and we've built the tool that automates the assessment of this exact metric.
Some context: we're not claiming we have a silver bullet or fit-it-all solution as some people got from the post (and that's my fault because I'm responsible for that text, and blogging is hard, haha). And we're also not claiming we can replace _the whole_ of your hiring loop (that would be the silver bullet).
We claim that choosing the right metric can drastically improve the hiring loop, and we've built the tool that automates the assessment of this exact metric.
- They are probably not going to be able to hire "great" engineers
- Even if they get a couple of great engineers, they are not going to be able to create an engineering culture where those engineers can do great work.
Instead of looking for "greatness", most companies in this position should simply settle for competence. Hire people who use boring technology and build systems that do things simply and directly.