Ask HN: What's with all the unpaid interview projects?
80 comments
I couldn't agree more - and it impact how we hire. I refuse to give someone a "take home test"; quite frankly it reaks of a lack of respect for the candidate. Interviewing is two-sided, the organization has to check out the candidate and the candidate has to check out the organization.
Our last hires were brought in for a two-part interview, on the same day. The first was an interview to ensure the personality fit was there, to ensure that the differences we all bring to the table could be a strength.
The candidates were then given a running API server, a postman collection, swagger docs, and an empty directory. We asked them to solve a specific problem - one that we already showed the candidate is solved in production, one we make clear cannot be solved in the two hours they're given. Then, the candidate is encouraged to work on the problem, and ask questions.
Multiple team members check in on the candidate throughout the period, in order to offer support, suggestions, pick things apart, and be a sounding board. Then, when things are done we sit down as a minimum of three people (candidate + 2 staff) to discuss the output and outcome.
This isn't the most fair, and can definitely be stressful. At the same time, I feel like it's pretty equitable.
Our last hires were brought in for a two-part interview, on the same day. The first was an interview to ensure the personality fit was there, to ensure that the differences we all bring to the table could be a strength.
The candidates were then given a running API server, a postman collection, swagger docs, and an empty directory. We asked them to solve a specific problem - one that we already showed the candidate is solved in production, one we make clear cannot be solved in the two hours they're given. Then, the candidate is encouraged to work on the problem, and ask questions.
Multiple team members check in on the candidate throughout the period, in order to offer support, suggestions, pick things apart, and be a sounding board. Then, when things are done we sit down as a minimum of three people (candidate + 2 staff) to discuss the output and outcome.
This isn't the most fair, and can definitely be stressful. At the same time, I feel like it's pretty equitable.
>> Our last hires were brought in for a two-part interview, on the same day. The first was an interview to ensure the personality fit was there, to ensure that the differences we all bring to the table could be a strength. The candidates were then given a running API server, a postman collection, swagger docs, and an empty directory. We asked them to solve a specific problem - one that we already showed the candidate is solved in production, one we make clear cannot be solved in the two hours they're given. Then, the candidate is encouraged to work on the problem, and ask questions.
As someone who did a few interviews last year, I wish they had been like this. I walked away from most interviews when they started asking me things like:
- "I dont really have time to think up any interview questions or a test so can you bring some code from your last job and we'll take through it..."
- Asked me to do a 2 hour online test without having even spoken to me in any capacity, or asked me to do a homework quiz "it should take around 10 hours" before having spoke to me in any capacity....and they were the ones who contacted me initially.
My favorite was hiring me as a contractor on a short term basis, paying market rates, and then using this time for us to see if we were a good fit.
As someone who did a few interviews last year, I wish they had been like this. I walked away from most interviews when they started asking me things like:
- "I dont really have time to think up any interview questions or a test so can you bring some code from your last job and we'll take through it..."
- Asked me to do a 2 hour online test without having even spoken to me in any capacity, or asked me to do a homework quiz "it should take around 10 hours" before having spoke to me in any capacity....and they were the ones who contacted me initially.
My favorite was hiring me as a contractor on a short term basis, paying market rates, and then using this time for us to see if we were a good fit.
I appreciate this. The reality though is that there's no setup cost for this. Our team always has a back end that generates swagger - and swagger to postman is a quick command. The most "expensive" part of the setup was creating a new user on the laptop that sat on the desk.
It sounds pretty reasonable. Kudos for doing all the setup beforehand for a smooth experience, and for defining the parameters of the problem and bounding the space for solving it. A guarantee of feedback and clarity around when, where and how that feedback will be given is great for the candidate. I know it already costs a lot to have so many people from your team involved, since you pay for their time (bravo for investing in your process). Is there some way you'd be able to work in compensating candidates for their time as well? What kind of volume do you typically need to put through such a nice-sounding process in order to get a good match?
There's no real cost if you think about it. It's cheaper to invest in the 1-3 candidates you want to bring in and actually hire (i.e they pass the personality fit) than it is to make the wrong hire.
We absolutely don't compensate candidates for their time. From an economics perspective there's no value in the output, and the shared value is the opportunity for each side to determine if we have an alignment on fit. Each side makes an investment with an outcome based goal - that's why this is as equitable as we could make it.
We absolutely don't compensate candidates for their time. From an economics perspective there's no value in the output, and the shared value is the opportunity for each side to determine if we have an alignment on fit. Each side makes an investment with an outcome based goal - that's why this is as equitable as we could make it.
well, you had me until this post.
your volume is so low you can absolutely afford to pay candidates. given that your setup process is also automatic, it's offensive that you aren't.
a candidate 'invests' their entire day's output in it and gets paid $0.
your team 'invests' their entire day in the project and gets paid $XYZ dollars.
are these equal "investments"? obviously not.
you can afford it - paying candidates will be a business write off for you anyways.
you're just not doing it because... reasons?
really lost me with this post, it doesn't sound compassionate at all. you're using candidates the same way everyone else is, but the bar is so low that describing the process and giving feeedback got me excited.
a candidate 'invests' their entire day's output in it and gets paid $0.
your team 'invests' their entire day in the project and gets paid $XYZ dollars.
are these equal "investments"? obviously not.
you can afford it - paying candidates will be a business write off for you anyways.
you're just not doing it because... reasons?
really lost me with this post, it doesn't sound compassionate at all. you're using candidates the same way everyone else is, but the bar is so low that describing the process and giving feeedback got me excited.
I don't expect to be paid when I'm investing my time in my future, any more than I'd expect to pay the organization for giving me the opportunity to explore a future.
There's an equal give and take during the interview process - at least if two parties understand that they're interviewing each other. For those that don't - neither rhetoric, nor money can instill those values.
There's an equal give and take during the interview process - at least if two parties understand that they're interviewing each other. For those that don't - neither rhetoric, nor money can instill those values.
[deleted]
Small companies as well do this. I’ve had an interview where the CEO just called in their lead infrastructure guy to join the “conversation” and it turned out to be a free consultation session which I could have easily charged a good number on a normal day.
After almost 5 hours of this interview I was told I’ll recieve a response on mail.
2 years later still nothing.
Company in question: Skalogs/Sentai
wow 5 hours.. 2 years..
I follow a simple rule about this when I'm the interviewer - any time commitment asked of a job candidate to write code, must be matched by an equal time commitment on my part.
For example, that permits collaborative code questions or white boarding, but not take home code challenges. Past experience has shown me those are always horribly estimated, and require more time on the candidate's part than issuers claim. Put yourself in the candidate's shoes, having 3-5 of these things to work through (unpaid, no doubt, as OP rightfully objects to). Not fun!
I've found that rule a simple, transparent, way of showing respect for candidates' time. Which I've always found admirable, from both sides of the hiring table.
For example, that permits collaborative code questions or white boarding, but not take home code challenges. Past experience has shown me those are always horribly estimated, and require more time on the candidate's part than issuers claim. Put yourself in the candidate's shoes, having 3-5 of these things to work through (unpaid, no doubt, as OP rightfully objects to). Not fun!
I've found that rule a simple, transparent, way of showing respect for candidates' time. Which I've always found admirable, from both sides of the hiring table.
Thanks for committing time and communicating clearly. Short of getting paid, it sounds like you're respecting candidates, so that's something! :)
Unpopular opinion: Interview problem is a solved problem for software engineer candidates for majority of use cases, it is called Data Structures and Algorithm.
Yes I've tried the other two: take home tests, work one day at company X. They all suck. They all optimized for the suffering of the software engineer candidates. You can't do multiple companies take home tests, it is a waste of time. You can't do multiple work one day at company X, Y, Z. Waste of time. And after all of those time wasters they still reject you for non obvious reasons such as cultural fit, pay, seniority level, etc.
Just cut to the chase, give me DS & A questions and be done with it.
Data Structures and Algorithms are the best mean to interview. Study once, apply multiple times to multiple companies and kill those and get multiple offers, with bonus it makes you a better engineer (at least in my case), though later it gives diminishing returns. You can jump companies every year easily with this skill.
But then people will ask, "how does that gauge software engineering skills?" or "what if you aren't good at data structures and algorithms but you are good at something else like design, product, css, etc" or "there are more to working than just data structures and algorithms" or "you will get similar people in your talent pipeline".
I don't know, that's not the problem of the software engineer candidates, that's the problem of the companies. Let them deal with it. We software engineer candidates just play the game, get the job or rejected from the job, and do the job, go home, collect paycheck, sleep, play, exercise, use our mind, talent and energy doing something else.
Yes I've tried the other two: take home tests, work one day at company X. They all suck. They all optimized for the suffering of the software engineer candidates. You can't do multiple companies take home tests, it is a waste of time. You can't do multiple work one day at company X, Y, Z. Waste of time. And after all of those time wasters they still reject you for non obvious reasons such as cultural fit, pay, seniority level, etc.
Just cut to the chase, give me DS & A questions and be done with it.
Data Structures and Algorithms are the best mean to interview. Study once, apply multiple times to multiple companies and kill those and get multiple offers, with bonus it makes you a better engineer (at least in my case), though later it gives diminishing returns. You can jump companies every year easily with this skill.
But then people will ask, "how does that gauge software engineering skills?" or "what if you aren't good at data structures and algorithms but you are good at something else like design, product, css, etc" or "there are more to working than just data structures and algorithms" or "you will get similar people in your talent pipeline".
I don't know, that's not the problem of the software engineer candidates, that's the problem of the companies. Let them deal with it. We software engineer candidates just play the game, get the job or rejected from the job, and do the job, go home, collect paycheck, sleep, play, exercise, use our mind, talent and energy doing something else.
I agree with some of your points. The problem is that a lot of engineers are just not good at DS puzzles under that kind of pressure in that amount of time. Also that's totally unrealistic and is not how Software Engineering works in real life. In real life you have group discussion, design sessions, time to think without the pressure of having to talk the whole time spitting thoughts, etc. I think some companies are starting to do pair programming and design sessions during the interviews. That sounds more appropriate to me. But it's also more expensive and time consuming.
If I can give you some encouragements,
I don't consider myself good in DS&A puzzles. My definition of good in DS&A puzzles is basically you can solve common/most Hard Leetcode problems in under 45 mins. I think if you can do this you'll have no problem jumping ship of FAANG-league companies every year. I am still not in FAANG.
My skill now is pretty much in Medium Leetcode problems, which I can solve common/most of them in under 45 mins. I used to be worse at this and couldn't even solve Easy problems for 2 hours. I just practiced and practiced and studied, until it becomes easier.
Now, what this skill gives me is, I can easily knock off majority of non FAANG interviews out there, in under 30 mins. In my last interviewing with companies I actually did most of their questions each in under 15 mins. This leaves them to talk about something else such as projects, past responsibilities, cultural fit, or even just dismiss the interview altogether to save everyone time.
DS&A interview is in a sense, sort of a pair programming/discussion thing because you need to voice your reasoning out loud to check with the interviewer that you are going in the right direction.
Also once you practiced these kinds of questions, then it is no longer "I need to find a new solution to this problem", but more "ah I see this question before and I know how to solve this". It is pretty much almost brain-dead at this moment. Like, dynamic programming, recursion, and BFS/DFS are pretty much very mechanical. Once you've seen a few patterns, then you can tackle most of it.
Yes this skill doesn't come easily, but once you get it, you get it. So the difficulty of attaining this skill is overrated. It is like swimming and driving. Once you get it, it takes a few weeks to ramp up again. Weighing the pros and cons of DS&A compared to other style of interviews, it is clear to me which is the winner to optimize my happiness and lessen my sufferings. It is cheap to execute, it works majority of the time.
If a software engineer realize this reality and start playing the game, he/she is already ahead in the curve by a few miles from other candidates.
Ironically, I actually don't want most software engineers to realize this, because this skill currently gives me an advantage. If everyone start doing this, then the interview bar will be raised higher and we as an industry will see other practices which are optimized for software engineering candidates suffering.
I don't consider myself good in DS&A puzzles. My definition of good in DS&A puzzles is basically you can solve common/most Hard Leetcode problems in under 45 mins. I think if you can do this you'll have no problem jumping ship of FAANG-league companies every year. I am still not in FAANG.
My skill now is pretty much in Medium Leetcode problems, which I can solve common/most of them in under 45 mins. I used to be worse at this and couldn't even solve Easy problems for 2 hours. I just practiced and practiced and studied, until it becomes easier.
Now, what this skill gives me is, I can easily knock off majority of non FAANG interviews out there, in under 30 mins. In my last interviewing with companies I actually did most of their questions each in under 15 mins. This leaves them to talk about something else such as projects, past responsibilities, cultural fit, or even just dismiss the interview altogether to save everyone time.
DS&A interview is in a sense, sort of a pair programming/discussion thing because you need to voice your reasoning out loud to check with the interviewer that you are going in the right direction.
Also once you practiced these kinds of questions, then it is no longer "I need to find a new solution to this problem", but more "ah I see this question before and I know how to solve this". It is pretty much almost brain-dead at this moment. Like, dynamic programming, recursion, and BFS/DFS are pretty much very mechanical. Once you've seen a few patterns, then you can tackle most of it.
Yes this skill doesn't come easily, but once you get it, you get it. So the difficulty of attaining this skill is overrated. It is like swimming and driving. Once you get it, it takes a few weeks to ramp up again. Weighing the pros and cons of DS&A compared to other style of interviews, it is clear to me which is the winner to optimize my happiness and lessen my sufferings. It is cheap to execute, it works majority of the time.
If a software engineer realize this reality and start playing the game, he/she is already ahead in the curve by a few miles from other candidates.
Ironically, I actually don't want most software engineers to realize this, because this skill currently gives me an advantage. If everyone start doing this, then the interview bar will be raised higher and we as an industry will see other practices which are optimized for software engineering candidates suffering.
it's a stupid game but you're smart to play it.
but if DS&A puzzles are a de-facto standard, why not have an actual standard? the whole thing is pretty haphazard.
it's like hiring someone to be your defense attorney because they're decent at the NYT Crossword Puzzle, or to be your doctor because they're good at Sudoku.
but if DS&A puzzles are a de-facto standard, why not have an actual standard? the whole thing is pretty haphazard.
it's like hiring someone to be your defense attorney because they're decent at the NYT Crossword Puzzle, or to be your doctor because they're good at Sudoku.
It is like most things in life or software, no silver bullet. Every team/product/company has different requirements that they need to fulfill, including off-the-mark measures such as diversity quota. Having an actual standard will make it difficult. Besides, this field moves so much nobody wants to get certifications that will be obsolete 3 months down the road. Not to mention certifications and credentials can be bad gatekeeper in itself.
This game is actually really easy to play, much easier than any other corporate/job game in other fields. You just need to know the rules, study the rules and get on with your life. You don't need bachelor/master from expensive universities or any other gate-keeping bst like in medical or law.
I wasn't born rich or smart or have good connections, and I'm glad setting aside a few months to study DS&A can make me earn what most Americans or the world envy. Much more if I'm good at it so I can jump ship at FAANG every year and living the dream. Let the rich/smart/well-connected people solve harder problems such as medicine or law. I just get on with my life.
This game is actually really easy to play, much easier than any other corporate/job game in other fields. You just need to know the rules, study the rules and get on with your life. You don't need bachelor/master from expensive universities or any other gate-keeping bst like in medical or law.
I wasn't born rich or smart or have good connections, and I'm glad setting aside a few months to study DS&A can make me earn what most Americans or the world envy. Much more if I'm good at it so I can jump ship at FAANG every year and living the dream. Let the rich/smart/well-connected people solve harder problems such as medicine or law. I just get on with my life.
I've been open sourcing my take home tests:
https://github.com/tlhunter?utf8=%E2%9C%93&tab=repositories&...
Not sure if there's any legal implications with this, like, say I own the code but the company owns the "idea".
Not sure if there's any legal implications with this, like, say I own the code but the company owns the "idea".
Unless you've been paid for it (work for hire) your work product is your own.
Can you change that contractually in the US? I know here you can. And if you are a freelancer, you can have default company terms which say you own it all, even when paid. I had very few changing that default contract and my last few paid jobs rather did a ‘partnership’ as in shared ownership for a lower hourly with an option to buy outright for shares and/or money at a later date.
There are more ways to skin a cat. I am not sure big ( especially software) companies (in other, more litigious countries than mine) would entertain things like this, but if you do not try...
For instance, I did a small intranet for a very large company (for my country); because it was not their core business, and as such they did not care if we kept the (c) or sold the source to others as long as they had eternal perpetual rights to do whatever with it as long as they do not sell or use it outside their company. They reasoned that if I have more clients, they might get features and/or bugfixes for free. They did, up to some point and it made me turn my 1 man services company into a product company with staff. Which was fun but a steep learning curve, especially with regards to the money required for the growth of such a company.
Their lawyers made up a very fair contract (again this was not their core business so they genuinely knew they would never want to sell this themselves) which had clauses to protect them and us if they exited etc.
I had this happen far more often; at the right level, many companies, especially none software companies, will be up for this if they benefit financially or otherwise. Recently (different company; only services again) I had a large car maker suggesting we sell a small LoB app we made for them to their competitors so they would, again, share the financial pain of bugfixes and intellectual gain of new features. Kind of a product/services mix without the pressure of roadmaps, growth etc (all companies pay per hour and get all versions, including forks for others, but not forks by others unless the other parties specifically agree). So kind of contained-closed-open-source way of working, which is far easier for the corp lawyers to wrap their heads around than throwing things into the actual open. Also the quality demands code, community wise, are far less important. Let’s be real; most LoB software is a burning heap of crap; usually first put together parttime by someone of the departmental business and then passed to an external company when growing out of bounds (getting too much uptake internally). This time we could basically do a rewrite, but that is not always the case (remember; my business is being paid for every single hour).
There are more ways to skin a cat. I am not sure big ( especially software) companies (in other, more litigious countries than mine) would entertain things like this, but if you do not try...
For instance, I did a small intranet for a very large company (for my country); because it was not their core business, and as such they did not care if we kept the (c) or sold the source to others as long as they had eternal perpetual rights to do whatever with it as long as they do not sell or use it outside their company. They reasoned that if I have more clients, they might get features and/or bugfixes for free. They did, up to some point and it made me turn my 1 man services company into a product company with staff. Which was fun but a steep learning curve, especially with regards to the money required for the growth of such a company.
Their lawyers made up a very fair contract (again this was not their core business so they genuinely knew they would never want to sell this themselves) which had clauses to protect them and us if they exited etc.
I had this happen far more often; at the right level, many companies, especially none software companies, will be up for this if they benefit financially or otherwise. Recently (different company; only services again) I had a large car maker suggesting we sell a small LoB app we made for them to their competitors so they would, again, share the financial pain of bugfixes and intellectual gain of new features. Kind of a product/services mix without the pressure of roadmaps, growth etc (all companies pay per hour and get all versions, including forks for others, but not forks by others unless the other parties specifically agree). So kind of contained-closed-open-source way of working, which is far easier for the corp lawyers to wrap their heads around than throwing things into the actual open. Also the quality demands code, community wise, are far less important. Let’s be real; most LoB software is a burning heap of crap; usually first put together parttime by someone of the departmental business and then passed to an external company when growing out of bounds (getting too much uptake internally). This time we could basically do a rewrite, but that is not always the case (remember; my business is being paid for every single hour).
I graduated college last year and I did quite a few of these unpaid interview projects. They never took more than a hour and all of the projects were clearly not intended to actually be used in real life projects. The projects are clearly there to see if I could follow the written spec and write reasonably efficient code which I think is reasonable.
If the take home project looks like something the company would actually use in their codebase, then yeah I would probably ask for compensation.
If the take home project looks like something the company would actually use in their codebase, then yeah I would probably ask for compensation.
I think if you're just getting into the industry, and if the projects are actually of small scale, then they're arguably more reasonable. If you have a resume with stuff on it and can talk through it, it's less so. Most of the ones I've seen have required way more hours to do reasonably well than the estimated time, and it's ridiculous to have every candidate prove themselves equally with some trite project.
I feel like interviewers can’t win here: no matter what, Hacker News will complain about it. HN says whiteboard questions are terrible, leetcode is terrible, Github resumés are terrible, take-home projects are terrible... these are all defensible arguments in isolation, but what should interviewers do? There’s a lot of griping re: interviewing but few suggestions on what to do instead.
I can tell you what's worked for me when hiring experienced programmers:
Ideally the candidate offers code samples, I then pick through the code samples to find things that strike me as unusual or problematic and ask questions about those parts. This gives me a great deal of insight about how people think, and their relative level of skill. I firmly believe this is the best way to interview.
Failing that, candidates frequently provide details on projects they worked on. I'll then ask questions about those projects like, "What language did you develop it in and why?" and "What do you think was the most difficult part about this project and why?" These questions are a gateway to the same kind of details you get from code samples. For example, a candidate once mentioned he had worked for the Navy, and had done water simulations for them and I asked, "What's so difficult about water simulations?" and he gave me a 20 minute lecture on how water simulations work, which won him the job. It's probably worth noting that the job had nothing to do with water simulations, but his lecture made it clear he had passion and knowledge and could do the job.
The worst interviews are ones where the candidates worked in an environment where they can't offer code samples or talk about previous projects (military contractor, for example) - these you just kind of muddle through by talking about programming in general.
Ideally the candidate offers code samples, I then pick through the code samples to find things that strike me as unusual or problematic and ask questions about those parts. This gives me a great deal of insight about how people think, and their relative level of skill. I firmly believe this is the best way to interview.
Failing that, candidates frequently provide details on projects they worked on. I'll then ask questions about those projects like, "What language did you develop it in and why?" and "What do you think was the most difficult part about this project and why?" These questions are a gateway to the same kind of details you get from code samples. For example, a candidate once mentioned he had worked for the Navy, and had done water simulations for them and I asked, "What's so difficult about water simulations?" and he gave me a 20 minute lecture on how water simulations work, which won him the job. It's probably worth noting that the job had nothing to do with water simulations, but his lecture made it clear he had passion and knowledge and could do the job.
The worst interviews are ones where the candidates worked in an environment where they can't offer code samples or talk about previous projects (military contractor, for example) - these you just kind of muddle through by talking about programming in general.
> Ideally the candidate offers code samples
Code samples are not like blood or urine samples, you should have serious and public projects to sample from.
Many great programmers are not coding outside of the work environment, what should they provide according to your method ?
And for those who do, they can explain things but you can never be sure they actually wrote it or will be able to write at a similar level (well, that's partially true for take home tests as well)
Code samples are not like blood or urine samples, you should have serious and public projects to sample from.
Many great programmers are not coding outside of the work environment, what should they provide according to your method ?
And for those who do, they can explain things but you can never be sure they actually wrote it or will be able to write at a similar level (well, that's partially true for take home tests as well)
People lie about their credentials in every profession, why do you think in tech you're going to somehow be able to stop that?
The problem with talking about previous projects is that it can be hard to tell whether the candidate was an important contributor to the project or just a subpar developer making the changes (s)he's told to do by more knowledgeable teammates. A candidate which is charismatic and good at talking can be quite convincing.
Provided code samples is good, but only if the candidate has written all the code themselves and isn't just passing off other's code on a project as their own.
When I was conducting interviews I was frequently surprised by just how bad some candidates with fancy titles and a decade of experience were at actually coding. And I mean things like a understanding a simple recursive function, integer division, inheritance and other core concepts. I would never hire someone without at least a brief code comprehension check nowadays.
Provided code samples is good, but only if the candidate has written all the code themselves and isn't just passing off other's code on a project as their own.
When I was conducting interviews I was frequently surprised by just how bad some candidates with fancy titles and a decade of experience were at actually coding. And I mean things like a understanding a simple recursive function, integer division, inheritance and other core concepts. I would never hire someone without at least a brief code comprehension check nowadays.
The problem is you’d probably do just as poorly under the gun and away from your subject of expertise.
Your process sounds awesome. It's definitely the exception to the rule. It would be great to drill down and speak about the parts that are interesting in projects I've done. Maybe a paid project makes sense for your example of someone without an available codebase to share. Good luck!
It's more like different subsets of HN will complain about each and there's no one size fits all solution. What interviewers should do is take that into account.
Github resumes are great for some people who like working on OSS stuff but horrible for others who can't share comercial work and want to have a non-coding life outside of it.
Take home projects can be terrible if they're too early in the hiring pipeline and also exclude anyone currently working full time.
Whiteboard questions are mostly terrible because of nerves and poor input methods, at least do it on a laptop connected to a projector, then it's acceptable for some people.
The one thing you can't do is have a single process for everyone and pretend there's scientific rigor behind your decisions.
Github resumes are great for some people who like working on OSS stuff but horrible for others who can't share comercial work and want to have a non-coding life outside of it.
Take home projects can be terrible if they're too early in the hiring pipeline and also exclude anyone currently working full time.
Whiteboard questions are mostly terrible because of nerves and poor input methods, at least do it on a laptop connected to a projector, then it's acceptable for some people.
The one thing you can't do is have a single process for everyone and pretend there's scientific rigor behind your decisions.
"The one thing you can't do is have a single process for everyone and pretend there's scientific rigor behind your decisions."
Right on.
Right on.
For me the irritating part is, I haven't encountered any other professions where the interviewee has to "prove" themselves with this kind of nonsense (isn't that what the resume and references are for?), especially over and over again in the case of consultants and frequent job hoppers - it gets tiring and I avoid companies that try it now (unless it's a small trivial application/test).
I can understand it to a small degree, given the nature of software development. I wonder how other professions would take it - "Excuse me Sandra, I know your resume says you're an electrician, but I'm going to need you to wire up this house so I can just check". I'm sure some other professions do do this, but none I've heard of.
I can understand it to a small degree, given the nature of software development. I wonder how other professions would take it - "Excuse me Sandra, I know your resume says you're an electrician, but I'm going to need you to wire up this house so I can just check". I'm sure some other professions do do this, but none I've heard of.
I assume you've never conducted interviews, am I correct? There are so many "developers" out there that simply can't code, regardless of what their CV or education says. 15 years experience with Java development as a senior developer but doesn't know integers have a limited size (never heard of BigInteger), knows a hash map is used for key-value but not what hash means nor what the point of an interface is? Sure, I've seen that candidate. There's just so many terrible developers that you pretty much have to do some extra filtering to weed these out.
A short technical screen to ask those things is reasonable. My post was originally about unpaid projects, although the discussion around interviewing in general has been interesting.
I wouldn't expect to get paid for a 20 minute tech trivia screen.
I wouldn't expect to get paid for a 20 minute tech trivia screen.
Sure there are plenty of other professions. Pilots being the first one I can think of, as they require some type of checkride (flight exam) which will include an oral examination by the instructor/interviewer. And they don't get paid for taking that checkride.
Professional musicians have to prove themselves in auditions, CPAs, auto mechanics (good ones at least), etc. The list goes on.
Professional musicians have to prove themselves in auditions, CPAs, auto mechanics (good ones at least), etc. The list goes on.
Professions like pilots, doctors and CPAs are bad examples because they have a certifying body that exists outside of the relationship between employee and employer. It's true they're not getting paid for that certification, but once they have it, it's valid and transferable between jobs.
Musicians and actors are better examples, as they do have auditions, but lots of actors and musicians get jobs based on their previous body of work, rather than an in-person audition.
I'm curious, can you name more professions that use what is essentially auditions for hiring, besides artist-types?
Musicians and actors are better examples, as they do have auditions, but lots of actors and musicians get jobs based on their previous body of work, rather than an in-person audition.
I'm curious, can you name more professions that use what is essentially auditions for hiring, besides artist-types?
So much this. I give people options of many interview workflows as I don’t care... I need some way to test and make a discussion about it. I want you to succeed more than you half the time! I’ve still seen failed candidates go on social media and complain that they’d rather do X when it was a freaking option. Entitled little twats
[deleted]
kudos for giving options; too bad the candidates didn't appreciate that. once they spend some more time in the wild, they might!
There's a bit of missing info in the OP: how much time is this project expected to take? If it's literally 2 hours or something, then fine, don't pay me. If you want me to spend 8+ hours developing a polished looking site or a standalone API, then I'm probably not going to do your interview unless passing the test gets me straight to the offer stage.
Roughly speaking, I can get to an offer decision after:
1 recruiter call (20-30 minutes, usually, pretty softball).
1 or 2 phone screens (30-60 minutes apiece, generally).
1 day onsite for 4-6 hours.
If you're going to demand much more than that amount of time from me, you have to offer something compelling. The definition of "compelling" varies depending how much I think I need a new job.
Roughly speaking, I can get to an offer decision after:
1 recruiter call (20-30 minutes, usually, pretty softball).
1 or 2 phone screens (30-60 minutes apiece, generally).
1 day onsite for 4-6 hours.
If you're going to demand much more than that amount of time from me, you have to offer something compelling. The definition of "compelling" varies depending how much I think I need a new job.
The most recent project I did last week was 3 days. Three unpaid days, and in the prompt no time limit per day. It's for a company that is top 10 most valuable startups in the world. Do you tell them you won't do it..? How long do you work on it? Open ended. I actually like what I made but I would normally charge thousands of dollars for it. It's so frustrating. Haven't heard back yet but regardless it's not a good process
Yes, I tell them no thanks and give them my reasoning. I would rather work for a public company that pays real money (like the company I work for now) than a startup with paper equity, anyway.
Google does it too, I've done projects for them the same way. Good on you for sticking to your principles though. If everyone did maybe it wouldn't be so commonplace. Not everyone is in the situation to refuse. Thanks for the viewpoints.
I have serious ethical issues with working for Google, so that would just make my decision easier.
Just an example of a publicly traded company that I've seen do it (and if the top ones do it, then the other thousands of companies do too).
What's this ethical, publicly traded company you work for that doesn't do BS interviews? I bet they'd get a lot of inbound if they made that known.
What's this ethical, publicly traded company you work for that doesn't do BS interviews? I bet they'd get a lot of inbound if they made that known.
I like Basecamp's approach of not having live coding in the interview. They discuss past projects and you walk through some code you're proud of or think is interesting.
1. https://m.signalvnoise.com/hiring-a-programmer-ditch-the-cod...
2. "... you’ll submit some code you’re proud of, review it, and tell its story. Then on to an interview. Our interviews are one hour, all remote, with your future colleagues, on your schedule. We’ll talk through some of your code and some of ours. No gotchas, brainteasers, or whiteboard coding." from https://apply.workable.com/basecamp/j/F2CB808F33/
edit: spacing
1. https://m.signalvnoise.com/hiring-a-programmer-ditch-the-cod...
2. "... you’ll submit some code you’re proud of, review it, and tell its story. Then on to an interview. Our interviews are one hour, all remote, with your future colleagues, on your schedule. We’ll talk through some of your code and some of ours. No gotchas, brainteasers, or whiteboard coding." from https://apply.workable.com/basecamp/j/F2CB808F33/
edit: spacing
Until you run into the guy who literally cannot code. I'm not talking hard algorithms either.
Then you fire them.
This - California is an at-will employment state. Given that, you'd think hiring and firing would both be quicker. Literally any reason that isn't illegal. So why are there so many barriers to being hired if I can be fired at whim?
Currently cargo-cult fashion.
This seems decent, except for the “submit some code you’re proud of“ part. What about applicants whose only jobs have been on proprietary code they can’t share? This is the same problem as using GitHub resumés: it’s kind of unfairly biased against people who either don’t work on open source projects, or like to have a life outside of coding when they’re not on the job.
That sounds like a good case for paying someone to do a project for you (if they're employed, give them an 'hours' budget over a longer 'days' or 'weeks' period). Then you'd have something to look at that they just built, fresh.
It's easy - let the candidate choose what works best for them.
I don't think of it as "unpaid work".... Unless you think they are turning around and using it...
As a noob that is bad at interview programming trivia I've really enjoyed the interview projects I did.
Granted they were hardly a lot of work, but I felt they did a far better job at showing what I can do than all the programming trivia out there.
As a noob that is bad at interview programming trivia I've really enjoyed the interview projects I did.
Granted they were hardly a lot of work, but I felt they did a far better job at showing what I can do than all the programming trivia out there.
We've had really good success giving a short programming project to candidates, but with caveats:
1. this follows an extensive screen, where we have a strong signal whether the candidate is a good fit
2. the project isn't "offloading a current project" to someone to do free work for us; it's marginally related to our core product but pretty generic
3. the project is explicitly meant to take at most a few hours (we even use this as a way to gauge whether a candidate tends to over-engineer things)
4. we do a proper code review and architecture discussion after the candidate has submitted an implementation; we take as much time and put as much effort as with production code reviews; we probably put in almost as much time evaluating the candidate as much as they take to do the assignment
All this has worked pretty well for us, a mostly remote team, where interviewing has its own difficulties.
1. this follows an extensive screen, where we have a strong signal whether the candidate is a good fit
2. the project isn't "offloading a current project" to someone to do free work for us; it's marginally related to our core product but pretty generic
3. the project is explicitly meant to take at most a few hours (we even use this as a way to gauge whether a candidate tends to over-engineer things)
4. we do a proper code review and architecture discussion after the candidate has submitted an implementation; we take as much time and put as much effort as with production code reviews; we probably put in almost as much time evaluating the candidate as much as they take to do the assignment
All this has worked pretty well for us, a mostly remote team, where interviewing has its own difficulties.
"3. the project is explicitly meant to take at most a few hours (we even use this as a way to gauge whether a candidate tends to over-engineer things)"
i think this is great - time limits help both sides, but companies don't tend to use them that way.
i think this is great - time limits help both sides, but companies don't tend to use them that way.
I paid a prospective candidate for project work like this. It was more than the market rate for the role they were applying for. For context, this was in India, and I'm also Indian, and so is my company -- this was not an offshoring/outsourcing thing. They disappeared for a month, sent me 20 lines of python that did not work and demanded payment. I made the payment, as promised. Obviously, I did not hire. For a small company like mine, which I'm funding from my own pocket, I don't think I can afford too many candidates like this. I've stopped that practice. Now I only hire people I've worked with on previous jobs before. I know the quality to expect and I know the price I'll be paying.
I'm sorry they wasted your time and money. I think if someone disappeared for a month that would break the expectation of payment, but a small contract would settle that.
The cost of litigating a contract would have far exceeded any value I could have possibly gotten from the project. The distraction and reputational damage from the ensuing fight would have been even worse. The best policy was to be be polite, and make the payment promised and move on. No need to belittle or argue with the candidate. I chalk it up to a learning cost in the business. I now know one way of hiring that won't work for me.
The irony is that the project had no value to me. I had already implemented that feature, and just wanted to understand if the candidate would be able to do the work I'd already done for myself. Indeed I picked the problem because I had solved it myself under timed conditions and knew how long it would take me. I wrote the code for the feature in about 20 mins and then tested and checked-in under an hour. I expected that a new person with less experience and skill than me would take about 3-5x the time, and that would still be okay. Even if they took a day that was not an issue for me. Which is why I paid them for one day of their time. They might have taken a whole day. But 30 days was somewhat too long. Thankfully I'd priced it at a fixed cost. So my downside was fixed.
Still the learning was not cheap. I was paying what would be considered consulting rates in India.
The irony is that the project had no value to me. I had already implemented that feature, and just wanted to understand if the candidate would be able to do the work I'd already done for myself. Indeed I picked the problem because I had solved it myself under timed conditions and knew how long it would take me. I wrote the code for the feature in about 20 mins and then tested and checked-in under an hour. I expected that a new person with less experience and skill than me would take about 3-5x the time, and that would still be okay. Even if they took a day that was not an issue for me. Which is why I paid them for one day of their time. They might have taken a whole day. But 30 days was somewhat too long. Thankfully I'd priced it at a fixed cost. So my downside was fixed.
Still the learning was not cheap. I was paying what would be considered consulting rates in India.
Hopefully you can find a standard contract template online that you can use without having to pay much, if anything. A contract would have had a specific date the result was expected, and if the candidate breached that, it would have clearly been their fault and you wouldn't have had to pay anything.
If you haven't licensed your code to them, they shouldn't be able to legally incorporate it in their product. (Although perhaps you implicitly or explicitly gave them permission during the interview). But this is usually not very relevant as you probably will never know whether these companies are using your code or not.
If you only want to prevent companies from using your code in proprietary software, you could send them your code with a GPL license.
Copyright aside, yes, unpaid length projects during interviews are an atrocity.
If you only want to prevent companies from using your code in proprietary software, you could send them your code with a GPL license.
Copyright aside, yes, unpaid length projects during interviews are an atrocity.
Definitely no license given. Additionally I usually throw an "all rights reserved" on there somewhere to acknowledge my authorship and ownership of the product.
That's great for code + the specific execution of it... for me I often have to also do design engineering and content/concept prototyping as part of these projects. Creative work in addition to the code. I'm not sure there's a way to protect those ideas the same way you can the code.
That's great for code + the specific execution of it... for me I often have to also do design engineering and content/concept prototyping as part of these projects. Creative work in addition to the code. I'm not sure there's a way to protect those ideas the same way you can the code.
I'm a consultant and have a practice of asking to be paid for these kind of silliness. If the company want they can look at my github profile instead.
How many companies have paid you?
That doesn’t make any sense tbh. You’re missing the point completely.
I’ve had the gall several times to ask that they pay me to do one of these mini projects but it never resulted in another interview round. And I've actually by now, developed a portfolio of mini projects and try to point there but that hasn’t worked. Ironically it only worked with YouTube who moved me to interview in person and in front of a whiteboard instead. As an engineer, you can’t win. Go figure.
Yeah so many people saying "I tell them I won't do it," without saying how many positive responses companies give them for their principled stands. I'd imagine it's because almost no companies continue with an interview after that point.
re: YT - welp, out of the frying pan, as they say lol
re: YT - welp, out of the frying pan, as they say lol
Many Software Engineers haven't internalized this concept of time value. A lawyer or a doctor would never do real work for free as part of an interview process but for some reason we don't see it that way when it's about writing code.
interesting observation! makes sense. i wonder if it's because engineers are generally salaried instead of billed hourly. given all the 'crunch' and unpaid overtime, we'd be a lot more expensive hourly - folks might realize their real value. instead we give free overtime and then extend that into free interview time
I have reviewed two take home programming tasks. for both of them, there was a high correlation between submitting quickly and passing. so I guess if you're down to the wire and spending all your time on the problems then you're not gonna pass anyway.
also they are by far the easiest way to screen people who will fail later. isn't that ideal?
also they are by far the easiest way to screen people who will fail later. isn't that ideal?
no, doesn't sound ideal to me. you probably don't need a take-home.
how much time are you expecting / "giving" candidates to work on these take-home programming tests? it sounds like you're wasting lots of people's time by putting a gap in between your certitude that someone who finishes in X amount of time will be hired and not "fail later", and the Y amount of time total. If you're not going to hire someone beyond the X value, you need to downward adjust your Y until it is the X. Otherwise you're literally wasting people's time.
No, that's not ideal.
Also: do you pay them for their time?
how much time are you expecting / "giving" candidates to work on these take-home programming tests? it sounds like you're wasting lots of people's time by putting a gap in between your certitude that someone who finishes in X amount of time will be hired and not "fail later", and the Y amount of time total. If you're not going to hire someone beyond the X value, you need to downward adjust your Y until it is the X. Otherwise you're literally wasting people's time.
No, that's not ideal.
Also: do you pay them for their time?
For me, it all depends. If I get to skip directly to an on-site and skip the whole phone screen business, I'm willing to do a short (1-2 hour) take home type task. If it's longer than that, and/or it's in addition to phone screens, etc. my willingness is directly proportional to how much I need a new job.
Isn't a much better use of VC money than all the ads -- or platforms like Hired, Angelist A-list, etc -- to just, you know, pay candidates directly? What's your cost for a recruiter to get a new candidate? 10% of first year salary of 120K is $12,000.
P.S - whiteboard interviews are also terrible and unpaid.
P.P.S. - "platforms" that whiteboard- e.g. Hired, A-list, etc - are also unpaid and terrible, at scale.
P.P.P.S. I'm not sure what the solution is but I know it has something to do with actually getting to know the PEOPLE you are interviewing. We're all individuals with unique histories. Pay me for an hour or two to tell you my story and you might be surprised at what you learn.
P.S - whiteboard interviews are also terrible and unpaid.
P.P.S. - "platforms" that whiteboard- e.g. Hired, A-list, etc - are also unpaid and terrible, at scale.
P.P.P.S. I'm not sure what the solution is but I know it has something to do with actually getting to know the PEOPLE you are interviewing. We're all individuals with unique histories. Pay me for an hour or two to tell you my story and you might be surprised at what you learn.
There are some solutions out there for people that do not want to do whiteboard algorithms or take home projects. There are interview processes like codebase-projects [0] which you contribute to a company repository. Most companies pay you for the day.
There definitely seems to be something wrong, in that a lot of companies don't bother at all in getting to know you. But there are a few out there that do make interviewing a two-way street, and make you feel like you aren't wasting your time.
[0] https://introview.io/processes/codebase-project/
[0] https://introview.io/processes/codebase-project/
thank you so much for the link - it's great to know that companies exist that are trying different things. showed it to my friend and he said, "Only 9 companies do that!? In the whole world!??!" :/
It’s not free work.
No company gets candidates to do coding tests as a way to get productive real world tasks completed.
Just don’t do it when a companies test task is too onerous.
No company gets candidates to do coding tests as a way to get productive real world tasks completed.
Just don’t do it when a companies test task is too onerous.
I work and don't get paid... It's free work. whether it gets used or not by them, I spent my time doing a task that was directed for me. That's work.
Actually there is a legal problem here- if you are not an employee you must be able to provide an invoice, which most of us cannot.
What do you mean? You can whip up an invoice in Google docs or use a service like Wave or something for free.
The trick is: who do you send the invoice to, and will they ever pay it?
I've invoiced companies that made me do unpaid projects and while it makes me feel better it has never once gotten me paid.
The trick is: who do you send the invoice to, and will they ever pay it?
I've invoiced companies that made me do unpaid projects and while it makes me feel better it has never once gotten me paid.
> You can whip up an invoice in Google docs or use a service like Wave or something for free.
The invoice is not the problem, handling taxes is. In most countries (I'm not an American) an Invoice means that you are self employed and that you should pay taxes on that part as self employed.
There are services around it that charge percentage of the transaction and I'm not sure will handle such small amounts.
The invoice is not the problem, handling taxes is. In most countries (I'm not an American) an Invoice means that you are self employed and that you should pay taxes on that part as self employed.
There are services around it that charge percentage of the transaction and I'm not sure will handle such small amounts.
I rarely, if ever, get feedback on these projects. In later rounds, it's obvious most interviewers haven't seen the project or even know you did it.
Why is software engineering the only industry I'm aware of that requires you to prove that you know how to do things you've been doing for years and years? Your projects on Github, degree in your field, and resume don't matter and are rarely discussed.
Unpaid projects, over and over.
Companies are too lazy to do their own work and we are all getting screwed.
How many hours of free work have you had to do, only to get ghosted, denied without feedback, or see your work turn up in their product later?
It isn't that hard of a problem to fix: stop being cheap and pay people for their time.
"But we can't afford it..." - nonsense! Esp. big companies. Poppycock.
If it's "too expensive", you need to have a better funnel upfront before the project stage.
Properly compensating for interviews would be a huge differentiator for recruiting.
Great engineer A: "I interviewed at Company X, and didn't get it. But they paid me $1000 for a day's work. It was pretty cool."
"Rockstar" engineer B: "Wow, that sounds neat. Too bad you didn't get it!" (Maybe I'll interview there too...)
vs. ill-will from unpaid
Great engineer a: "Yeah I interviewed with company X and they made me do a 3 day unpaid project. I didn't get any feedback and they made me do a technical interview AFTER I had already submitted the project, where they just asked unrelated trivia. One line rejection e-mail."
"Rockstar" engineer B: "Yeah that sucks. They sound awful. You dodged a bullet" (continues working where they are)
If I do work for you, pay me for it. If I do work for you, give me feedback. If I do work for you, respect my time.