Fizzbuzz isn't a coding test(kapilkale.com)
kapilkale.com
Fizzbuzz isn't a coding test
http://www.kapilkale.com/blog/fizzbuzz/
54 comments
I think that FizzBuzz is just filtering out incompetent applicants who shouldn't be trying for programming jobs to begin with. Joel on Software described this phenomenon back in 2005. [1] His key takeaway is that 99%+ of people applying for developer jobs may be unqualified for pretty much any coding job, but they keep applying anyway.
He describes how he and other companies get 200 applicants for most job postings, of which only one may actually be "good enough." I've seen numbers like that myself when on the hiring end. So everyone thinks they're hiring the top 0.5%, even though the same 199 losers are spamming their resumes to every job posting.
Well, a simple way to filter out that 199 is to apply FizzBuzz or equivalent.
[1] http://www.joelonsoftware.com/items/2005/01/27.html
He describes how he and other companies get 200 applicants for most job postings, of which only one may actually be "good enough." I've seen numbers like that myself when on the hiring end. So everyone thinks they're hiring the top 0.5%, even though the same 199 losers are spamming their resumes to every job posting.
Well, a simple way to filter out that 199 is to apply FizzBuzz or equivalent.
[1] http://www.joelonsoftware.com/items/2005/01/27.html
So everyone thinks they're hiring the top 0.5%, even though the same 199 losers are spamming their resumes to every job posting.
Yeah, when you think about it, this whole "we only hire the top 0.5%" condescension really amounts to a basic statistical reasoning fallacy.
To construct an alternate hypothesis: the "real" threshold for ranked competence[1] at "elite" companies might, just might, be more like "the top 5%". But we can safely assume that 95% of the available talent pool in this category will be employed (and not actively seeking), compared with say 70% in the bottom, or "sub-elite" category, at any given time.[2]
So for example in a "town" of 1000 putative developers, with skill levels ranging from god-like to "good enough" to mediocre to utterly bogus and self-deluding, let's say 50 (or 5%) of these are truly "elite", and 950 are "non-elite".
Further, of the elite category, 47.5 will be employed (and not actively looking), and only 2.5 will be in the job market -- versus, out of the 950 non-elite developers, we might have 665 (or 70%) employed, versus 285 (or 30%) on the market and actively looking.
Given this over-simplified (but arguably "less wrong") hypothesis, and assuming that a given opening draws applicants whose fitness ranking is evenly distributed across the pool of in-the-market candidates, it's easy to see that for any given opening, we'll have a ratio of 2.5:285 for elite:non-elite developers -- or a paltry 0.869%(!) of the candidate pool (2.5/287.5) will appear to be from the "elite" category.
That is, for every 1 "elite" resume in your inbox, you'll be seeing 114 "non-elite" resumes. And since tech geeks are, by and large, an insecure, envious lot, they like to brag and fudge a bit -- so that 114 easily becomes 199 in their minds (or the 1 a 0.6, whichever).
Which gives you the infamous "we only hire the top 0.5%" urban legend right there. Even though they're all really just angling over the... top 5% (ranked over the general pool of putative developers).
And since hot-shots like Joel Spolsky, not to mention nearly all of the investment bank / hedge fund types, and well, probably half or more of everyone in the monthly "Who's hiring?" thread just keeps repeating it, and repeating it, and repeating it... well, shucks, it must be true.
[1] As such a thing as generic competence exists can be objectively measured, etc, etc, but that's a separate issue.
[2] Which corresponds -- conservatively, for the purposes of this argument -- to another widely espoused, but never quantitatively substantiated belief out in startupland: that "all" of the good developers are "always" employed and "almost never" on the job market longer than 2 days.
Yeah, when you think about it, this whole "we only hire the top 0.5%" condescension really amounts to a basic statistical reasoning fallacy.
To construct an alternate hypothesis: the "real" threshold for ranked competence[1] at "elite" companies might, just might, be more like "the top 5%". But we can safely assume that 95% of the available talent pool in this category will be employed (and not actively seeking), compared with say 70% in the bottom, or "sub-elite" category, at any given time.[2]
So for example in a "town" of 1000 putative developers, with skill levels ranging from god-like to "good enough" to mediocre to utterly bogus and self-deluding, let's say 50 (or 5%) of these are truly "elite", and 950 are "non-elite".
Further, of the elite category, 47.5 will be employed (and not actively looking), and only 2.5 will be in the job market -- versus, out of the 950 non-elite developers, we might have 665 (or 70%) employed, versus 285 (or 30%) on the market and actively looking.
Given this over-simplified (but arguably "less wrong") hypothesis, and assuming that a given opening draws applicants whose fitness ranking is evenly distributed across the pool of in-the-market candidates, it's easy to see that for any given opening, we'll have a ratio of 2.5:285 for elite:non-elite developers -- or a paltry 0.869%(!) of the candidate pool (2.5/287.5) will appear to be from the "elite" category.
That is, for every 1 "elite" resume in your inbox, you'll be seeing 114 "non-elite" resumes. And since tech geeks are, by and large, an insecure, envious lot, they like to brag and fudge a bit -- so that 114 easily becomes 199 in their minds (or the 1 a 0.6, whichever).
Which gives you the infamous "we only hire the top 0.5%" urban legend right there. Even though they're all really just angling over the... top 5% (ranked over the general pool of putative developers).
And since hot-shots like Joel Spolsky, not to mention nearly all of the investment bank / hedge fund types, and well, probably half or more of everyone in the monthly "Who's hiring?" thread just keeps repeating it, and repeating it, and repeating it... well, shucks, it must be true.
[1] As such a thing as generic competence exists can be objectively measured, etc, etc, but that's a separate issue.
[2] Which corresponds -- conservatively, for the purposes of this argument -- to another widely espoused, but never quantitatively substantiated belief out in startupland: that "all" of the good developers are "always" employed and "almost never" on the job market longer than 2 days.
...and then you'll offer a salary that has been painstakingly researched and calculated to be as close to median as you can get it, without going over.
...or a notch under, for good measure.
Who cares the about what FizzBuzz REALLY is? Can't we all just agree that it's a programming test because anybody who calls them self a programmer should possess these painfully simple analytic skills. If you can translate pseudocode to code, but can't write pseudocode for FizzBuzz, you're a translator not a programmer.
Exactly. Who cares what it is, it remains useful for weeding out the people who think they can get away with lying about being a programmer.
I've been doing FizzBuzz and a few other code-related tests for the last few years. It was an odd shock when I realized that was the /only/ part of the interview process that had any value to me whatsoever. I mean it. I have seen people display radically different personality traits between the interview and work on Monday. The guy who was almost too nervous to speak in the interview won't stop talking peoples' ears off. The guy who really sold himself well is very shy and quiet now that he has to show up every day. I wouldn't believe if it I hadn't seen it. Not to mention language lawyers, etc. But coding tests have predictive value. YMMV of course.
I'm also surprised how many people get it wrong. I see a lot of this (pseudocode):
I'm also surprised how many people get it wrong. I see a lot of this (pseudocode):
for (i = 0; i < 21; i++) {
if (i % 3 == 0) {
print "fizz";
} else if (i % 5 == 0) {
print "buzz";
} else if (i % 3 == 0 && i % 5 == 0) {
print "fizzbuzz";
}
}
Consider yourself fortunate, I've screened candidates who didn't get past the "for" keyword because they couldn't construct a loop in a language of their own choosing, even after I reminded them that perfect syntax is not necessary.
I've had a surprising number of candidates who did not know about the existence of the modulo operator. Not just "I completely forgot about it", but more like "what's the % sign mean?".
I've had a surprising number of candidates who did not know about the existence of the modulo operator. Not just "I completely forgot about it", but more like "what's the % sign mean?".
I'd forgive an entry-level candidate (straight out of college) for not knowing about the modulo operator. We had one who solved it correctly using floor/ceiling.
If they're claiming 3+ years of experience though, I'd say not knowing about modulo is a red flag. It means they're being dishonest about their actual experience, or those 3+ years of experience are very narrow, and/or they're not terribly curious about the field.
If they're claiming 3+ years of experience though, I'd say not knowing about modulo is a red flag. It means they're being dishonest about their actual experience, or those 3+ years of experience are very narrow, and/or they're not terribly curious about the field.
I once had a candidate claiming to be a senior PHP programmer solve fizzbuzz by doing something like this:
if (strpos($i/3, ".") != null)
or something like that. I was sad.
I googled to see whether this was creativeness or just copying stuff from the net, and came across http://www.php.net/manual/en/language.operators.arithmetic.p.... It states:
"Operands of modulus are converted to integers (by stripping the decimal part) before processing."
=> 4.5 % 1.5 is zero, as is 4.6 % 1.9. So, in the world of PHP, there may be logic in that strpos approach. As far as I can tell, it certainly isn't much worse than http://www.webdeveloper.com/forum/showthread.php?76114-Work-... advocates.
And no that is not worth much. I know almost nothin about PHP, and prefer to keep things that way.
"Operands of modulus are converted to integers (by stripping the decimal part) before processing."
=> 4.5 % 1.5 is zero, as is 4.6 % 1.9. So, in the world of PHP, there may be logic in that strpos approach. As far as I can tell, it certainly isn't much worse than http://www.webdeveloper.com/forum/showthread.php?76114-Work-... advocates.
And no that is not worth much. I know almost nothin about PHP, and prefer to keep things that way.
> "Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”
Aside from explicitly stating "Hey, encapsulate IF/ELIF/ELSE statements in a FOR loop", FizzBuzz doesn't seem too analytical--am I wrong?
Aside from explicitly stating "Hey, encapsulate IF/ELIF/ELSE statements in a FOR loop", FizzBuzz doesn't seem too analytical--am I wrong?
I've been interviewing a bit recently and not a single candidate has actually managed to complete it successfully (I use a slightly simpler variation with different numbers and strings). They have all been juniour but all claimed at least a few years of programming experience (counting university/school). Thinking I was going nuts, I tried it out on my colleagues including one whose total programming experience was trying to learn Python a few months ago, and they finished 100% correctly in under a minute.
I don't ask for correct syntax, an actual programming language, fret over the 100 being included or not etc. Heck I even ask them what the output would be for the "code" they just wrote and they usually get that wrong too! For a for loop and 3 arms of conditionals.
I suspect it is a standin for experience. You can get pretty far copying and pasting sample code you find on the net, maybe altering a number or two. (Or the far more telling commenting blocks of code out.) These days you can have something that appears to work by using project creators and tinkering, but not actually doing a reasonable amount of what could be considered older school programming.
I don't ask for correct syntax, an actual programming language, fret over the 100 being included or not etc. Heck I even ask them what the output would be for the "code" they just wrote and they usually get that wrong too! For a for loop and 3 arms of conditionals.
I suspect it is a standin for experience. You can get pretty far copying and pasting sample code you find on the net, maybe altering a number or two. (Or the far more telling commenting blocks of code out.) These days you can have something that appears to work by using project creators and tinkering, but not actually doing a reasonable amount of what could be considered older school programming.
I ask people to write strlen() for me. (If they don't know C, it's a showstopper, btw).
Supposedly senior candidates with ten years of experience have failed this. It's truly amazing. "Oh, I haven't written code in a while" is a common response. That's a show-stopper, too -- what, you want to /manage/ me or something? And you can't write code? God forbid.
Supposedly senior candidates with ten years of experience have failed this. It's truly amazing. "Oh, I haven't written code in a while" is a common response. That's a show-stopper, too -- what, you want to /manage/ me or something? And you can't write code? God forbid.
[deleted]
It seems trivial, but yes, to some degree, it requires a little analytical thinking to get over the hurdle of deciding what to print and when. It's shocking just how many "programmers" can't figure it out, even with a little prompting.
Agree completely, the description is basically pseudocode on its own.
There's no abstraction, it's literally instructing the programmer exactly what to do.
There's no abstraction, it's literally instructing the programmer exactly what to do.
No IF/ELIF/ELSE necessary :)
take 100 $ zipWith (++) (cycle ["", "", "Fizz"]) $ cycle ["", "", "", "", "Buzz"]I see a frightening number of people buying into the disturbing assertions made in TFA:
- that analytical thinking isn't necessary for programming.
- that translating pseudocode is a sufficient test of programming skill.
- that Excel isn't equivalent to a programming environment, or somehow doesn't count as programming.
- that FizzBuzz requires higher-order thinking. It's not like you're asking someone to implement lists as lambdas. Or the this website's namesake (the Y Combinator.)
Each one of these items is a big fat WTF.
- that analytical thinking isn't necessary for programming.
- that translating pseudocode is a sufficient test of programming skill.
- that Excel isn't equivalent to a programming environment, or somehow doesn't count as programming.
- that FizzBuzz requires higher-order thinking. It's not like you're asking someone to implement lists as lambdas. Or the this website's namesake (the Y Combinator.)
Each one of these items is a big fat WTF.
I don't think engineers should ever spend their time asking Fizzbuzz. Engineers' time is valuable. And blocking off an hour for a phone screen can interrupt the day. If a preliminary code sample filtering step is used then engineers can jump straight into the meaty interview questions.
The vast majority of people who don't know programming basics like loops and conditionals won't make it to the phone screen stage. Someone might look up the solution to the programming quiz but they'll immediately fail the first interview question. And this can be partially mitigated by rotating puzzles or even changing numbers in the puzzles.
The code sample filtering step should have both a sufficiently tricky puzzle and a bank of at least 20 common corner cases. Just run the test cases automatically. The vast majority of candidates will be eliminated without wasting any engineers' time.
The vast majority of people who don't know programming basics like loops and conditionals won't make it to the phone screen stage. Someone might look up the solution to the programming quiz but they'll immediately fail the first interview question. And this can be partially mitigated by rotating puzzles or even changing numbers in the puzzles.
The code sample filtering step should have both a sufficiently tricky puzzle and a bank of at least 20 common corner cases. Just run the test cases automatically. The vast majority of candidates will be eliminated without wasting any engineers' time.
IMO phone screen (if it really is a "screen" and not a full "interview") should be fifteen minutes or less. I usually ask a few very specifically CS101 type questions to see if they remember any of it, and if they don't, they don't come in for an interview.
I agree about the phone screen length. Whether you call it an interview or a screen it can probably be much shorter than hour with almost all of the benefits.
I've got to disagree here. The whole point of FizzBuzz is that it's a straight-forward analytical problem that you should be able to do easily if you understand the fundamentals.
I think some part of this issue has to be that computer science does not fully equate to coding, and the balance of coding to theory curriculum varies significantly. If you're hiring recent grads then it might make sense to give a little leeway, to help them along and see if you see some spark of analytic ability. Maybe interview nerves are an issue.
If your candidate can't do fizzbuzz but is claiming to be an experienced programmer in the required language, the situation should raise a red flag.
If your candidate can't do fizzbuzz but is claiming to be an experienced programmer in the required language, the situation should raise a red flag.
It's not coding, in the strictest sense that it has no computer science or is solely syntactic, but it works at weeding out people who can't build anything. At least this level of analytical thinking is necessary for any programming task, and therefore Fizzbuzz serves its purpose.
>"Ask some smart business analysts with no coding experience to solve FizzBuzz in Excel. All should be able to do it."
Bahaha. I want to see you try. And weep.
Bahaha. I want to see you try. And weep.
How many people have actually been asked Fizzbuzz in an interview? I recently interviewed at several places, before deciding I'd rather try my hand at independent consulting. At least two were companies with (apparently) superb engineering teams. But no one asked me this question, or any preliminary coding question at all. I was asked to code, but only after the companies had paid money to fly me out, put me up in a hotel, and have their team spend a day talking with me. I would be delighted to be asked this question, because I'd take it as a sign that the company values its time and is discriminating in its hiring.
FizzBuzz isn't a coding test, I agree. It's an interview prep test. If a candidate can't take 15 minutes to read up on common interview questions and prepare a simple answer, then they shouldn't be hired.
It is a coding test, and here are the answers:
http://www.geekschool.org/programming/fizzbuzz/
FizzBuzz in lots of languages, including LOLCode.
http://www.geekschool.org/programming/fizzbuzz/
FizzBuzz in lots of languages, including LOLCode.
Higher order thinking is pretty much required to make any working program of minimal complexity. aka. be a programmer.
If you have to hand a programmer pseudo code in order to write a program, you'd be better off hiring competent programmers to write you a compiler and then you can get rid of the programmers who can't think and code it yourself in your fancy new higher order language.
If you have to hand a programmer pseudo code in order to write a program, you'd be better off hiring competent programmers to write you a compiler and then you can get rid of the programmers who can't think and code it yourself in your fancy new higher order language.
This is definitely the relevant point when it comes to hiring programmers. FizzBuzz is already so close to pseudocode that if you had to write it more clearly, it wouldn't take any more time to just write the code yourself. A programmer who can only work at that level is not a benefit to your team.
You really want people who will improve your team's output by an amount greater than the value of the extra time it takes to communicate with them.
FizzBuzz can be an effective filter to very quickly get rid of the lowest level of applicants and start having a real discussion with the ones who are left.
You really want people who will improve your team's output by an amount greater than the value of the extra time it takes to communicate with them.
FizzBuzz can be an effective filter to very quickly get rid of the lowest level of applicants and start having a real discussion with the ones who are left.
If fizz buzz is too complicated, what would be an example of a problem a programmer should be able to solve? Why do programmers exist?
There is analysis and there is coding. I've met and worked with people who had both skills (awesome) and only one of the two (less awesome). If you're limited in the number of people you can hire (startup) then you want both skills in the same person. I think any business analyst would agree with that.
How useful is translating "pseudocode into a working program"? I doubt the utility of that skill as a programmer
Starting from pseudocode is a big leg up. In most situations the programmer won't have pseudocode unless he/she writes it. Users don't describe what they want clearly and unambiguously most of the time -- that's the other big skill programmers need. The first skill is being able to write FizzBuzz or something equally trivial in a couple of minutes.
If a so-called programmer or coder can't write FizzBuzz from pseudocode then presumably they won't be able to write it from a prose description, which means they are going to be close to useless solving any real problem.
If a so-called programmer or coder can't write FizzBuzz from pseudocode then presumably they won't be able to write it from a prose description, which means they are going to be close to useless solving any real problem.
[deleted]
[deleted]
seeing this makes me cry. I can code this is at least three languages (java, php, python) and I can't even get an interview for a junior dev position. I got bored the other day and tried to write it recursively because why not, right?
It is, of course, up to each manager to decide what a person is being hired to do. If you want to hire someone to do something useful for you at your company, it's a very good idea to give the job candidate a work-sample test. For some kinds of programming jobs, a fizzbuzz coding test is a (very easy) work-sample test. I've heard a story similar to those in other comments here about 100 applicants to a company that uses Web back-end programming to serve up online, adaptive lessons. The test the hiring supervisor used was "Write a program using a for loop to print the integers from 1 to 100." No language was specified. He has not yet had ANY applicant, even though all his applicants have computer science degrees, many having master's degrees, submit a working program to do that. Not one.
This is discussed so often on Hacker News I have a FAQ on the issue of company hiring procedures. There are many discussions here on HN about company hiring procedures. From participants in earlier discussions I have learned about many useful references on the subject, which I have gathered here in a FAQ file. The review article by Frank L. Schmidt and John E. Hunter, "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings," Psychological Bulletin, Vol. 124, No. 2, 262-274
http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...
sums up, current to 1998, a meta-analysis of much of the HUGE peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring procedures. There are many kinds of hiring criteria, such as in-person interviews, telephone interviews, resume reviews for job experience, checks for academic credentials, personality tests, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.
http://www.siop.org/workplace/employment%20testing/testtypes...
EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well. One is a general mental ability (GMA) test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. (But the calculated validity of each of the two best kinds of procedures, standing alone, is only 0.54 for work sample tests and 0.51 for general mental ability tests.) Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general mental ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than any other single-factor hiring procedure that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
Because of a Supreme Court decision in the United States (the decision does not apply in other countries, which have different statutes about employment), it is legally risky to give job applicants general mental ability tests such as a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424 (1971) case
http://scholar.google.com/scholar_case?case=8655598674229196...
interpreted a federal statute about employment discrimination and held that a general intelligence test used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring procedure had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring procedure could be challenged as illegal if it can be shown to have disparate impact on some job applicants. A company defending a brain-teaser test for hiring would have to defend it by showing it is supported by a validation study demonstrating that the test is related to successful performance on the job. Such validation studies can be quite expensive. (Companies outside the United States are regulated by different laws. One other big difference between the United States and other countries is the relative ease with which workers may be fired in the United States, allowing companies to correct hiring mistakes by terminating the employment of the workers they hired mistakenly. The more legal protections a worker has from being fired, the more reluctant companies will be about hiring in the first place.)
The social background to the legal environment in the United States is explained in many books about hiring procedures
http://books.google.com/books?hl=en&lr=&id=SRv-GZkw6...
http://books.google.com/books?hl=en&lr=&id=SRv-GZkw6...
Some of the social background appears to be changing in the most recent few decades, with the prospect for further changes.
http://intl-pss.sagepub.com/content/17/10/913.full
http://www.economics.harvard.edu/faculty/fryer/files/Fryer_R...
http://books.google.com/books?hl=en&lr=&id=frfUB3GWl...
Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article showed that multi-factor procedures work better than single-factor procedures, a summary of that article we can find in the current professional literature, for example "Reasons for being selective when choosing personnel selection procedures" (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias Berchtold, and Martin Kleinmann:
"Choosing personnel selection procedures could be so simple: Grab your copy of Schmidt and Hunter (1998) and read their Table 1 (again). This should remind you to use a general mental ability (GMA) test in combination with an integrity test, a structured interview, a work sample test, and/or a conscientiousness measure."
http://geb.uni-giessen.de/geb/volltexte/2012/8532/pdf/prepri...
But the 2010 article notes, looking at actual practice of companies around the world, "However, this idea does not seem to capture what is actually happening in organizations, as practitioners worldwide often use procedures with low predictive validity and regularly ignore procedures that are more valid (e.g., Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page, 1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir, 2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work sample tests are hardly used in the US, and the potentially rather useless procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied somewhere between occasionally and often in France (Ryan et al., 1999). In Germany, the use of GMA tests is reported to be low and to be decreasing (i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use them)."
Integrity tests have limited validity standing alone, but appear to have significant incremental validity when added to a general mental ability test or work-sample test.
http://en.wikipedia.org/wiki/Employment_integrity_testing
http://apps.opm.gov/ADT/Content.aspx?page=3-06&JScript=1
http://www.princeton.edu/~ota/disk2/1990/9042/9042.PDF
http://www.hotelschool.cornell.edu/research/chr/pubs/reports...
To sum up, if you want someone who has some minimal level of programming ability to work for you, you have to TEST that ability among all applicants for your position.
This is discussed so often on Hacker News I have a FAQ on the issue of company hiring procedures. There are many discussions here on HN about company hiring procedures. From participants in earlier discussions I have learned about many useful references on the subject, which I have gathered here in a FAQ file. The review article by Frank L. Schmidt and John E. Hunter, "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings," Psychological Bulletin, Vol. 124, No. 2, 262-274
http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...
sums up, current to 1998, a meta-analysis of much of the HUGE peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring procedures. There are many kinds of hiring criteria, such as in-person interviews, telephone interviews, resume reviews for job experience, checks for academic credentials, personality tests, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.
http://www.siop.org/workplace/employment%20testing/testtypes...
EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well. One is a general mental ability (GMA) test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. (But the calculated validity of each of the two best kinds of procedures, standing alone, is only 0.54 for work sample tests and 0.51 for general mental ability tests.) Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general mental ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than any other single-factor hiring procedure that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
Because of a Supreme Court decision in the United States (the decision does not apply in other countries, which have different statutes about employment), it is legally risky to give job applicants general mental ability tests such as a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424 (1971) case
http://scholar.google.com/scholar_case?case=8655598674229196...
interpreted a federal statute about employment discrimination and held that a general intelligence test used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring procedure had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring procedure could be challenged as illegal if it can be shown to have disparate impact on some job applicants. A company defending a brain-teaser test for hiring would have to defend it by showing it is supported by a validation study demonstrating that the test is related to successful performance on the job. Such validation studies can be quite expensive. (Companies outside the United States are regulated by different laws. One other big difference between the United States and other countries is the relative ease with which workers may be fired in the United States, allowing companies to correct hiring mistakes by terminating the employment of the workers they hired mistakenly. The more legal protections a worker has from being fired, the more reluctant companies will be about hiring in the first place.)
The social background to the legal environment in the United States is explained in many books about hiring procedures
http://books.google.com/books?hl=en&lr=&id=SRv-GZkw6...
http://books.google.com/books?hl=en&lr=&id=SRv-GZkw6...
Some of the social background appears to be changing in the most recent few decades, with the prospect for further changes.
http://intl-pss.sagepub.com/content/17/10/913.full
http://www.economics.harvard.edu/faculty/fryer/files/Fryer_R...
http://books.google.com/books?hl=en&lr=&id=frfUB3GWl...
Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article showed that multi-factor procedures work better than single-factor procedures, a summary of that article we can find in the current professional literature, for example "Reasons for being selective when choosing personnel selection procedures" (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias Berchtold, and Martin Kleinmann:
"Choosing personnel selection procedures could be so simple: Grab your copy of Schmidt and Hunter (1998) and read their Table 1 (again). This should remind you to use a general mental ability (GMA) test in combination with an integrity test, a structured interview, a work sample test, and/or a conscientiousness measure."
http://geb.uni-giessen.de/geb/volltexte/2012/8532/pdf/prepri...
But the 2010 article notes, looking at actual practice of companies around the world, "However, this idea does not seem to capture what is actually happening in organizations, as practitioners worldwide often use procedures with low predictive validity and regularly ignore procedures that are more valid (e.g., Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page, 1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir, 2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work sample tests are hardly used in the US, and the potentially rather useless procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied somewhere between occasionally and often in France (Ryan et al., 1999). In Germany, the use of GMA tests is reported to be low and to be decreasing (i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use them)."
Integrity tests have limited validity standing alone, but appear to have significant incremental validity when added to a general mental ability test or work-sample test.
http://en.wikipedia.org/wiki/Employment_integrity_testing
http://apps.opm.gov/ADT/Content.aspx?page=3-06&JScript=1
http://www.princeton.edu/~ota/disk2/1990/9042/9042.PDF
http://www.hotelschool.cornell.edu/research/chr/pubs/reports...
To sum up, if you want someone who has some minimal level of programming ability to work for you, you have to TEST that ability among all applicants for your position.
Great literature summary. The apocryphal story however - I call BS. Everybody can write a loop from 1 to 100.
You'd be surprised. There are quite a few "programmers" who can't. I've met a few. Hell, I've even heard someone claim years experience and they couldn't write a simple loop.
The apocryphal story was, this interviewer had never met any programmer who could write a loop. Obvious bs.
If I were presented with the 'fizzbuzz' test on a job interview I'd walk out.
I would really hate to work in a place that had such a ridiculously low bar to entry that they would consider that a valid test.
I would really hate to work in a place that had such a ridiculously low bar to entry that they would consider that a valid test.
You misunderstand the demographic of people who apply to programming jobs.
The vast majority (in my experience, 80%+) have no business writing code at any level. We're not talking about bad programmers here, we're talking about people who cannot program, full stop.
I can deal with "I'm not nearly as good as I say I am", I cannot deal with "What's an if statement?", but yet the majority of applicants at most large software firms are like this.
If you are a competent dev, FizzBuzz isn't designed to trap you, it's designed to trap someone far, far, far, far, far, far, far, far less competent than you. Just jump through this hoop and we can start the real interview.
You also make the mistake that FizzBuzz is the whole interview. It isn't. I've run many highly rigorous interviews, FizzBuzz is just the bar for me to talk to you past the 5 minute mark.
What I still don't get is how these people even exist. It's like showing up to an accountant's interview with a box of crayons and a bib. It's disturbing.
The vast majority (in my experience, 80%+) have no business writing code at any level. We're not talking about bad programmers here, we're talking about people who cannot program, full stop.
I can deal with "I'm not nearly as good as I say I am", I cannot deal with "What's an if statement?", but yet the majority of applicants at most large software firms are like this.
If you are a competent dev, FizzBuzz isn't designed to trap you, it's designed to trap someone far, far, far, far, far, far, far, far less competent than you. Just jump through this hoop and we can start the real interview.
You also make the mistake that FizzBuzz is the whole interview. It isn't. I've run many highly rigorous interviews, FizzBuzz is just the bar for me to talk to you past the 5 minute mark.
What I still don't get is how these people even exist. It's like showing up to an accountant's interview with a box of crayons and a bib. It's disturbing.
If I were presented with such a display while holding an interview, I would say, sayonara. I would really hate to work with someone that had such a ridiculously high bar that they would consider that invalidating the entire interview.
But I have this odd suspicion that, in real life, you wouldn't actually walk out.
But I have this odd suspicion that, in real life, you wouldn't actually walk out.
Really? That's a pretty knee-jerk reaction. A lot of companies will ask multiple technical questions. If a candidate has sweet talked his way into the building for an interview without possessing real skills, then a fizzbuzz test will weed out those people in a few minutes. A serious coder could whip up an answer almost instantly. Perhaps the last guy the hiring manager saw that day turned out to be a dud? He might feel an itch to make sure you're not of the same cloth.
The point of Fizzbuzz is to save time and money weeding out the clear incompetents so you don't have to take time to interview them later.
If that's the hardest question they ask you, then maybe you're right to leave. You're at least right not to take the job in that case.
Actually leaving is a bit harsh, though, since you don't know why they're asking the question. And you're definitely burning a bridge, which is probably a bad idea.
Why don't you ask the interviewer why they're using that question - after you answer it correctly and quickly? That will get you more insight into the company or at least the interviewer.
I sometimes use questions like that very early in an interview to make sure candidates didn't fake their way through the phone screen to get to the on-site interview. People who can't do the simple stuff do sometimes slip through.
I'm not trying to insult the intelligence or skills of the candidate, I'm just trying to see if they can pass that low bar before getting to the interesting questions.
Actually leaving is a bit harsh, though, since you don't know why they're asking the question. And you're definitely burning a bridge, which is probably a bad idea.
Why don't you ask the interviewer why they're using that question - after you answer it correctly and quickly? That will get you more insight into the company or at least the interviewer.
I sometimes use questions like that very early in an interview to make sure candidates didn't fake their way through the phone screen to get to the on-site interview. People who can't do the simple stuff do sometimes slip through.
I'm not trying to insult the intelligence or skills of the candidate, I'm just trying to see if they can pass that low bar before getting to the interesting questions.
I don't think FizzBuzz is intended to be an actual interview question so much as a thought experiment.
Instead of asserting this fact, put the hypothesis to the test. Ask some smart business analysts and see how many of them can solve fizz-buzz. You're much more likely to win over the audience if you have some data.
Really though, I think this post is needlessly controversial. The whole text could have been, "It makes sense to hire people with no programming experience who can solve fizz-buzz." I don't know how many people fall into that category. I'm not sure how good they'd be or how long they'd take to get up to speed, but that's fine. It's something I can think about and engage with.
The rest of the post is just arguing over definitions. If by "coding" you only mean "turning pseudocode into code", then yes a lot of things people call coding aren't coding. Likewise if by "think analytically" you mean "is able to solve fizz-buzz," then of course companies want programmers who can "think analytically."
No matter what definitions you use, the hiring practices are the same. Fizz-buzz still acts as a negative filter. People who can't solve it aren't hired. People who can solve it may or may not be hired depending on how other things go.