Thank goodness we hired someone who can reverse a b tree on a whiteboard (2016)(twitter.com)
twitter.com
Thank goodness we hired someone who can reverse a b tree on a whiteboard (2016)
https://twitter.com/ieatkillerbees/status/808831573785243649
33 comments
> We pick something boring, say from the standard library, and nothing tricky like a hash table. Really boring like floor(). If you can't figure out how floor() does its "magic" that's scary.
What I find scary is that you think floor() is an easy algorithm. Here's a typical high quality implementation of floor() (Plauger's implementation in "The Standard C Library" is different, but of similar complexity). Not something you'd derive on a whiteboard: http://www.netlib.org/fdlibm/s_floor.c
I'm concerned that a question like this is selecting for mediocre programmers who'd brute force some solution, while causing programmers who are aware of the complexity of the problem to freeze.
What I find scary is that you think floor() is an easy algorithm. Here's a typical high quality implementation of floor() (Plauger's implementation in "The Standard C Library" is different, but of similar complexity). Not something you'd derive on a whiteboard: http://www.netlib.org/fdlibm/s_floor.c
I'm concerned that a question like this is selecting for mediocre programmers who'd brute force some solution, while causing programmers who are aware of the complexity of the problem to freeze.
Interviewers underestimate the complexity and problems floating point numbers cause in interview questions.
I've recently had an interview where the interviewer scraped my most recent project off of GitHub, then asked me to add a feature, using a whiteboard. In my opinion this is the RIGHT WAY to do it.
The codebase is obviously familiar to the candidate, along with the problem domain. This rules out memorization, puzzle gaming, etc... It informs on programming style, ability to somewhat think on ones feet, but doesn't exclude candidates who didn't happen to guess what the interviewer wanted before the interview began.
No, that's the completely wrong way to do it. Download the code onto a computer. Ask the candidate to add a feature.
Or, do a screen share.
The moment I'm asked to write code on a whiteboard, I politely ask to use a computer. If they can't accommodate (I bring my own laptop just in case), the interview ends.
Or, do a screen share.
The moment I'm asked to write code on a whiteboard, I politely ask to use a computer. If they can't accommodate (I bring my own laptop just in case), the interview ends.
Yeah I agree that in a perfect world, people whose jobs will entirely consist of working on a computer, mostly in an IDE, should not be interviewed on a whiteboard. But the whiteboard is a filter for people who are so unfamiliar with programming that they can't reason without the crutch of an IDE. For example, I've seen some people who can do amazing things with Scratch IDE, but can't sit down in front of nano or vim and hammer out code to save their lives. IDE's can be a crutch for thorough learning of a language, and I learned that by leaning too hard on JetBrains, then trying to dev over ssh on a raspi.
how many times have you ended an interview that way?
That’s a good idea
In the real world, we aren’t writing code on the board. We may be drawing architecture. So why not sit them down to a computer and tell them to solve something approaching a real world problem?
We even have these things called “unit tests” that you can have already written and you can let them write code that make the unit tests pass as a pair programming problem.
We even have these things called “unit tests” that you can have already written and you can let them write code that make the unit tests pass as a pair programming problem.
That's a good question, as it seems I was not clear.
The execution risk and performance anxiety of being at the machine is, IMHO, higher. Your machine may have an ergo kbd; you may be used to using an editor that's not installed, or want to call a library that hasn't been installed, or... You can make various typos (include files, whatnot) and be embarrassed.
OTOH with a whiteboard, a friendly couple of interviewers are going to ignore if you wrote #include <iostreams> (which I type all the time) and look what you wrote. The compiler will not. etc.
I find it weird that companies try to make their interviews hazing rituals (which include but are not limited to the WTF of unfriendly interviewers). We want to hire someone to help us get to our goal. If the person is in for an interview they've already been through a phone screen, and the cost to the company of interviewing is high. Yes, a bad hire is a problem, but as the original tweet says: tricks that make things difficult do nothing to help you find a great person to work with. so try really hard to damp out the distracting transients -- make the process a low pass filter.
The execution risk and performance anxiety of being at the machine is, IMHO, higher. Your machine may have an ergo kbd; you may be used to using an editor that's not installed, or want to call a library that hasn't been installed, or... You can make various typos (include files, whatnot) and be embarrassed.
OTOH with a whiteboard, a friendly couple of interviewers are going to ignore if you wrote #include <iostreams> (which I type all the time) and look what you wrote. The compiler will not. etc.
I find it weird that companies try to make their interviews hazing rituals (which include but are not limited to the WTF of unfriendly interviewers). We want to hire someone to help us get to our goal. If the person is in for an interview they've already been through a phone screen, and the cost to the company of interviewing is high. Yes, a bad hire is a problem, but as the original tweet says: tricks that make things difficult do nothing to help you find a great person to work with. so try really hard to damp out the distracting transients -- make the process a low pass filter.
That also tells you something.
For instance, the first time I had a pair programming type of interview like I described, they in fact did have an ergonomic split keyboard for me to use. For physical reasons, I type with one hand and couldn’t possibly type efficienctly on one. I had no problem asking them for a normal keyboard. If they had seem to have a problem, that would have told me a lot about the company.
You could even ask what their favorite editor is before hand even as part of the phone screen.
For instance, the first time I had a pair programming type of interview like I described, they in fact did have an ergonomic split keyboard for me to use. For physical reasons, I type with one hand and couldn’t possibly type efficienctly on one. I had no problem asking them for a normal keyboard. If they had seem to have a problem, that would have told me a lot about the company.
You could even ask what their favorite editor is before hand even as part of the phone screen.
I’ve never heard a coherent explanation of what ‘inverting’ a binary tree even means.
And B-trees are much trickier to get right than binary trees. Anyone who can whiteboard a b-tree from memory without errors is some kind of savant.
I think this whole reverse a b-tree story is apocryphal.
And B-trees are much trickier to get right than binary trees. Anyone who can whiteboard a b-tree from memory without errors is some kind of savant.
I think this whole reverse a b-tree story is apocryphal.
I agree with it seems apocryphal. I can only imagine interpretations of the statement "invert a tree" that are so trivial as to test nothing (as in "invert the multiplication of A * B" uhh, "B * A"?).
I suppose if I were asked I'd ask for clarification of what was wanted, and either they want someone who asks such questions or both of us leave the room relieved that we won't be working together.
I suppose if I were asked I'd ask for clarification of what was wanted, and either they want someone who asks such questions or both of us leave the room relieved that we won't be working together.
So true. And so deliciously snarky. Confession: I'm guilty of far greater stupidity. I've asked "Ship of Theseus" questions when interviewing candidates for object database companies, and conservation puzzles, like tessellating a chessboard missing diagonal corners with dominoes. Because "Thank goodness I hired coders who can discuss philosophical identity problems". At least people who can reverse a b tree on a whiteboard can code an algorithm.
Interviewing has been solved a long time ago with the invention of FizzBuzz. If they can solve it, hire them. If they can't don't. If you're worried its too easy then turn up the difficulty and ask them write an implementation of fizzbuzz that makes use of specific features of a language or framework. If you are BigCo and need to crank the difficulty up really high you can ask the candidate to implement a solution to calculate which number contributes the most the letter z appearing in the output of fizzbuzz between the numbers 5 trillion and 143 trillion, preferably involving a Hadoop cluster.
If you're a SaaS company and you're hiring a sales person, instead of the typical "sell me this pen" you can even replace it with "sell me FizzBuzz".
I'm telling you, it's bullet proof.
If you're a SaaS company and you're hiring a sales person, instead of the typical "sell me this pen" you can even replace it with "sell me FizzBuzz".
I'm telling you, it's bullet proof.
How can FizzBuzz be complicated for anybody? The solution is so obvious. I thought it was some sort of trick question but apparently not.
http://wiki.c2.com/?FizzBuzzTest
http://wiki.c2.com/?FizzBuzzTest
You'd be amazed at the percentage of people who fail this test. I can't say I have interviewed many people, but even seeing one or two people fail this test for a technical role is mind-boggling.
I could imagine someone not from a CS background to fail it by perhaps not recognizing they need to the modulus operator or even being aware of its existence?
Outside that, yeah. It indeed boggles the mind that the vanilla version of it could be failed by anyone who has taken elementary programming course.
Outside that, yeah. It indeed boggles the mind that the vanilla version of it could be failed by anyone who has taken elementary programming course.
In order to test your real problem-solving skills, you should allow the use of Internet search, because not all computer professionals can remember a particular algorithm.
Well... vanilla FizzBuzz is easy peasy. Ask them to draw an architecture diagram for FizzBuzz Enterprise Edition on a whiteboard. Or ask them to implement it without using the modulus operator... muahaha. I'm sure that'd stump almost everyone.
You can make FizzBuzz incredibly complicated if you deviate from the vanilla FizzBuzz.
You can make FizzBuzz incredibly complicated if you deviate from the vanilla FizzBuzz.
Asking for it without using the modulus operator is a bit evil haha.
I like this exercise and I sent it to a friend who has a few years of programming experience but is still learning heavily.
The first solution lacked the number being printed but everything else was right. Second solution had the number, but he missed the FizzBall when the number was divisible by both three and five. The third solution lacked the Fizz when it was divisible by three. Finally his fourth solution was correct.
I like this exercise and I sent it to a friend who has a few years of programming experience but is still learning heavily.
The first solution lacked the number being printed but everything else was right. Second solution had the number, but he missed the FizzBall when the number was divisible by both three and five. The third solution lacked the Fizz when it was divisible by three. Finally his fourth solution was correct.
My favorite is the variation where the base numbers and replacement strings are supplied as run-time parameters.
RunFizzBuzz("");
RunFizzBuzz("3=Fizz,5=Buzz");
RunFizzBuzz("7=Rutabaga");It is really funny that some people spend months preparing for a job interview though it is equally morbid because they gotta be a bit insane to do it, and those are the people that will end up working on the next AI surveillance/manipulation technology.
For a person who put management as an algorithm, these computer professional nouns can not remember, and did not deliberately to learn.
The common data structure or algorithm questions to do the test content, can only recruit people who memorize by rote.
I think only need to give the initial data status and final data status, the candidate is required to write pseudo-code (or schematic).
https://github.com/linpengcheng/PurefunctionPipelineDataflow
The common data structure or algorithm questions to do the test content, can only recruit people who memorize by rote.
I think only need to give the initial data status and final data status, the candidate is required to write pseudo-code (or schematic).
https://github.com/linpengcheng/PurefunctionPipelineDataflow
BALL-PITS and CLOWN NOSES
I worked with a guy, Bob, in our college computer room. Bob had returned to get his BS after many years in industry. He had great stories. He also had great advice. He taught me that you can learn a lot about how a company will treat their employees by paying attention to the way they treat you during the interview. He suggested several sign that indicated it was best to just "walk away".
If you go into a restaurant for a professional lunch meeting and they have a ball-pit / swing set /etc. it might not be the most professional place to hold the meeting. Just walk away.
If you go to a 5 star restaurant and they want you to wear a clown nose (Everybody Does It!) Just walk away.
The big scam is "We hire the best and the brightest". "We mandate a whiteboard test (Everybody Does It)". It is my opinion (based on experience) that companies that insist on a whiteboard test, despite years of programming on your resume and open source code, are either Ball-Pit companies (who don't know what it means to be a professional) or Clown-Nose companies (who know what it means to be a professional but still treat you like a commodity). Just walk away.
When you object, please reference one study in a journal that shows that whiteboard tests are effective in any measure when used during interviews.
I understand why Google does it. I've even interviewed there. They get thousands of resumes (which they never read, at least nobody did at my interviews). They have to weed out the "learn coding in 6 weeks" crowd. But you're a professional. You should expect to be treated like one. You don't ask your surgeon to cut up hams in an interview. You don't ask your builder to nail two boards together. Professional accountants are not interviewd by bank tellers and ask to count change.
Google, like Microsoft before it (remember the "why are manholes round? quizes?"), have combined the marketing claim "We hire the best and brightest" with the ball-pit mentality of whiteboarding. Google HR (sorry, People Resource?) SHOULD be embarrassed but apparently they, and upper management, LIKE ball-pits.
A real professional interview involves a discussion about what the company does, what it current problems are (aka why are they hiring), and a discussion of if you, as a professional, have the skills to solve the problem. What do they need? How can you contribute? It is likely that your interviewer has no idea what you are being interviewed for and cannot answer these basic questions.
We, as professionals, should consider whiteboarding as an insult.
Just walk away.
I worked with a guy, Bob, in our college computer room. Bob had returned to get his BS after many years in industry. He had great stories. He also had great advice. He taught me that you can learn a lot about how a company will treat their employees by paying attention to the way they treat you during the interview. He suggested several sign that indicated it was best to just "walk away".
If you go into a restaurant for a professional lunch meeting and they have a ball-pit / swing set /etc. it might not be the most professional place to hold the meeting. Just walk away.
If you go to a 5 star restaurant and they want you to wear a clown nose (Everybody Does It!) Just walk away.
The big scam is "We hire the best and the brightest". "We mandate a whiteboard test (Everybody Does It)". It is my opinion (based on experience) that companies that insist on a whiteboard test, despite years of programming on your resume and open source code, are either Ball-Pit companies (who don't know what it means to be a professional) or Clown-Nose companies (who know what it means to be a professional but still treat you like a commodity). Just walk away.
When you object, please reference one study in a journal that shows that whiteboard tests are effective in any measure when used during interviews.
I understand why Google does it. I've even interviewed there. They get thousands of resumes (which they never read, at least nobody did at my interviews). They have to weed out the "learn coding in 6 weeks" crowd. But you're a professional. You should expect to be treated like one. You don't ask your surgeon to cut up hams in an interview. You don't ask your builder to nail two boards together. Professional accountants are not interviewd by bank tellers and ask to count change.
Google, like Microsoft before it (remember the "why are manholes round? quizes?"), have combined the marketing claim "We hire the best and brightest" with the ball-pit mentality of whiteboarding. Google HR (sorry, People Resource?) SHOULD be embarrassed but apparently they, and upper management, LIKE ball-pits.
A real professional interview involves a discussion about what the company does, what it current problems are (aka why are they hiring), and a discussion of if you, as a professional, have the skills to solve the problem. What do they need? How can you contribute? It is likely that your interviewer has no idea what you are being interviewed for and cannot answer these basic questions.
We, as professionals, should consider whiteboarding as an insult.
Just walk away.
Get a sense of somebody in an interview using minimal time, hire them, try them out and if they can’t do the job fire them.
That hones your skills in spotting fakes, and your skills in giving people the hard word. Both of which are incredibly valuable in business.
I don’t see the time trade off in having complex interview systems. It has to be mostly a power play.
I wonder when somebody will write the book “agile interviewing - twelve steps to get it done in an hour”
That hones your skills in spotting fakes, and your skills in giving people the hard word. Both of which are incredibly valuable in business.
I don’t see the time trade off in having complex interview systems. It has to be mostly a power play.
I wonder when somebody will write the book “agile interviewing - twelve steps to get it done in an hour”
The only bit I disagree with is "get a sense of somebody in an interview", which is likely to lead to unintentional discrimination.
Twitter snark is a larger net negative contribution to our profession than misguided interview techniques.
The criticism misses the point of these tests, no one cares if you can reverse a b-tree or not. It's meant to serve as proxy for a good candidate, one that can reverse a b-tree is expected to have more coding chops than someone that can't.
Whether they're good or bad proxies is certainly debatable, but at least that's asking the right question.
Whether they're good or bad proxies is certainly debatable, but at least that's asking the right question.
I can derive a b-tree reversal on a whiteboard in a Faraday cage given a few hours and a pot of coffee. I can memorize someone else's reversal solution in maybe half an hour. The first method tests my ability to logically break down a problem and find an elegant solution. The second tests my ability to memorize shit. Candidates don't have a chance to do the first during any interview I've ever heard of; so logically you'll be hiring those of the second variety. Good luck with that.
I don't think anyone is missing the point that an interview can't literally be exactly what the job entails.
Wasn't it a binary tree?
Reversing a binary tree is well defined and obvious even to people, who never heard of it. But a b-tree? I would simply turn the problem into walking the tree in reverse and don't reverse it in-place.
Reversing a binary tree is supposed to be an easy warmup question, not the meat of an interview. With a b-tree, you add some fluff to the problem.
I completely agree that "trick" questions only select for people comfortable with trick questions. It doesn't hurt if someone who is good at them that is on the team but let it happen organically -- and let it be a non-jerk, which is more important!
I completely agree that coding problems can be excuses for jerks to nitpick, and about the only "good" thing that could come out of such a situation is that the candidate learns that the company employs jerks and doesn't mind them talking to candidates.
However:
The whiteboard can be a welcoming tool if used right. How do you think, how do you talk about the problem, do you actually know the basics?
We use Java, C++, Python and Javascript. We do want you to know one of them. We don't care, in the interview if you forget or misname the name of a routine, or get the arguments backwards (you're not in front of a computer; you don't have to worry about your typing speed, looking up the name, unfamiliar settings on the machine you're in front of, house style etc). We don't care if you forgot a semicolon or even a closing brace, much less a #include. Use whichever language you're most comfortable from the set you'd be using in your job. The best "problems" should be less than a dozen lines, even without any fancy tricks.
We pick something boring, say from the standard library, and nothing tricky like a hash table. Really boring like floor(). If you can't figure out how floor() does its "magic" that's scary. If you need to look up how to balance a tree in place, that would be no big deal so don't put someone on the spot. If you claim to know python but can't print your outputm, that's bad. If you claim to know python and your indentation isn't clear on the whiteboard who cares?
I've told this story on HN before: almost 20 years ago I interviewed a candidate who wrote a routine to shuffle a deck of cards on the whiteboard (we use even simpler examples today). His program ended up elaborately coping the input to the output without shuffling. One of us asked, "Is that really a shuffle?" He said, "Oh I'm a fucking idiot" and fixed it. He was an awesome hire.
At one company we had a standard question (again, write XXX from the standard library) that had a special outlying case. Only one candidate got that, but we hired several people who missed it. Stuff like that can be caught in code review.
But I've interviewed several candidate who spoke well and claimed to know, say, C, but couldn't write "hello, world" in that language. Whiteboard caught it.