People are still bad at gauging their own interview performance(blog.interviewing.io)
blog.interviewing.io
People are still bad at gauging their own interview performance
http://blog.interviewing.io/people-are-still-bad-at-gauging-their-own-interview-performance-heres-the-data/
151 comments
> This is IMO and experience probably based on a "sour grapes" type defense mechanism
It's not just going to be 'sour grapes' but also going to be preparing themselves for the rejection they now believe is coming. They're making it so they're not surprised when it shows up or when they don't hear back.
They're also making themselves more open to other offers instead of focusing on this one.
It's not just going to be 'sour grapes' but also going to be preparing themselves for the rejection they now believe is coming. They're making it so they're not surprised when it shows up or when they don't hear back.
They're also making themselves more open to other offers instead of focusing on this one.
I absolutely judge a company by the interview process and the questions they ask.
"Would I want to work for a company where the employees pass their given process?"
"Would I want to work for a company where the employees pass their given process?"
When I was at Code for America I redid their hiring process to optimize for interviewee experience rather than hiring manager experience.
One of the things that really struck me at the time was that traditional interview processes are built around a notion of worker surplus and job scarcity. They make very little sense when the opposite condition applies.
Even little things like contacting all applicants within 24 hours really wowed people.
One of the things that really struck me at the time was that traditional interview processes are built around a notion of worker surplus and job scarcity. They make very little sense when the opposite condition applies.
Even little things like contacting all applicants within 24 hours really wowed people.
Oh gosh, this is totally true.
I honestly find it almost disrespectful when you feel some company is putting you through a pipeline. They set up an interview day, line everyone up, grind through them one by one.
The other thing is the technical interview which is unreflective of the job itself. In one recently, I was told before going in I had to code on a whiteboard. I ask, "Can I have I use a text editor, not an IDE or anything, just a text editor?" I get told, no (for nonsensical reasons I won't list here). That sort of arbitrariness and inflexibility turn me off, not to mention the standoffishness of having people stare over your shoulder and judge you rather than work as collaborators.
So I walked away from the interview. I'm happy where I am and you're gonna have to woo me, just a little bit, to get me to care.
Sometimes I wish we were unionized so we could set some standards (for many other things, but this as well).
I honestly find it almost disrespectful when you feel some company is putting you through a pipeline. They set up an interview day, line everyone up, grind through them one by one.
The other thing is the technical interview which is unreflective of the job itself. In one recently, I was told before going in I had to code on a whiteboard. I ask, "Can I have I use a text editor, not an IDE or anything, just a text editor?" I get told, no (for nonsensical reasons I won't list here). That sort of arbitrariness and inflexibility turn me off, not to mention the standoffishness of having people stare over your shoulder and judge you rather than work as collaborators.
So I walked away from the interview. I'm happy where I am and you're gonna have to woo me, just a little bit, to get me to care.
Sometimes I wish we were unionized so we could set some standards (for many other things, but this as well).
Lucky you, I was once asked to write code on a napkin and on another occasion to say lines of code over the phone :)
On the flip side, as an interviewer I give all candidates the opportunity to use a text editor, and less than 15% of candidates take it. The rest opt for whiteboard.
This was my experience as well (actually, even more extreme). Here's a room with a whiteboard or a laptop+TV. All 10 people I interviewed in that room chose the whiteboard.
One of them mentioned feeling less pressure to have perfect syntax and everything at a whiteboard than in an editor. In retrospect, wish I'd asked more of them about their preference, although at the same time, it wasn't something I judged them on so I wouldn't want them to think I was nitpicking their choice.
I'd probably choose a whiteboard too, honestly, for spatial reasons (able to write up separate things off to the side, easily annotate stuff in a different color to step through the code, etc).
One of them mentioned feeling less pressure to have perfect syntax and everything at a whiteboard than in an editor. In retrospect, wish I'd asked more of them about their preference, although at the same time, it wasn't something I judged them on so I wouldn't want them to think I was nitpicking their choice.
I'd probably choose a whiteboard too, honestly, for spatial reasons (able to write up separate things off to the side, easily annotate stuff in a different color to step through the code, etc).
I think that's because a lot of people are already trained to use whiteboard during interview.
That really truly surprises me. I'm so much quicker and more in my element with a text editor.
[deleted]
For what it's worth, one of the things I did with my interview process was to make all the coding involve an actual computer. I gave the interviewee their choice of editor, and was happy to let them use their own computer.
My theory was that in an interview I wanted to see people at their best. I wasn't hiring them to code on whiteboards or learn how to use new text editors; I was hiring them to spend a lot of time coding in an environment they liked.
More details here: http://williampietri.com/writing/2015/slightly-less-awful-hi...
My theory was that in an interview I wanted to see people at their best. I wasn't hiring them to code on whiteboards or learn how to use new text editors; I was hiring them to spend a lot of time coding in an environment they liked.
More details here: http://williampietri.com/writing/2015/slightly-less-awful-hi...
One of the things I've noticed is that scammers (ask for your SSN/birthday/etc, bad sales jobs, etc) are the most attentive with callbacks and 'hiring'. Because it's all they do and they actually optimize their hiring for applicant satisfaction.
I agree with you, but here's the thing. If the vast majority of companies are running like that and succeeding, does that not suggest that there really is a workeer surplus and job scarcity, and that impressions to the contrary are self-imposed?
Well, all it really shows is that the "market" in this instance is incredibly inefficient. People need jobs, and if the vast majority of companies have a poor interview process, and 1% have a good one, assuming that this is the only thing that people care about (it's not), all that's going to happen is the 1% get the employees they want, and the other 99% still get some employees.
The market is likely inefficient for the same reason that the advertising industry is inefficient - difficulty to show than any given change independently improves outcomes.
The market is likely inefficient for the same reason that the advertising industry is inefficient - difficulty to show than any given change independently improves outcomes.
For some jobs, sure. But what I really think it suggests is a power asymmetry. A company can do without 1 worker much more easily than a worker can do without 1 job. Plus, current processes let hiring managers feel powerful. Given the high percentage of hiring managers who are social primates, I expect that's a big factor.
In tech, though, you hear endless complaints about how hard it is to hire, so I didn't hesitate to optimize the process for interviewee happiness. I expect it would make people more likely to apply, more likely to stay through the process, more likely to accept an offer, and more likely to refer friends.
Regardless, there's enormous waste to be removed from the process for anybody who's willing to put a little thought and work in.
In tech, though, you hear endless complaints about how hard it is to hire, so I didn't hesitate to optimize the process for interviewee happiness. I expect it would make people more likely to apply, more likely to stay through the process, more likely to accept an offer, and more likely to refer friends.
Regardless, there's enormous waste to be removed from the process for anybody who's willing to put a little thought and work in.
> worker surplus and job scarcity
I'm convinced that's the only reason why processes at places like google aren't considered completely broken.
I'm convinced that's the only reason why processes at places like google aren't considered completely broken.
Google, Facebook, Amazon are all a different category than most other companies that are hiring software developers. They care a lot less about false negatives because their hiring pool is so large. They are prestigious places that a lot of people really want to work at. As a result they have more candidates than they can reasonably get through without some massive filtering.
Some of that filtering, whether intentional or not, is how rigorous the interview process is. They can afford to be that rigorous though because the fact is that even if they miss out on one really good candidate there are 10 or more replacements in the pipeline.
The same does not apply for smaller companies with less engineering centric workplaces. If they try to do it Google's way they'll be shooting themselves in the foot.
Some of that filtering, whether intentional or not, is how rigorous the interview process is. They can afford to be that rigorous though because the fact is that even if they miss out on one really good candidate there are 10 or more replacements in the pipeline.
The same does not apply for smaller companies with less engineering centric workplaces. If they try to do it Google's way they'll be shooting themselves in the foot.
Very true. Some well known companies, like Google, still have more applicants than jobs, and so they need to filter people out. Most companies have the opposite problem - they need to attract people. This shift means that the interview process has to be re-thought.
> "still have more applicants than jobs"
This is true for a lot of companies - but there's only a surplus of labor if the applicants are qualified.
For a company to be able to be frivolous with their process, they'd need to have more qualified applicants than jobs, and I think even for many name-brand BigCorps, this isn't the case.
This is true for a lot of companies - but there's only a surplus of labor if the applicants are qualified.
For a company to be able to be frivolous with their process, they'd need to have more qualified applicants than jobs, and I think even for many name-brand BigCorps, this isn't the case.
I suspect most name-brand BigCos (including Google and friends) are pretty bad at first-pass decisions about what applicants are qualified. Has anybody stopped to consider that the root cause of their stream of Terri le interview candidates is a terrible filtering process for deciding who gets interviewed?
>I absolutely judge a company by the interview process and the questions they ask.
And rightfully so.
The term these days is "candidate experience" for the overall interview process, which can include everything from a company's handling of logistics and time spent in a waiting room to the types of questions being asked.
That said, what I'm referring to in my comment isn't necessarily related to the interview process at all. A candidate can be asked questions that he/she felt were relevant and entirely fair for the job at hand, yet if he/she performed poorly there is still a tendency to try and find flaws in the overall opportunity.
My reference wasn't to candidates performing poorly and thus giving a bad review to the interview process. Rather, the candidate might perform poorly and say that the vacation time was inadequate or the commute would have been too long.
And rightfully so.
The term these days is "candidate experience" for the overall interview process, which can include everything from a company's handling of logistics and time spent in a waiting room to the types of questions being asked.
That said, what I'm referring to in my comment isn't necessarily related to the interview process at all. A candidate can be asked questions that he/she felt were relevant and entirely fair for the job at hand, yet if he/she performed poorly there is still a tendency to try and find flaws in the overall opportunity.
My reference wasn't to candidates performing poorly and thus giving a bad review to the interview process. Rather, the candidate might perform poorly and say that the vacation time was inadequate or the commute would have been too long.
A company's bad interview process might have nothing at all to do with the rest of the company.
At this point, isn't it generally accepted that all tech interviewing sucks, and companies would do better throwing darts at resumes?
If all interviewing is bad, and all companies are not bad, there's no benefit to judging a company based on that process.
At this point, isn't it generally accepted that all tech interviewing sucks, and companies would do better throwing darts at resumes?
If all interviewing is bad, and all companies are not bad, there's no benefit to judging a company based on that process.
People have been talking about the terribleness of tech interviewing for years. If a company isn't able/willing to improve important processes, that's definitely a sign to me.
There are many different constraints here. Many companies that hire software developers aren't necessarily in the "business" of software which colors their approach to hiring. They aren't experts in hiring developers so they tend to outsource it. As someone who is part of hiring decisions at just such a company I regularly see things like:
1. Pushback on work sample tests from the recruiting companies.
2. Upper management wondering why it's so hard to find candidates that can pass our work sample tests.
3. Suggestions that maybe we should be using an online coding test. (multiple choice questions of poor quality)
It can be really difficult for companies that don't have a lot of in-house experience here to figure out how to hire good developers. Much of what is known in Silicon Valley style companies isn't really known by HR in a smaller company in the midwest. They don't typically read HN or follow the tech blogs. And unless they luck into hiring a vocal Senior Developer like me to specifically guide their recruiting and interviewing process they won't even realize how badly their hiring process is.
1. Pushback on work sample tests from the recruiting companies.
2. Upper management wondering why it's so hard to find candidates that can pass our work sample tests.
3. Suggestions that maybe we should be using an online coding test. (multiple choice questions of poor quality)
It can be really difficult for companies that don't have a lot of in-house experience here to figure out how to hire good developers. Much of what is known in Silicon Valley style companies isn't really known by HR in a smaller company in the midwest. They don't typically read HN or follow the tech blogs. And unless they luck into hiring a vocal Senior Developer like me to specifically guide their recruiting and interviewing process they won't even realize how badly their hiring process is.
But then, apparently, there are virtually no good companies to work for. So most people shouldn't bother worrying about it.
Nope. I only have time for one job, and I want it to be the best one possible. It doesn't matter if most companies are shitty; I won't be working for most companies.
This has been a deepening mystery throughout my career. I see most companies are utterly uninterested in improving their recruitment processes, and are at the same time frustrated that most projects fail or don't quite deliver.
Meanwhile, the best engineers I know have to look for a really long time to find suitable jobs.
> "I see most companies are utterly uninterested in improving their recruitment processes"
I think it's a bit more nuanced than that. Most companies seem to be interesting in improving their recruitment processes - but certain fundamental systems and assumptions are treated as inviolate and sacred.
Here is a quick but incomplete list of the ones that come to mind:
- Open-ended unstructured interviews are the optimal way to evaluate candidates.
- Whiteboard coding is an acceptable, reliable, and reasonably accurate way to gauge coding ability.
- Algorithms and data structure knowledge predicts overall software engineering capability.
- Culture fit (in however nebulous ways) is a stronger predictor of future performance than standardized evaluation.
The list goes on.
The tech industry overall seems very interested in improving recruitment and hiring, so long as these fundamental tenets are never violated. Suffice it to say, this stands in contrast to tech's (in)famous claims of thinking outside the box, breaking the rules, and holding nothing sacred to disruption.
I've personally seen companies try to improve candidate sourcing, scoring systems, interview scheduling, etc etc, but practically never any of the above. Companies try really, really hard to optimize within an extremely narrow band of parameters.
I think it's a bit more nuanced than that. Most companies seem to be interesting in improving their recruitment processes - but certain fundamental systems and assumptions are treated as inviolate and sacred.
Here is a quick but incomplete list of the ones that come to mind:
- Open-ended unstructured interviews are the optimal way to evaluate candidates.
- Whiteboard coding is an acceptable, reliable, and reasonably accurate way to gauge coding ability.
- Algorithms and data structure knowledge predicts overall software engineering capability.
- Culture fit (in however nebulous ways) is a stronger predictor of future performance than standardized evaluation.
The list goes on.
The tech industry overall seems very interested in improving recruitment and hiring, so long as these fundamental tenets are never violated. Suffice it to say, this stands in contrast to tech's (in)famous claims of thinking outside the box, breaking the rules, and holding nothing sacred to disruption.
I've personally seen companies try to improve candidate sourcing, scoring systems, interview scheduling, etc etc, but practically never any of the above. Companies try really, really hard to optimize within an extremely narrow band of parameters.
Agreed, but I'd put it that way: most companies I see are very interested in pretending to improve their recruitment processes.
What they don't really want, is to take on people who are noticeably better than (or different in any way from) the people already working there.
What they don't really want, is to take on people who are noticeably better than (or different in any way from) the people already working there.
Perhaps a more exact personal observation would be that hiring managers always hire the same type of person.
> A company's bad interview process might have nothing at all to do with the rest of the company.
Everyone who works at a company, got there by passing through the] interview filter. A bad interview process has everything to do with the company, particularly in organizations that have grown a lot in the recent past.
Everyone who works at a company, got there by passing through the] interview filter. A bad interview process has everything to do with the company, particularly in organizations that have grown a lot in the recent past.
Yes and no. I have a large contingent of coworkers from another local firm that is shedding workers. As these are usually referrals, they either get a softball process or a rubber stamp.
At the company I work for, most of the employers were hired based on the strength of referrals from existing employees. The very first employees were people that the founders knew personally and thought highly of. I never had an interview because several of the employees had worked with me before and knew me. My first contact with the company was when they told me they wanted to hire me and extended an offer. I was very surprised by that because they hadn't even seen my resume or anything. That method of hiring seems to have worked out pretty well so far for employee hiring and retention. The employees we've been getting seem to be quite competent and stick around.
At the time I was interviewing with another company who seemed interested in me, but expressed a deep suspicion of new hires after having been burned in the past, and told me outright that they were skeptical of me, and that I could very well turn out to be lazy and unproductive. After three interviews in three weeks, they still hadn't made any offer. So when another company comes along unsolicited and asks me to join them because they know I'm good at what I do, I took their offer. When a company demonstrates that they value you, you're going want to work for them instead of the one that doesn't.
There are a few people at my current company that were hired with an interview based on the company needing skills that weren't available from referrals from employees, but that's uncommon. I don't even know if our company has any sort of standard interview process.
At the time I was interviewing with another company who seemed interested in me, but expressed a deep suspicion of new hires after having been burned in the past, and told me outright that they were skeptical of me, and that I could very well turn out to be lazy and unproductive. After three interviews in three weeks, they still hadn't made any offer. So when another company comes along unsolicited and asks me to join them because they know I'm good at what I do, I took their offer. When a company demonstrates that they value you, you're going want to work for them instead of the one that doesn't.
There are a few people at my current company that were hired with an interview based on the company needing skills that weren't available from referrals from employees, but that's uncommon. I don't even know if our company has any sort of standard interview process.
I wouldn't 100% base my acceptance of a job on an interview, but I would use it as one of many factors.
Were they on time to the interview? Did it seem like a good culture fit for me? Do they seem positive about working there? Was the interview full of poor communication? or confused looks?
Were they on time to the interview? Did it seem like a good culture fit for me? Do they seem positive about working there? Was the interview full of poor communication? or confused looks?
[Wouldn't] companies would do better throwing darts at resumes?
Or perhaps by actually reading resumes instead of keyword-filtering them. Really, it's not that hard.
"Throwing darts at resumes", OTOH, is exactly what many companies seem to be doing right now. Specifically, the practice of setting up a "chat" with an internal recruiter (who inevitably tells you that they think you'll be a "wonderful fit") before passing the resume over to the tech side (which is when you find out whether they really think you're a potential fit or not) comes to mind.
Or perhaps by actually reading resumes instead of keyword-filtering them. Really, it's not that hard.
"Throwing darts at resumes", OTOH, is exactly what many companies seem to be doing right now. Specifically, the practice of setting up a "chat" with an internal recruiter (who inevitably tells you that they think you'll be a "wonderful fit") before passing the resume over to the tech side (which is when you find out whether they really think you're a potential fit or not) comes to mind.
>Or perhaps by actually reading resumes instead of keyword-filtering them
This one is a two-way street. I launched a resume writing side-business a couple years ago (resumeraiders.com) because I saw how poor many developer resumes were and how they were getting ripped off by resume writers who had no experience in hiring or in tech. Most were just failed writers.
If you expect a recruiter to read a poorly-written 5 page resume when the yes/no interview decision could have easily been made based on a summary and a few decent bullet points, that's not on the recruiter. If you think the 5 page resume is some kind of anomaly, ask the recruiters at your company how common they are.
This one is a two-way street. I launched a resume writing side-business a couple years ago (resumeraiders.com) because I saw how poor many developer resumes were and how they were getting ripped off by resume writers who had no experience in hiring or in tech. Most were just failed writers.
If you expect a recruiter to read a poorly-written 5 page resume when the yes/no interview decision could have easily been made based on a summary and a few decent bullet points, that's not on the recruiter. If you think the 5 page resume is some kind of anomaly, ask the recruiters at your company how common they are.
This is fascinating to me. I recently did a resume swap with an American lady I know (I'm Australian). We're both around mid thirties. She's in SF.
Her resume was 1 page. It's the first time I've ever seen a 1 page resume in my life except for someone out of highschool with no history or experience.
Mine is 4 pages. I've worked really hard to cut it down to that while including relevant things and nice formatting/readability/white space. Compared to some of the 10 to 15 page resumes we get here for people who have done so little, it's positively succinct.
We had a chat, and we just had to agree that it was a cultural difference.
But still, I admit, if anyone asked me here, I'd say "you can't have a one page resume, unless you are literally applying for McDonald's and have done nothing with your life, this isn't twitter"
Her resume was 1 page. It's the first time I've ever seen a 1 page resume in my life except for someone out of highschool with no history or experience.
Mine is 4 pages. I've worked really hard to cut it down to that while including relevant things and nice formatting/readability/white space. Compared to some of the 10 to 15 page resumes we get here for people who have done so little, it's positively succinct.
We had a chat, and we just had to agree that it was a cultural difference.
But still, I admit, if anyone asked me here, I'd say "you can't have a one page resume, unless you are literally applying for McDonald's and have done nothing with your life, this isn't twitter"
There is a cultural element, and I've worked with candidates using longer form CVs to shorten them to the preferred US length. It's part formatting, part redundancy, and usually just sharing more information than is necessary.
The resume doesn't need to include everything you did. It's a highlight reel, not a bio. You don't need to list every responsibility or even every accomplishment.
If I forced you to tell me your background in one page, you'd be able to do it if you have any sense as to what is most impressive, what is important, and what isn't. The reason your resume is four pages is that it's acceptable where you are.
I once reduced a 26 page resume to a couple pages for a non-US client. It's usually just triage and deleting, some sentence "refactoring", and formatting. Not too tough, as long as my client doesn't have too much emotional attachment to every detail from their career.
The resume doesn't need to include everything you did. It's a highlight reel, not a bio. You don't need to list every responsibility or even every accomplishment.
If I forced you to tell me your background in one page, you'd be able to do it if you have any sense as to what is most impressive, what is important, and what isn't. The reason your resume is four pages is that it's acceptable where you are.
I once reduced a 26 page resume to a couple pages for a non-US client. It's usually just triage and deleting, some sentence "refactoring", and formatting. Not too tough, as long as my client doesn't have too much emotional attachment to every detail from their career.
Honestly, I'm incredibly skeptical that you need 4 pages.
Almost anyone can fit their entire experience onto a single page. You don't need to write an essay for each position—just highlight a few bullet points.
Almost anyone can fit their entire experience onto a single page. You don't need to write an essay for each position—just highlight a few bullet points.
To this and some other comments saying essentially "you can cut it down".
Oh sure! I can cut it down to 1 word if I want: awesome :p
One of my points is, I have literally not seen a 1 page resume before outside of fast food/janitorial work. Before chatting with her, I would have advised her to fix that up, because no one will take her seriously with a dot point resume that looks like it was put together to hit some automated text search and no natural human reading. I admit after some discussion, I fell back to "wow...they're not even pretending they're nothing but a keyword search meat grinder", because I do respect her so I thought she was telling the truth. And now I've got additional corroboration from HN.
I think there are arguments for longer/shorter resumes. In my, and my jobs/culture defence, I need to see you can write in plain English, have some sense of aesthetics, prioritisation, presentation. By that age I expect you've worked in several jobs (about which your summary might be my first impression/knowledge) and dot points aren't going to cut it, because your job titles, product names and keywords are meaningless noise to me: I joke my grandmother is an entrepreneur/CEO and has worked with everything. Your dot point resume looks like an indian from mechanical Turk put it together by amalgamation of keywords from job ads and pop culture. I want personal details, but summaries, on your projects, products, papers, in plain communicable error free English. 1 page screams kid/unprofessional to me. 2 - 4 pages unless you're 50 - 60 and have written 100 books, done something truly remarkable.
Now I'm not saying its right, I'm just saying that's my world.
Oh sure! I can cut it down to 1 word if I want: awesome :p
One of my points is, I have literally not seen a 1 page resume before outside of fast food/janitorial work. Before chatting with her, I would have advised her to fix that up, because no one will take her seriously with a dot point resume that looks like it was put together to hit some automated text search and no natural human reading. I admit after some discussion, I fell back to "wow...they're not even pretending they're nothing but a keyword search meat grinder", because I do respect her so I thought she was telling the truth. And now I've got additional corroboration from HN.
I think there are arguments for longer/shorter resumes. In my, and my jobs/culture defence, I need to see you can write in plain English, have some sense of aesthetics, prioritisation, presentation. By that age I expect you've worked in several jobs (about which your summary might be my first impression/knowledge) and dot points aren't going to cut it, because your job titles, product names and keywords are meaningless noise to me: I joke my grandmother is an entrepreneur/CEO and has worked with everything. Your dot point resume looks like an indian from mechanical Turk put it together by amalgamation of keywords from job ads and pop culture. I want personal details, but summaries, on your projects, products, papers, in plain communicable error free English. 1 page screams kid/unprofessional to me. 2 - 4 pages unless you're 50 - 60 and have written 100 books, done something truly remarkable.
Now I'm not saying its right, I'm just saying that's my world.
Honestly, your comment is extremely rude and offensive. I'm willing to chalk up your preference for longer resumes to cultural differences, but then you come out with bullshit like this:
> Your dot point resume looks like an indian from mechanical Turk put it together by amalgamation of keywords from job ads and pop culture.
> 1 page screams kid/unprofessional to me.
Your 4-page resume is unprofessional, overlong, self-centered, arrogant, and naive. You apparently are so bad at English that you can't even write concise summaries. A long resume screams that you have an overinflated ego and have no sense of professional decency.
A short resume doesn't imply a lack of aesthetics or English. I certainly paid attention to the design of mine. [0]
[0] https://www.morgante.net/resume.pdf
> Your dot point resume looks like an indian from mechanical Turk put it together by amalgamation of keywords from job ads and pop culture.
> 1 page screams kid/unprofessional to me.
Your 4-page resume is unprofessional, overlong, self-centered, arrogant, and naive. You apparently are so bad at English that you can't even write concise summaries. A long resume screams that you have an overinflated ego and have no sense of professional decency.
A short resume doesn't imply a lack of aesthetics or English. I certainly paid attention to the design of mine. [0]
[0] https://www.morgante.net/resume.pdf
I do not mean any offense. I really don't.
If you can think of a nice way to convey the same sentiment/message, but without the offence, I'm all ears and would happily edit the post.
Again, let me stress, I think there are arguments for long and short. I do not advocate for right and wrong, merely try to convey the reality I'm in, given that it's so different apparently from SF.
Realise also, in good faith, how offensive the "1 page resume" sounds to my ears. There is not any hyperbole present when I say I haven't ever seen one. Indeed, the only reason I believe it exists is because otherwise intelligent people I respect have told me it does, and now hacker news backs that up.
Really though, is it so hard to believe that people think that summarising yourself down to a page (as opposed to three or four...lol) is also an offensive idea?
Are we really so departed from one another that we believe that we can't just agree that the way our two respective cultures do things is offensive, especially given the fact that neither of them appear to have much success with outcomes regardless of which culture is in play?
If you can think of a nice way to convey the same sentiment/message, but without the offence, I'm all ears and would happily edit the post.
Again, let me stress, I think there are arguments for long and short. I do not advocate for right and wrong, merely try to convey the reality I'm in, given that it's so different apparently from SF.
Realise also, in good faith, how offensive the "1 page resume" sounds to my ears. There is not any hyperbole present when I say I haven't ever seen one. Indeed, the only reason I believe it exists is because otherwise intelligent people I respect have told me it does, and now hacker news backs that up.
Really though, is it so hard to believe that people think that summarising yourself down to a page (as opposed to three or four...lol) is also an offensive idea?
Are we really so departed from one another that we believe that we can't just agree that the way our two respective cultures do things is offensive, especially given the fact that neither of them appear to have much success with outcomes regardless of which culture is in play?
I'm replying to my own comment, since I unfortunately do not appear to be able to reply to the one below...
But...
You do realise what "mechanical Turk" is...as in...the Amazon service called "mechanical Turk" based upon the chess playing automaton, and that it's basic premise is often westerners paying people from poorer countries, say...India, to spit out quick small pieces of work for them.
There is no offense intended in these statements :/
But...
You do realise what "mechanical Turk" is...as in...the Amazon service called "mechanical Turk" based upon the chess playing automaton, and that it's basic premise is often westerners paying people from poorer countries, say...India, to spit out quick small pieces of work for them.
There is no offense intended in these statements :/
There is no offense intended in these statements.
One of the important lessons we learn in this life is that we're responsible for the offense we cause (or more specifically: the offense implicit in what we say), not the offense we intend.
One of the important lessons we learn in this life is that we're responsible for the offense we cause (or more specifically: the offense implicit in what we say), not the offense we intend.
I'm well aware of what mechanical turk is. Indeed, I've used it before.
It's offensive to imply that people with 1-page resumes just had it generated via mechanical turk. I've spent hours perfecting my resume to have a good layout and clear copy so it could fit on 1 page.
You really need to adjust your value system. Writing a short resume is harder than writing a longer one. In fact, I have a 4-page "master" resume which I pull from to build my 1-page version.
> There is no offense intended in these statements :/
Of course there is. You don't call skilled professionals "kids" unless you're trying to be offensive. Don't play dumb.
It's offensive to imply that people with 1-page resumes just had it generated via mechanical turk. I've spent hours perfecting my resume to have a good layout and clear copy so it could fit on 1 page.
You really need to adjust your value system. Writing a short resume is harder than writing a longer one. In fact, I have a 4-page "master" resume which I pull from to build my 1-page version.
> There is no offense intended in these statements :/
Of course there is. You don't call skilled professionals "kids" unless you're trying to be offensive. Don't play dumb.
wtf... do ppl write essays on resumes? even 4 pages is too long nowadays. need to tl;dr.
Well, understand that it is not uncommon where I live to draft a cover letter at least.
For government jobs it's even worse, there you basically do have to write essays against the "selection criteria" (don't get me started on what it actually achieves, again, I'm not trying to advocate).
The general rationale is often on several grounds:
-to create a "merit" based document trail - to create a basic barrier to entry that has to be crossed for all but the most basic positions. Think of it like a spam filter/ensuring candidates who want THAT job as opposed to just A job. The very thing you don't want is to have hundreds of 1 page dot point resumes coming in. How would you possibly find the time to go through them? You want to stop people applying just by sending in a generic summary resume and clicking a button. You want people who are willing to put a bit of work in to apply specifically to the job you're advertising.
It is not uncommon for me, for instance, to write a cover letter for each job, and craft a specific version of my resume for each position, emphasising and exposing those parts of my experience most applicable to that particular position.
You can only really do one or two of these well per week, but that's kinda the point.
For government jobs it's even worse, there you basically do have to write essays against the "selection criteria" (don't get me started on what it actually achieves, again, I'm not trying to advocate).
The general rationale is often on several grounds:
-to create a "merit" based document trail - to create a basic barrier to entry that has to be crossed for all but the most basic positions. Think of it like a spam filter/ensuring candidates who want THAT job as opposed to just A job. The very thing you don't want is to have hundreds of 1 page dot point resumes coming in. How would you possibly find the time to go through them? You want to stop people applying just by sending in a generic summary resume and clicking a button. You want people who are willing to put a bit of work in to apply specifically to the job you're advertising.
It is not uncommon for me, for instance, to write a cover letter for each job, and craft a specific version of my resume for each position, emphasising and exposing those parts of my experience most applicable to that particular position.
You can only really do one or two of these well per week, but that's kinda the point.
By all means -- skim the resumes first, and pull out all 5-pagers and other immediate fail cases (which are generally easy to spot). And save your reading skills for the 15% or so which might have something worth your time. That's what I meant.
> "Throwing darts at resumes"
That reminds me of another technique - that is you select roughly half of the resumes from the stack and bin it. Nobody wants to work with unlucky people ;)
That reminds me of another technique - that is you select roughly half of the resumes from the stack and bin it. Nobody wants to work with unlucky people ;)
That's kind of what a lot of companies seem to do these days, by default:
"Eh, I don't have time for this. Let's get the junior people to filter resumes and do phone screens. Especially when trying to hire senior people... Too many resumes? Just make them do a 3-hour HackerRank session first. It's what all the other places are doing..."
"Eh, I don't have time for this. Let's get the junior people to filter resumes and do phone screens. Especially when trying to hire senior people... Too many resumes? Just make them do a 3-hour HackerRank session first. It's what all the other places are doing..."
>At this point, isn't it generally accepted that all tech interviewing sucks, and companies would do better throwing darts at resumes?
Except I've had good interview experiences. So, no, there are better and worse kinds of interviews. Just because it is an overwhelmingly neglected aspect of the industry is neither here nor there.
Except I've had good interview experiences. So, no, there are better and worse kinds of interviews. Just because it is an overwhelmingly neglected aspect of the industry is neither here nor there.
All tech interviewing does not suck, just most. A company that had a good hiring process would definitely have gone a long way to selling me on working there.
>A company's bad interview process might have nothing at all to do with the rest of the company.
The rest of the company didn't care enough to fix a bad interview process. That has plenty to do with the rest of the company.
>isn't it generally accepted that all tech interviewing sucks,
No. The interview process I went through was phenomenal. And my job is very pleasant and rewarding.
The rest of the company didn't care enough to fix a bad interview process. That has plenty to do with the rest of the company.
>isn't it generally accepted that all tech interviewing sucks,
No. The interview process I went through was phenomenal. And my job is very pleasant and rewarding.
>some portion of interviewees are losing interest in joining your company just because they didn’t think they did well, despite the fact that they actually did.
Wait. They think they didn't do well, got an offer, but turned it down because they think they didn't do well on the interview?
Wait. They think they didn't do well, got an offer, but turned it down because they think they didn't do well on the interview?
Usually, in my experience anyway, it doesn't get that far.
The blog post just refers to interviewees saying "I wouldn't want to work with the interviewer" when they mistakenly feel that they did poorly in the interview.
Anecdotally from the real world, when I (headhunter) get feedback from a candidate shortly/immediately after an interview, if they feel they did poorly they tend to add that there were things they didn't like about the opportunity (a person, the office, something). People that feel they did well don't usually go negative as easily.
But I also go back to the employer and get their feedback ASAP, and when that comes back positive I immediately pass that message to the candidate. Assuming the employer's feedback is somewhat prompt, the candidate doesn't get too much opportunity to go negative, and once they "feel wanted" by the employer they usually trend back positive. Offers come shortly thereafter.
The blog post just refers to interviewees saying "I wouldn't want to work with the interviewer" when they mistakenly feel that they did poorly in the interview.
Anecdotally from the real world, when I (headhunter) get feedback from a candidate shortly/immediately after an interview, if they feel they did poorly they tend to add that there were things they didn't like about the opportunity (a person, the office, something). People that feel they did well don't usually go negative as easily.
But I also go back to the employer and get their feedback ASAP, and when that comes back positive I immediately pass that message to the candidate. Assuming the employer's feedback is somewhat prompt, the candidate doesn't get too much opportunity to go negative, and once they "feel wanted" by the employer they usually trend back positive. Offers come shortly thereafter.
It blows my mind that any applicant gives negative feedback to the company before finding out of they got an offer. Even if they hated the interview and would never want to work there, an offer is still fodder to get a better offer somewhere else. Being positive is a strictly dominant strategy before an offer is made.
My comment was as a 3rd party recruiter, and the blog post was based on feedback from anonymous interviewees after anonymous interviews. I don't expect that candidates would give negative feedback to a company directly without having a sense of the employer's opinion.
It doesn't have to be as extreme as that.
If I think I did poorly on an interview, I will probably stop following up with the company and will more aggressively pursue other options. In particular, I might accept another offer prematurely because I assume one isn't forthcoming from the company.
As the post mentions, this relates heavily to timing. It's the long delays which many companies introduce that force candidates to make assumptions/prune early, especially since some companies will never even tell you you were rejected. Speeding up candidate responses is one of the easiest ways to dramatically improve a hiring process.
If I think I did poorly on an interview, I will probably stop following up with the company and will more aggressively pursue other options. In particular, I might accept another offer prematurely because I assume one isn't forthcoming from the company.
As the post mentions, this relates heavily to timing. It's the long delays which many companies introduce that force candidates to make assumptions/prune early, especially since some companies will never even tell you you were rejected. Speeding up candidate responses is one of the easiest ways to dramatically improve a hiring process.
> Wait. They think they didn't do well, got an offer, but turned it down because they think they didn't do well on the interview?
Yup. Happened to me: I felt like I did poorly in one of the interviews and had to ask myself whether I'd be happy in the job if the day-to-day was like the interview.
It probably wouldn't have been, but I had another offer on the table and the negative experience swayed me in the direction of the other offer.
Yup. Happened to me: I felt like I did poorly in one of the interviews and had to ask myself whether I'd be happy in the job if the day-to-day was like the interview.
It probably wouldn't have been, but I had another offer on the table and the negative experience swayed me in the direction of the other offer.
Yep. They had a bad experience during the interview. When they ask themselves "how do I feel about this company?", their main data point is negative.
I think the idea is that they think they didn't do well, decide that on second thought, they don't really want the job anyway, and move on to the next possible opportunity. Then if later they get an offer, they don't change their mind.
No, that's not what he said. After they think they didn't do well, people subconsciously start looking for reasons why they don't want to work there anyway: commute is bad, maybe they won't fit the culture, no free lunches etc.
My wife is currently in this situation. She's not a new job seeker by any means; she does mid-to-high-level management, leading complex projects. She recently applied for a new job that she's really enthusiastic about, but with them not responding, she's rapidly losing interest. She would be a perfect fit, but lack of a timely HR response could cost them a good employee.
Of course lack of a timely response also reflects badly on the company, so that could also be a very valid reason for people to lose interest.
Of course lack of a timely response also reflects badly on the company, so that could also be a very valid reason for people to lose interest.
My current employer has a great way of giving feedback about this. If you like the candidate her something along the lines of, "I really enjoyed our conversation." It's not a commitment to anything, but as you move through the process you know that you are doing well.
For candidates who don't do well? Thank them for the time and move on.
For candidates who don't do well? Thank them for the time and move on.
The company I work at has a good solution to this. We fly candidates out for an onsite interview and at the end of the day, if we decide we want to hire them, we give them an offer in person. Our goal is to get them to verbally accept before they head back home.
Kind of high pressure, no?
We don't pressure them, just give them the offer and continue to sell the company. It isn't like we hold the door shut until they accept :)
I'd be curious what percentage accept/reject on the spot, but I'd never ask a candidate to provide an answer either way if you present an offer right away. It's too important a decision for them not to at least take the night (especially after a day of interviews, a flight/hotel stay, an impending flight home). Even suggesting an immediate answer feels salesy ('whaddya think?'), and I'd think we'd see better results by presenting the offer and saying "take this home with you and get back to us when you're ready".
This approach should also tend to reduce counteroffer acceptances or accept/renege scenarios.
This approach should also tend to reduce counteroffer acceptances or accept/renege scenarios.
I like asking the question, "if we were to extend you an offer you like, would you accept it?" It's not perfect, but I haven't of a better approach that isn't pushy.
How about "Is there anything in your situation right now that would prevent you from accepting a solid competitive offer?" That's a little less salesy IMO.
If "not illegally confining someone" is all it takes not to be a "high-pressure sale tactic", then I confess I don't understand the term.
If you were to indicate, in any way, that you'd retract the offer unless I accept right there before I head home, I'd reject it right there before I head home.
In other words, I don't think that's a good solution at all.
In other words, I don't think that's a good solution at all.
That quote reminded me of something my mentor told me when I was training to interview people in-person. He said if they're really struggling, change it up with an easy question for exactly the reason you gave, but also to get their confidence up so they don't start the next interview feeling bad.
I would draw the opposite conclusion from that data. The mode difference was 0, and 91% of people were within 1 star of the interviewer.
Without knowing what the interviewer's bar is for any particular star level, I don't see how the interviewees could do any better. Rounding to 1-star increments amplifies relatively small changes -- if the interviewer rounds up 3.6 to 4 and you round 3.4 down to 3.
Without knowing what the interviewer's bar is for any particular star level, I don't see how the interviewees could do any better. Rounding to 1-star increments amplifies relatively small changes -- if the interviewer rounds up 3.6 to 4 and you round 3.4 down to 3.
Yeah, the way the data was presented was not informative. The heat maps are interesting to look at, but don't really convey the information well - the colors are basically useless. You have to hover over everything, at which point it should be presented as a table or a better picture.
It is pretty clear from one of the plots that interviewers and interviewees grade on a different curve - the mean interviewee scores themselves lower than the mean interviewer does. But once you correct for this, it seems that people are actually pretty good judges of how well they did. I'm honestly surprised by how low the R-squared was, I wonder if they included an intercept?
It is pretty clear from one of the plots that interviewers and interviewees grade on a different curve - the mean interviewee scores themselves lower than the mean interviewer does. But once you correct for this, it seems that people are actually pretty good judges of how well they did. I'm honestly surprised by how low the R-squared was, I wonder if they included an intercept?
"You probably interviewed as bad as you think you did" is not a catchy clickbait title.
+/- 1 star is a huge range. That's 50th to 90th percentile.
There's also the possibility that interviewees are better at rating themselves than interviewers.
For the purpose of this article, interviewer ratings are considered the absolute truth while interviewee ratings are not, which is obviously not always the case in reality.
For the purpose of this article, interviewer ratings are considered the absolute truth while interviewee ratings are not, which is obviously not always the case in reality.
>some portion of interviewees are losing interest in joining your company just because they didn’t think they did well, despite the fact that they actually did.
Maybe this is because the interviewee feels partly abused, as if communication isn't a two-way street. I think interviews would be less terrible if interviewers actually told you at the end that you did above/aswell as/below expectations, and/or why instead of just waiting 1-2 weeks before getting back to you and then telling you that you are/aren't moving on to the next step. A huge part of the interview process produces emotional ups/downs that no one wants to deal with and which ostracize potential employees, but the interviewers don't really do much to help with this.
I also think that having a 1-4 range is not a good guide for gathering sweeping datasets. Adding any amount of numbers will help to show actually how much. Or actually using human language to define them would be better. 1) Terrible/Unhirable. 2) Unhirable/Not Passing 3) Passing 4) Exemplar would actually be beneficial right now arbitrary numbering doesn't help to give people much context as to what they should choose, and the numbers are so close that you're going to get outliers where maybe some people think that a 2 is passable but not great, and interviewer thinks 3 is passable but not great, or vice-versa.
Maybe this is because the interviewee feels partly abused, as if communication isn't a two-way street. I think interviews would be less terrible if interviewers actually told you at the end that you did above/aswell as/below expectations, and/or why instead of just waiting 1-2 weeks before getting back to you and then telling you that you are/aren't moving on to the next step. A huge part of the interview process produces emotional ups/downs that no one wants to deal with and which ostracize potential employees, but the interviewers don't really do much to help with this.
I also think that having a 1-4 range is not a good guide for gathering sweeping datasets. Adding any amount of numbers will help to show actually how much. Or actually using human language to define them would be better. 1) Terrible/Unhirable. 2) Unhirable/Not Passing 3) Passing 4) Exemplar would actually be beneficial right now arbitrary numbering doesn't help to give people much context as to what they should choose, and the numbers are so close that you're going to get outliers where maybe some people think that a 2 is passable but not great, and interviewer thinks 3 is passable but not great, or vice-versa.
>I think interviews would be less terrible if interviewers actually told you at the end that you did above/as well as/below expectations
This happens quite often in phone interviews. My clients have regularly told candidates "We'll be inviting you in" at the end of a phone interview.
The issue with this for on-site interviews is that they tend to be with multiple individuals/groups, and getting those groups together to discuss the candidate's performance while the candidate is still on-site is a mess logistically. It would be ideal, but usually there will need to be some discussions among the panels.
This happens quite often in phone interviews. My clients have regularly told candidates "We'll be inviting you in" at the end of a phone interview.
The issue with this for on-site interviews is that they tend to be with multiple individuals/groups, and getting those groups together to discuss the candidate's performance while the candidate is still on-site is a mess logistically. It would be ideal, but usually there will need to be some discussions among the panels.
> usually there will need to be some discussions among the panels.
The way we've handled that (startup, ~30 devs) is that the VP of Engineering goes last, and will go around getting quick reactions from everyone on the loop. If everyone says "I'm all for hiring", the VP will go ahead and let the candidate know we'll be making an offer. If anyone says "I'd like to discuss", then we don't give feedback until after getting together to discuss the candidate.
The way we've handled that (startup, ~30 devs) is that the VP of Engineering goes last, and will go around getting quick reactions from everyone on the loop. If everyone says "I'm all for hiring", the VP will go ahead and let the candidate know we'll be making an offer. If anyone says "I'd like to discuss", then we don't give feedback until after getting together to discuss the candidate.
That's a solid idea, so long as the VP trusts the devs' judgement and ability to make a decision on something this important quickly. The one issue I see with this is that you'd need the "I'm all for hiring" to essentially mean "Not a single red flag - perfect candidate", which I'd assume must be the case.
There are lots of positives in providing that immediate positive feedback - showing the ability to make a quick decision, organization, and not least making the candidate feel wanted right away.
There are lots of positives in providing that immediate positive feedback - showing the ability to make a quick decision, organization, and not least making the candidate feel wanted right away.
> Therefore, every bar above 0 is impostor syndrome country, and every bar below zero belongs to its foulsome, overconfident cousin, the Dunning-Kruger effect.
Dunning-Kruger effect is attributed to "a metacognitive inability of the unskilled to recognize their ineptitude." [1] Seems like a harsh accusation for being off by a single star.
And the author makes a generalization that the imposter effect is "better" than Dunning-Kruger effect. I don't clearly see why this would be the case for the interviewee.
Dunning-Kruger effect is attributed to "a metacognitive inability of the unskilled to recognize their ineptitude." [1] Seems like a harsh accusation for being off by a single star.
And the author makes a generalization that the imposter effect is "better" than Dunning-Kruger effect. I don't clearly see why this would be the case for the interviewee.
I'm so mind-bendingly sick of these two terms (Dunning-Kruger and Imposter Syndrome). They're very much in-vouge as of late and it seems like an overwhelming number of /r/programming and HN posts have at least some reference to it.
Pulling some piece of psychology jargon out of thin air doesn't make it immediately applicable or accurate. Presumably, the terms get used so frequently for the "look how smart I sound" effect, which serves only to piss me off.
I would categorize these two groups as "overconfident" and "under-confident", at most. They're not suffering from some genuine long term psychological condition just because you think it sounds cool to use the terms.
/rant
Pulling some piece of psychology jargon out of thin air doesn't make it immediately applicable or accurate. Presumably, the terms get used so frequently for the "look how smart I sound" effect, which serves only to piss me off.
I would categorize these two groups as "overconfident" and "under-confident", at most. They're not suffering from some genuine long term psychological condition just because you think it sounds cool to use the terms.
/rant
Yep. You can pretty much tell when people use those terms that they're more driven by a desire to use those terms than out of having anything interesting to say.
Isn't a single instance not indicative of a syndrome? The syndrome presence or not can only be inferred if the same candidate heard multiple times on various interviews is consistently under- or overvaluing. And by a noticeable amount too.
Maybe I'm missing something, but why isn't the implication going the other direction?
In my experience, the problem is that interviewers have no idea how to correctly value a candidate's performance. Maybe the candidates are closer to being well-calibrated, but their self-assessments don't match up with the interviewers' because the interviewers don't know how to gauge what they are looking for?
Making the assumption that an interviewer knows how to measure the response of a candidate, even in cases of extremely quantitative questions with well-defined answers, is highly suspect to me. I think virtually no one knows how to do that effectively.
In my experience, the problem is that interviewers have no idea how to correctly value a candidate's performance. Maybe the candidates are closer to being well-calibrated, but their self-assessments don't match up with the interviewers' because the interviewers don't know how to gauge what they are looking for?
Making the assumption that an interviewer knows how to measure the response of a candidate, even in cases of extremely quantitative questions with well-defined answers, is highly suspect to me. I think virtually no one knows how to do that effectively.
:)) Just the other day a manager said "HR is overwhelmed. They have a student that filters the CVs and she is doing all she can." The senior programmer exploded "How is a student any good at filtering programmers, she does not know anything about programming!".
If you consider how well someone does as how likely they are to be hired, then it makes sense that interviewers are the source of "truth", because it's their grade or opinion which will decide whether you get hired or not.
We certainly can speculate whether whose opinions are more correct or valid, but if we just objectively consider that doing perfectly means you get hired, then what the interviewer thinks matters and what the interviewee thinks doesn't.
I do think you raise an interesting question though. I do wonder whether, in a scenario where many interviewers saw the same interview performance, how varied their scores would be.
After all, most people fail at many job interviews before landing one they get, but is that because of variation in the performance of the interviewee or because of the differences in interviewers?
We certainly can speculate whether whose opinions are more correct or valid, but if we just objectively consider that doing perfectly means you get hired, then what the interviewer thinks matters and what the interviewee thinks doesn't.
I do think you raise an interesting question though. I do wonder whether, in a scenario where many interviewers saw the same interview performance, how varied their scores would be.
After all, most people fail at many job interviews before landing one they get, but is that because of variation in the performance of the interviewee or because of the differences in interviewers?
Using the measurement, "how likely are they to be hired" seems like it's precisely the problem. "How likely someone is to be hired" is a totally erratic and unpredictable property of a bureaucratic hiring process. It's exactly not the sort of thing you want to use to define an absolute notion of skill or performance in a job interview.
This is my concern too. Any person who's gone in for many interviews has seen that the interviewer isn't the absolute measure of performance: we've all seen problems that are just trivia questions, are designed to show off the skills of the interviewer, or (worst of all) where a second solution isn't accepted because it's not what the interviewer thought of. All of these issues can result in disagreements on performance, in both directions.
I would like to see a different analysis: for multiple interviewers of the same candidate, how similar are the ratings?
I would like to see a different analysis: for multiple interviewers of the same candidate, how similar are the ratings?
These interview training programs seems to be really reminiscent of coding bootcamps and appear to have grown rapidly in a very short period of time. I think I'd almost prefer interviewees being judged based on the schools they attended and companies they worked at. I'm not sure that I find the ability to cram for data structures and algorithms questions to be indicative of intelligence and problem solving skills. Schools attended and companies worked at could be a very strong potential signal of applicant performance and not involve several rounds of "implement that algorithm from memory". Are interviews becoming more difficult as a result of the proliferation of all the interview test prep material?
Algorithms and elite schools doesn't tell me whether you'll use the right-sized solution for a problem and be able to prioritize and focus on tasks accordingly. It doesn't tell me if you can perceive and extract requirements well and have a process of communication and a professionalism of management in how you go about getting to solutions that work for the humans that will be using it.
It also doesn't tell me about your personal value system and whether you are passionate about creating good user experiences or if you are just interested in a technical playground or if you are a social climber or what.
It also doesn't tell me if you are highly opinionated and want to do things your own way or if you are flexible enough to deal with the imperfections of the real world. I don't know whether you value the mantra that the perfect shouldn't be the enemy of the good.
No, none of these brain teasers or abstract bootcamp style things tell me that - and those are the actual things that will make or break a company and determine whether your product will hit the market on time. Those are the things that will get you a customer base and will create a company culture of productivity and not something that doesn't lead to results.
Until interviews become more about how people approach work and their ethic and process of conducting it (which is a major part of classical engineering), these are ineffective tools for finding the right people.
---
[1] although as a disclaimer, I strongly prefer companies of under 20 employees. If you have 200, then the employees can be more insulated from the exposure to these types of risk and the necessity of this type of vertical thinking. But it's an "insulation" - everyone is working either directly or tangentially on something that is intended to be used by something in the world (usually humans).
It also doesn't tell me about your personal value system and whether you are passionate about creating good user experiences or if you are just interested in a technical playground or if you are a social climber or what.
It also doesn't tell me if you are highly opinionated and want to do things your own way or if you are flexible enough to deal with the imperfections of the real world. I don't know whether you value the mantra that the perfect shouldn't be the enemy of the good.
No, none of these brain teasers or abstract bootcamp style things tell me that - and those are the actual things that will make or break a company and determine whether your product will hit the market on time. Those are the things that will get you a customer base and will create a company culture of productivity and not something that doesn't lead to results.
Until interviews become more about how people approach work and their ethic and process of conducting it (which is a major part of classical engineering), these are ineffective tools for finding the right people.
---
[1] although as a disclaimer, I strongly prefer companies of under 20 employees. If you have 200, then the employees can be more insulated from the exposure to these types of risk and the necessity of this type of vertical thinking. But it's an "insulation" - everyone is working either directly or tangentially on something that is intended to be used by something in the world (usually humans).
> I'm not sure that I find the ability to cram for data structures and algorithms questions to be indicative of intelligence and problem solving skills.
I liked the interview approach of my last job - they had me debug some simple code. This was C++ for gamedev, so they showed me - a class violating the rule of 3 leading to a double delete, some multi-threaded code without memory barriers - or even volatile - and another example that escapes me at the moment. I was able to blitz through that. I floundered slightly when asked to explain the observed behavior of the buggy code pre-fixes - the exact behavior of the multi-threaded code before fixes surprised me, and I couldn't figure out the probable cause in my own head - but that segued easily into a discussion of the disassembly (at which point I was able to explain the behavior) when I suggested I'd look at the disassembly when asked how I'd go about finding out.
They asked about basic usage and design on a broader range of topics as well - I failed to remember to store a salt (doh!) for the simple user DB example schema, and floundered a little on my 3D math skills (in what to apply to solve a problem, not to recite the exact math - an area I admitted up front I was weak in) but otherwise did okay. Also a lot of discussion about previous problems I'd tackled and how (I've got a file of notes I keep called "War Stories".) I ended up getting the job at the price I named - they didn't even try to negotiate me down, perhaps I should've asked for more? :3
I liked the interview approach of my last job - they had me debug some simple code. This was C++ for gamedev, so they showed me - a class violating the rule of 3 leading to a double delete, some multi-threaded code without memory barriers - or even volatile - and another example that escapes me at the moment. I was able to blitz through that. I floundered slightly when asked to explain the observed behavior of the buggy code pre-fixes - the exact behavior of the multi-threaded code before fixes surprised me, and I couldn't figure out the probable cause in my own head - but that segued easily into a discussion of the disassembly (at which point I was able to explain the behavior) when I suggested I'd look at the disassembly when asked how I'd go about finding out.
They asked about basic usage and design on a broader range of topics as well - I failed to remember to store a salt (doh!) for the simple user DB example schema, and floundered a little on my 3D math skills (in what to apply to solve a problem, not to recite the exact math - an area I admitted up front I was weak in) but otherwise did okay. Also a lot of discussion about previous problems I'd tackled and how (I've got a file of notes I keep called "War Stories".) I ended up getting the job at the price I named - they didn't even try to negotiate me down, perhaps I should've asked for more? :3
Maybe if you've been fortunate to have attended an elite school. For the rest of us no name state school alumns, there's the nice illusion of meritocracy either way.
>I think I'd almost prefer interviewees being judged based on the schools they attended and companies they worked at.
I'd be surprised if you're saying this as somebody who doesn't have a track record of working at big name companies who went to a big name school, but you probably are so.
I'd be surprised if you're saying this as somebody who doesn't have a track record of working at big name companies who went to a big name school, but you probably are so.
At this rate, no matter how coding interviews change, there will be a new bootcamp or training program created to assist applicants to pass them.
I often wonder how would these people hire soldiers? "So I see you have no previous experience in killing people... We are looking for people that have at least 5 years of experience in killing people."
My point is you need smart enough people for your job and nothing more. You do an IQ test, a personality one and that's all. The rest can be learned.
My point is you need smart enough people for your job and nothing more. You do an IQ test, a personality one and that's all. The rest can be learned.
Not that I disagree with you, but sadly, most companies don't seem to want to train people anymore. They rather push that responsibility to either universities or the applicants themselves.
In fairness, I do think that someone who is very comfortable with a language or framework will be able to do things much faster than someone who isn't. Considering how quickly people change jobs nowadays, I can also see how companies don't want to train people only so they will leave shortly after or even before they're useful.
Obviously this is just based on my own experience in the US.
In fairness, I do think that someone who is very comfortable with a language or framework will be able to do things much faster than someone who isn't. Considering how quickly people change jobs nowadays, I can also see how companies don't want to train people only so they will leave shortly after or even before they're useful.
Obviously this is just based on my own experience in the US.
15 years ago, my employer easily invested $15-20k in training into me and most of the other techs. It was a major factor in me sticking around vs taking a raise at another employer.
We've reverted to the mean in recent years unfortunately. No training, and lots of nickel and dime bullshit. Today, if I found the right role, it would just be a matter of finding the right number.
We've reverted to the mean in recent years unfortunately. No training, and lots of nickel and dime bullshit. Today, if I found the right role, it would just be a matter of finding the right number.
Define "much faster." I am seriously trying to think of a developer I'd describe as competent and smart who could not pick up a new language or framework with sufficient understanding to be minimally productive (fixing minor to moderate bugs, contributing to new if simple features) inside a week or two. If that's not fast enough, something has gone terribly wrong planning the project.
I actually didn't mean the time it took for them to start contributing, I meant the time it would take for them to complete the same task.
Someone who knows the ins and outs of their language won't have to look up or learn new stuff as they go for example. This isn't even necessarily restricted to languages or frameworks, if you've done something similar to an assigned task, you would likely have already encountered certain pitfalls and know the best way to complete the task.
Someone who knows the ins and outs of their language won't have to look up or learn new stuff as they go for example. This isn't even necessarily restricted to languages or frameworks, if you've done something similar to an assigned task, you would likely have already encountered certain pitfalls and know the best way to complete the task.
I agree with you about the training. However if the stack is too specific, then the stack it's like a fingerprint and they are looking for somebody with the same fingerprint...
This is actually one of my interview red flags. For example if I say "I've worked with VB.NET 5 years, but not much C#" and they want 1 year C# experience. If they say that's a problem most of the time they don't know what they're talking about or they're not a company willing to invest in training their employees. I want a job that lets me learn.
@Jemmeh I "failed" a similar interview (VB, C#) Don't they know that the VB and C# examples are shown on Microsoft's site side by side?!
The only conclusion I reached is that one should navigate in life in such a way that your well being should not depend on people that are less smart, less experienced, less curious than yourself.
The only conclusion I reached is that one should navigate in life in such a way that your well being should not depend on people that are less smart, less experienced, less curious than yourself.
I think this is probably because there is still a stigma associated with being a VB programmer. If you knew F# instead you might have been hired anyway.
What do you define as "training?"
Personally I don't think companies should be paying for specific training because as a developer you can learn anything you need online.
On the other hand, my experience is that many/most companies are willing to hire a senior developer even if they don't have experience with the company's specific stack. They rightly assume that any developer can pick up a new stack quickly enough.
Personally I don't think companies should be paying for specific training because as a developer you can learn anything you need online.
On the other hand, my experience is that many/most companies are willing to hire a senior developer even if they don't have experience with the company's specific stack. They rightly assume that any developer can pick up a new stack quickly enough.
My experience has been that most companies aren't even willing to talk to me if I don't have experience in their specific stack. That's probably the downside of liking embedded/systems level development much more than webdev.
I think the personalty trait we call "grit" or "stick-to-it-ive-ness" is hard to test for. And I think the best way to test for it is by looking at someone's track record. And it's one of the big three requirements. (which are: smart, works hard, someone you want to work with. )
Don't test then. Have somebody with real life experience judge the candidate. And by real life experience I don't mean 10 years in the same job...
Anyway the interview process is so broken that they could actually improve it by just flipping a coin.
Anyway the interview process is so broken that they could actually improve it by just flipping a coin.
If I am a business owner, and I need someone to create a website for me to do something, the first thing I will reach for is someone who has already done similar projects and has a track record of getting it done. Teaching people is expensive, takes time (my website could be making me money -right now-) and doesn't always pay off. Whenever possible, it's often preferred to grab someone who can just hit the ground running.
I'm not saying this is perfect, just that's why experience is sought after. We have an annoying problem of way too many job ads being for seniors and very few for entry level. People have to get their foot in the door to ever become senior.
I'm not saying this is perfect, just that's why experience is sought after. We have an annoying problem of way too many job ads being for seniors and very few for entry level. People have to get their foot in the door to ever become senior.
> If I am a business owner, and I need someone to create a website for me to do something, the first thing I will reach for is someone who has already done similar projects and has a track record of getting it done.
If that is the case, what you need is a contractor, not an employee, and you should hire accordingly.
If that is the case, what you need is a contractor, not an employee, and you should hire accordingly.
I get what you're saying, but in this case the end result is still that the overall field is has high demands for people who already know how to do what they need done.
I disagree. Aside from an entry level position, you want someone with previous experience.
Google wouldn't hire someone to manage their datacenter with no previous experience managing anything like it.
Google wouldn't hire someone to manage their datacenter with no previous experience managing anything like it.
Voted down for being about as wrong as possible about Google's hiring practices.
> Google wouldn't hire someone to manage their datacenter with no previous experience managing anything like it.
That's exactly what they do. The interviews test for basic skills — an understanding of TCP/IP, unix, programming ability and intelligence — and all the rest is on-the-job training.
That's exactly what they do. The interviews test for basic skills — an understanding of TCP/IP, unix, programming ability and intelligence — and all the rest is on-the-job training.
Your challenge there would be finding anything remotely like Google's datacenter.
Trust me, if you have a passionate non-web developer he can do the job just fine. And he/she applied for the job so they are interested to learn something new, maybe already started doing so but there is nothing to put down in a CV. They will read the tutorials, the documentation and they will deliver. I feel that nobody wants to give a chance to anybody.
The business owner's response is that the experienced person won't take time reading tutorials or asking questions and thus get it done faster.
FWIW I agree most companies should train.
FWIW I agree most companies should train.
I totally agree that they -can- do the job. The most important thing for a software developer is the ability to learn. We need to give people chances so that they even even enter the field to one day become senior level.
But I also understand how a lot of business owners feel about it. It takes time to learn these things vs someone who has basically already done this project before in this stack before. A business in a hurry (most of them) would rather have someone who already knows how to do it.
Look at Stack Overflow Careers. Look how many more ads there are asking for senior devs.
But I also understand how a lot of business owners feel about it. It takes time to learn these things vs someone who has basically already done this project before in this stack before. A business in a hurry (most of them) would rather have someone who already knows how to do it.
Look at Stack Overflow Careers. Look how many more ads there are asking for senior devs.
I think what you're getting at taps into the heart of the issue- it's not just that the software industry is fast-paced in terms of innovation, it's fast-paced in terms of business requirements, in getting products out of market to scramble for users, which translates to higher valuation, and more VC dumb money. So in SV we kinda have a HFT-type situation where companies have to continuously get more and more speedy and optimize greater, even if it comes at a human cost- i.e., everyone's hiring only for senior engineers and there's not enough openings for junior developers, much less the resources to train juniors.
As an aside, it's paradoxical that the industry simultaneously desires 10x seniors, while prizes youth (who they don't have to pay at senior salaries). So the true profile of a unicorn coder is basically a 10x dev with 10+ years of experience who is also a new grad that requires no training. Keeps down the cost, all around.
As an aside, it's paradoxical that the industry simultaneously desires 10x seniors, while prizes youth (who they don't have to pay at senior salaries). So the true profile of a unicorn coder is basically a 10x dev with 10+ years of experience who is also a new grad that requires no training. Keeps down the cost, all around.
Right. It's that dream candidate who started programming at 9 and has no other hobbies besides coding.
Even jobs that don't really need developers of that caliber will hire like they do. I need someone to yet another billing app? Let me just hire one rockstar genius and squeeze the life out of them instead of two regular developers because it's cheaper.
Even jobs that don't really need developers of that caliber will hire like they do. I need someone to yet another billing app? Let me just hire one rockstar genius and squeeze the life out of them instead of two regular developers because it's cheaper.
> My point is you need smart enough people for your job and nothing more. You do an IQ test, a personality one and that's all. The rest can be learned.
Company culture matters. Even the most intelligent people that are willing to grow will fizzle if they are in an environment that isn't productive.
Company culture matters. Even the most intelligent people that are willing to grow will fizzle if they are in an environment that isn't productive.
There's a line there somewhere. Hiring someone for knowing Node is pretty silly, but hiring someone who's spent 5 years developing web apps is a useful metric.
In my experience, not that much. The 5 years experience might mean one year repeated 5 times. The person might be bored but he has no choice but to continue with web development and so on. Get somebody who is interested and smart enough for the job. Note I didn't say "the smartest".
5 years of 1 year experience repeated 5 times is better than someone that will introduce spaghetti code and security flaws simply because they've never experienced it.
It takes years to figure out patterns (when to use and not use them), good coding practices, avoiding newbie mistakes like sql injection and xss and how to actually make a deadline (its not easy to finish a project when you also have to learn the language and other skills in addition to the task at hand).
As a business owner, I don't want someone learning the above on the job and on my dime.
There is a company in MI that tried this called Compuware back in the 90s. It was a complete and utter disaster.
It takes years to figure out patterns (when to use and not use them), good coding practices, avoiding newbie mistakes like sql injection and xss and how to actually make a deadline (its not easy to finish a project when you also have to learn the language and other skills in addition to the task at hand).
As a business owner, I don't want someone learning the above on the job and on my dime.
There is a company in MI that tried this called Compuware back in the 90s. It was a complete and utter disaster.
[deleted]
Once you have an engineering job, a typical project tasking involves: some team design discussion, some engineering research, some work, some review, more work. The requirements are either known upfront, or determined as a team.
I do not understand why interviews are not conducted in this manner. Pick some arbitrary idea and have the candidate work with the interviewer to design something, then let the candidate do a little implementation research/design, review with the next interviewer, wipeboard code and review some part of the project with next reviewer. Have a coffee break at some point to get to know the person. Treat the interview process like a normal day at work.
I do not understand why interviews are not conducted in this manner. Pick some arbitrary idea and have the candidate work with the interviewer to design something, then let the candidate do a little implementation research/design, review with the next interviewer, wipeboard code and review some part of the project with next reviewer. Have a coffee break at some point to get to know the person. Treat the interview process like a normal day at work.
That's a good idea but logistically is hard, both requiring the interviewer to spend considerable time on site as well as costing your own employee's time.
It might work well for a small company especially with a small number of candidates but it doesn't really scale, unless you are a really big company (googleish size) that can dedicate so many resources to the interviewing process.
Doing design and research takes time, you can't do it in 30 min, you can't also ask someone to spend 8 hours interviewing, and if you have 10 candidates you can't spend half a man month or more on interviewing them either.
I've seen some companies that "interview" by contracting candidates for 1-3 days of work and paying them for it, but it really only works on positions where you hiring effectively interns/fresh graduates, but as most candidates especially those who end up in the face to face phase are currently employed you can't really do that.
It might work well for a small company especially with a small number of candidates but it doesn't really scale, unless you are a really big company (googleish size) that can dedicate so many resources to the interviewing process.
Doing design and research takes time, you can't do it in 30 min, you can't also ask someone to spend 8 hours interviewing, and if you have 10 candidates you can't spend half a man month or more on interviewing them either.
I've seen some companies that "interview" by contracting candidates for 1-3 days of work and paying them for it, but it really only works on positions where you hiring effectively interns/fresh graduates, but as most candidates especially those who end up in the face to face phase are currently employed you can't really do that.
I dunno, I've personally spent on average half a day (3-4 hours) with on-site interviews as a candidate. When on the other side of the table, it's been about the same (~3 hours). With a tractable enough problem, I think you can fit it into a half-day. You could also begin the design/research process as part of a phone screen, have the candidate further prepare on their own time, and then present/discuss as part of the on-site, along with the more in-depth technical stuff.
After you've run this process enough times, your interviewers will have a good grasp on the various solutions and technical problems. Most people's training on how to interview comes from how they themselves were interviewed - which was probably poorly done. Improve the interview process and you can train people to be better interviewers just by being interviewed. :)
After you've run this process enough times, your interviewers will have a good grasp on the various solutions and technical problems. Most people's training on how to interview comes from how they themselves were interviewed - which was probably poorly done. Improve the interview process and you can train people to be better interviewers just by being interviewed. :)
"I do not understand why interviews are not conducted in this manner. "
It truly is crazy.
It truly is crazy.
After having interviewed dozens of people, the idea occurred to me that my ratings for interviewees were a Gaussian curve with a normal distribution. Most people blended into each other - they knew the topics the same amount. Enough to be employed in their current job. Then one in six was a standard deviation below or worse, and one in six was a standard deviation above or better.
That might be one reason for the impostor syndrome. Because if we interviewed six people, one would be unqualified, four would be OK but not spectacular, and one would know the subjects better than anyone. So in an interview of six people the best qualified one of the six would have the job. The average four were not truly impostors, but would not get the job unless they had an inside recommendation or if we were really needed someone and there was a shortage of candidates coming in.
That might be one reason for the impostor syndrome. Because if we interviewed six people, one would be unqualified, four would be OK but not spectacular, and one would know the subjects better than anyone. So in an interview of six people the best qualified one of the six would have the job. The average four were not truly impostors, but would not get the job unless they had an inside recommendation or if we were really needed someone and there was a shortage of candidates coming in.
I understand companies are unwilling to give negative feedback to applicants for fear of legal repercussion. But often not getting any form of feedback is highly detrimental to an applicant who's invested days if not weeks of going through the process, only to receive a boilerplate "thank you for applying, please try again" response.
One wonders if it's possible to devise some sort of neutral, yet helpful, assessment that could be given to applicants without fear of legal retaliation. Or, if eventually an applicant will sue a company for not providing feedback after they were rejected.
One wonders if it's possible to devise some sort of neutral, yet helpful, assessment that could be given to applicants without fear of legal retaliation. Or, if eventually an applicant will sue a company for not providing feedback after they were rejected.
In related news, "People are still bad at knowing the unknown" and "Everybody thinks they're average; perhaps because their sample size can only be 1."
"Everybody thinks they're average; perhaps because their sample size can only be 1."
It's my understanding that this is not the case.
People typically think they're above average.
https://en.wikipedia.org/wiki/Illusory_superiority http://www.livescience.com/26914-why-we-are-all-above-averag...
It's my understanding that this is not the case.
People typically think they're above average.
https://en.wikipedia.org/wiki/Illusory_superiority http://www.livescience.com/26914-why-we-are-all-above-averag...
I've never interviewed for a job I didn't get. Possibly because I aim too low. But if I were to present myself honestly as a potential employee, I would enter the interview without greeting or making eye contact, then I would sit as far away from the interviewer as possible and doodle in a notebook for 15 minutes before raising my head and explaining everything I think is wrong about the company.
Mmmhhh I would have drawn the opposite conclusion, most people estimate ±1 star away from the real result, it's not bad considering the actual performance is not entirely objective.
Glad to see that once again Dunning-Krüger proved wrong.
Glad to see that once again Dunning-Krüger proved wrong.
> not bad considering the actual performance is not entirely objective
It's not even clear (to me) what's being compared here. What's the "actual performance" that the blog post very carefully does not define? The screenshot highlighting the technical performance question seems to suggest that it might be only that one point out of the three that interviewers fill in.
But it's completely possible to be technically competent and still feel that one didn't do well overall because one came across as nervous or boring or whatever. In other words, the comparison here seems to be between apples and undefined oranges.
It's not even clear (to me) what's being compared here. What's the "actual performance" that the blog post very carefully does not define? The screenshot highlighting the technical performance question seems to suggest that it might be only that one point out of the three that interviewers fill in.
But it's completely possible to be technically competent and still feel that one didn't do well overall because one came across as nervous or boring or whatever. In other words, the comparison here seems to be between apples and undefined oranges.
Surely a lot depends timing and circumstances. One interviewer said they gave me the job based on asking if the guys accent was Dutch (he was Norwegian and found that hilarious). It was a maintenance programming job and had a high turnover of staff. Other interviews you may have done well but had some serious competition.
I used to do a lot of interviews for front-end engineers. The process I settled on was to give them the following setup:
- some mock-ups from when we first designed our product gallery
- a json file that listed the products names, pictures, description, etc.
- a blank css file
- a html file that loaded the css and jQuery, and then made an ajax request to load the json file. (This was back when jQuery was pretty much state of the art for front-end.)
With all of that in place, I'd explain that I'd like them to work on building a product gallery that looked similar to the mockups. I would further explain that, since we only had an hour, I didn't expect a polished or even fully functional product, but I'd just like them to dive in and see how far they got. They could use their editor of choice and ask any questions they wanted.
I felt like this gave better insight to candidates skill and workflow. My colleagues all gave more traditional interviews, so I also felt like there was a good counterbalance if someone didn't do well with this kind of interview.
- some mock-ups from when we first designed our product gallery
- a json file that listed the products names, pictures, description, etc.
- a blank css file
- a html file that loaded the css and jQuery, and then made an ajax request to load the json file. (This was back when jQuery was pretty much state of the art for front-end.)
With all of that in place, I'd explain that I'd like them to work on building a product gallery that looked similar to the mockups. I would further explain that, since we only had an hour, I didn't expect a polished or even fully functional product, but I'd just like them to dive in and see how far they got. They could use their editor of choice and ask any questions they wanted.
I felt like this gave better insight to candidates skill and workflow. My colleagues all gave more traditional interviews, so I also felt like there was a good counterbalance if someone didn't do well with this kind of interview.
Doesn't that heavily bias towards candidates who have used jQuery before and/or built a product gallery before?
Skewing interview performance based on some specific previous experience might be what you want if you expect your candidates to hit the ground running, but might not be the best to gauge long-term performance.
Skewing interview performance based on some specific previous experience might be what you want if you expect your candidates to hit the ground running, but might not be the best to gauge long-term performance.
This was back in the day when everyone used jQuery for everything, and the company was only looking for experienced people, so a candidate who hadn't used jQuery before probably wouldn't have been a good match for the position anyways. I don't think a single candidate had that issue though.
As far as the product gallery goes, I wasn't looking for a pixel-perfect product. Just spit out a bit of html and matching CSS that look vaguely like the mockup. The mockup images had things like the hex values of the colors written on the image to make that part easy.
Basically, it was trying to focus on things that would be the main part of the candidate's job there. The folks that we did hire all did well on both that interview and long-term at the company.
As far as the product gallery goes, I wasn't looking for a pixel-perfect product. Just spit out a bit of html and matching CSS that look vaguely like the mockup. The mockup images had things like the hex values of the colors written on the image to make that part easy.
Basically, it was trying to focus on things that would be the main part of the candidate's job there. The folks that we did hire all did well on both that interview and long-term at the company.
That there still needs to be discussions about that topic. On a workplace people should be treated equally.
No matter how many topics will be brought up.
A world with different genders, stereotypes, whatever can only work if both sides won't interpret too much into it and just enjoy their work with their colleagues.
Actually we don't employ a woman (I'm working at a 3 person company). And we actually hire, but the only woman will probably decline on their side, she actually has experience in things we actually not need yet, so I guess we can't provide her a new challenge. way probably the best application.
Actually we don't employ a woman (I'm working at a 3 person company). And we actually hire, but the only woman will probably decline on their side, she actually has experience in things we actually not need yet, so I guess we can't provide her a new challenge. way probably the best application.
Don't miss the footnote: "I’m always terrified of misspelling “Dunning-Kruger” and not double-checking it because of overconfidence in my own spelling abilities."
What a hatchet job.
Unfortunately we don't have the raw data available; but all we can infer from these heatmaps is that yes (and exactly as one would expect) there's a bit of variance between between the two sets of ratings (in both directions). Maybe a bit more than can be explained by the sample size, or from the simple fact people (on both sides of the process) are forced to discretize their assessments (so even if both parties were to objectively agree that the "real" performance was in the 2.4-2.7 range, which it is a good chunk of the time.... inevitably you'll find them be off by +/- 1 in their quantile assignments a good chunk of the time -- which is exactly what the data seem to show).
In particular: independent of how we explain the variance (whether due to sampling effects, or bona fide DK/IS) note that the vast majority of the variance is in the +/- 1 range. And on the alleged DK side (-2 variance) the measure is apparently quite miniscule.
However, that's not the original D-K effect. What D-K specifically predicted (and were able to show with much better data and methodology -- in their very specific sample population, at least) is that, among the general population there will be a striking degree of -2 variance, specifically in the bottom quartile. Going by the oft-quoted abstract from their original paper:
Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd.
In other words: "Performers in the bottom quartile estimated themselves to be performing, on average, at mean performance in the third quartile."
So if were to look for something analogous to a D-K effect, we'd expect to find a "hot spot" (or at least the strongest amount of red ink, for that quartile) in the upper left of the first chart.
Instead we find most performers in the bottom quartile (correctly) identifying themselves as in the bottom quartile; with a smaller portion bleeding up into the second quarter; and an an almost negligible portion placing themselves in the D-K zone -- namely, the third quarter.
If anything what the data show is that (even assuming there's nothing to contend with in how the actual data were taken), while inevitably there's some variance (both ways) in the two sets of rankings, and probably some of due to distorted self-perception -- on the whole there's actually much less of a D-K effect among engineers than among the general population (or rather, "Cornell undergraduates taking psychology classes" -- which, lest we forget, were the actual subject class used in the original D-K study).
So basically the opposite of what the author of the original article is claiming to infer from this data.
Unfortunately we don't have the raw data available; but all we can infer from these heatmaps is that yes (and exactly as one would expect) there's a bit of variance between between the two sets of ratings (in both directions). Maybe a bit more than can be explained by the sample size, or from the simple fact people (on both sides of the process) are forced to discretize their assessments (so even if both parties were to objectively agree that the "real" performance was in the 2.4-2.7 range, which it is a good chunk of the time.... inevitably you'll find them be off by +/- 1 in their quantile assignments a good chunk of the time -- which is exactly what the data seem to show).
In particular: independent of how we explain the variance (whether due to sampling effects, or bona fide DK/IS) note that the vast majority of the variance is in the +/- 1 range. And on the alleged DK side (-2 variance) the measure is apparently quite miniscule.
However, that's not the original D-K effect. What D-K specifically predicted (and were able to show with much better data and methodology -- in their very specific sample population, at least) is that, among the general population there will be a striking degree of -2 variance, specifically in the bottom quartile. Going by the oft-quoted abstract from their original paper:
Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd.
In other words: "Performers in the bottom quartile estimated themselves to be performing, on average, at mean performance in the third quartile."
So if were to look for something analogous to a D-K effect, we'd expect to find a "hot spot" (or at least the strongest amount of red ink, for that quartile) in the upper left of the first chart.
Instead we find most performers in the bottom quartile (correctly) identifying themselves as in the bottom quartile; with a smaller portion bleeding up into the second quarter; and an an almost negligible portion placing themselves in the D-K zone -- namely, the third quarter.
If anything what the data show is that (even assuming there's nothing to contend with in how the actual data were taken), while inevitably there's some variance (both ways) in the two sets of rankings, and probably some of due to distorted self-perception -- on the whole there's actually much less of a D-K effect among engineers than among the general population (or rather, "Cornell undergraduates taking psychology classes" -- which, lest we forget, were the actual subject class used in the original D-K study).
So basically the opposite of what the author of the original article is claiming to infer from this data.
Probably because interviewing companies have stopped giving performance feedback.
This is the key takeaway for me. This is IMO and experience probably based on a "sour grapes" type defense mechanism, where candidates will internally try to talk themselves out of wanting a job that they felt they didn't get. They walk out of the interview feeling they didn't perform, and instead of regretting the lost opportunity they start to focus on even the most minute "negatives" about the job/company.
The more time you give a candidate to stew about those reasons they don't want the job, the more time they'll have to find reasons that may not even exist.
Positive feedback should be given almost instantly. Some companies and candidates are reluctant to give positive feedback quickly because they feel it may hurt their negotiation leverage.
I don't have data on this, but anecdotally (over almost 20 years in recruiting) I've seen this countless times while collecting post-interview feedback from candidates. The ones that feel they did poorly are likely to mention a negative about the interviewer or company, which becomes somewhat awkward when we come back with a job offer.