Some Reflections on Being Turned Down for a Lot of Data Science Jobs(tdhopper.com)

288 pointsby tdhopperhace 9 años165 comments

Some Reflections on Being Turned Down for a Lot of Data Science Jobs

http://tdhopper.com/blog/2017/Mar/06/some-reflections-on-being-turned-down-for-a-lot-of-data-science-jobs/

174 comments

I manage a data science team and revamped the hiring process pretty substantially about a year ago, to good results. Nothing in here is particularly original, but here's what we do:

1. Break down "data science" into several different roles–in our case, Analyst (business-oriented), Scientist (stats-heavy), Engineer (software-heavy). Turns out that what we mostly want are Engineers-Analysts, so our process screens heavily for those.

2. Figure out which types of people can be trained to be good at those roles, given the team's current skillset. I opted to look primarily for people with strong analysis skills and some engineering.

3. Design interview tasks/questions that screen for those abilities. In my case, the main thing I did was make sure that the interviews depended very little on pre-existing knowledge, and a lot on resourcefulness/creativity/etc. E.g. the (2-hour) takehome is explicitly designed to be heavily googleable.

4. Develop phone screens that are very good at filtering people quickly, so that we don't waste candidates' time. By the time someone gets to an onsite interview on our team there's something like a 50% chance they'll get an offer.

On the candidate side, when I'm applying I try to figure out first and foremost what a company means by "data scientist", usually by networking & talking to someone who already works there. This filters out maybe 90% of jobs with that title, and then I put more serious effort into the rest.

Declanomoushace 9 años

I've been looking for a job, and I've found how vaguely organizations define their data scientist and analyst roles in their job postings really frustrating. They tend to have a short description of the role, which is generally filled with buzzwords, followed by a list of requirements. I wish organizations would talk about what they wanted to do with their data instead.

For instance, a common description might say the candidate will be working with "big data" to help with "data-driven initiatives" and the requirements will be something like "knowledge of Excel, with a Masters in Statistics, or equivalent experience".

It's really hard to tailor a cover-letter or a resume to a job posting like that. For one thing, I can't even imagine what kind of work they are doing if they are using Excel for "big data". Second of all, I currently have a job, and writing cover letters and creating resumes takes a lot of time. By the time I get to the phone screen I've probably already spent at least a couple of hours applying. Plus, in the interest of keeping my cover letter and resume short, I have to leave off a fair amount of my experience and performance metrics.

Honestly, at this point I think I'm just going to start reaching out to people in the fields I'm interested in and asking them if they know of any roles that would fit my skill-set. The way I see it, I'd at least have a chance of getting feedback from someone who can view my skill-set holistically, rather than HR, who will let me know that I don't tick all their boxes (or vice versa).

apohnhace 9 años

>>I've been looking for a job, and I've found how vaguely organizations define their data scientist and analyst roles in their job postings really frustrating.

I lead a Data Science team and part of the struggle with writing sensible job descriptions is that there are too many people providing input into the job description. HR can also put their hand in the pot when they try to use buzzwords (e.g. Hadooop) to internally justify why a role with 2 years of experience needs to be paid like other roles (e.g. traditional Excel based Analyst) with 5-10 years of experience.

>>They tend to have a short description of the role, which is generally filled with buzzwords, followed by a list of requirements. I wish organizations would talk about what they wanted to do with their data instead.

One major challenge for Data Scientists is how hyped the role is, leading to people in an organization believing whatever they want about Data Scientists. Are you a leader who wants a business analyst who can use software and interface with IT? Data Scientist. Are you an engineering manager who wants a person who can interface with the business and use machine learning? Data Scientist. Are you a VP who thinks big data and ML is the problem to your bad or non-existent data? Data Scientist. Do you want somebody who can exhale the maximum amount of hot air while still sounding like a tech and math genius? Data Scientist.

Also add in that business people with minimal experience in modern Analytics are trying to build up Data Science and Analytics capabilities in their own part of the organization because they realize Excel is not the answer to every question. I've spent a lot of time speaking with people to help them understand the type of people they need to hire. Sometimes people are sensible and sensible job descriptions and expectations come from that. Other times they are adamant about what they need (even if they are wrong) and the end result are convoluted job descriptions that are either never filled or filled with the wrong person.

autokadhace 9 años

how do you choose candidates for an interview?

I can't even get a call back for an interview lined up. I have done NLP, got a masters degree in computer science from Penn, plenty of experience with big data such as hdfs and hive, spend my free time doing what ever data science I can. but obviously doing something wrong.

any suggestions? Here is my linkedin account: https://www.linkedin.com/in/karl-dailey-02557b65

shepardrtchace 9 años

May I offer a couple of quick suggestions? I've never hired for Data Scientist positions, but I've hired for plenty of other ones.

1) Change your profile picture to something serious. Get a collared shirt and a nice background outdoors. No tie. People will unconsciously judge you on your picture, so you want something that shows you're a professional, but you're confident and happy in life.

2) Think hard about your job titles. Your latest job is far more than just a "Data Analyst". It seems closer to a Scientist or Engineer role, even if your company doesn't call it that. "Analyst" makes people think of entry-level positions. Your DBA-Programmer position is more like a Software Engineer/DBA/System Admin position. If you can't pick one, generally those all-in-one positions can be known as Systems Engineers. Whatever you do, make sure that DBA isn't the primary thing people see. In general, sell your previous positions more. Like "IT". That's not a title, that's a department.

3) Expand on your projects section more.

4) Make sure your resume matches your LinkedIn. People absolutely look you up on there. I did it all the time. When the two didn't match, I was suspicious.

Good luck.

apohnhace 9 años

I've got some feedback and you might see it as mean and brutal. But what I'm doing is being honest as how I would evaluate a resume that looked like your LinkedIn profile. I'm just telling you what I'm thinking.

The first thing I did was look at your current position and I immediately became skeptical. You have been at your position for 10 months, yet you have done a lot of fancy sounding things that seem to have no connection to each other. To me this is a huge red flag that you're just grabbing some data, churning through some code you found on StackOverflow or in a book/documentation, and then making grand claims about the work you are doing at ComCast.

Your use of buzzwords only makes me think this more.

For example "Churn forecasting: probabilistic modeling of deletion rates on dVR with a beta binomial distribution to forecast the number of devices that carried a show over 300 days. Maximum Likelihood Estimation was used to derive distribution parameters."

This honestly looks like you saw a tutorial about an R-library and then you copy/pasted the documentation for one of the functions into your profile. By using so many buzzwords (including the term deletion rates), you've left me wondering if this is something you actually did or just something you made up. If you actually defined and derived the Likelihood function you should state that. Otherwise saying you used MLE means nothing since many functions in R use MLE under the hood.

Another one is "Built node.js/angularjs integrated tool to allow analysis team to test the sensitivity of KNIME workflows for forecasting." In my mind I'm thinking, what exactly does web programming have to do with KNIME? What exactly do you mean by sensitivity? How are KNIME workflows sensitive? Do you mean you are checking the accuracy of forecasting models built in KNIME? Lots of Data Scientists will have no idea what node.js and angular are and many will not care when they find out. Data Scientists may build charts using JS, but very few will be building Web based UIs themselves (I assume this is what you are trying to say?)

To be honest, I have no idea what you do in your job. Are you actually a Data Scientist as part of your job duties or did you develop an interest in Data Science and you have access to Comcast data so you've been playing around on your own?

At a hiring manager, I want to know the person I'm evaluating has spend time working on a business problem from start to finish. This means they thought about the problem, defined how to answer it (or figured out how the business owners want them to define it), figured out which data was relevant, thought through a model if it's a business problem that can be addressed by a statistical or ML method - this includes thinking through how the output of model will be used by the larger business - and making a lot of mistakes and changes in this journey.

If the person is just getting into Data Science, I want to know the person has the analytical and metacognition skills to think through a business problem. If they have that then I evaluate their thinking with models.

I'd recommend you trim down your LinkedIn profile and focus on the projects which took you some time (e.g. a month or longer) and where you had to iterate a solution. Use these projects to illustrate what you actually do at your job.

It's possible that you've done everything you state. You may be working with a team of people. At a large company you'll have other teams supporting you with data collection, cleaning, and putting the data into a shape that makes it amenable to machine learning and joining with other relevant sources. If that's the case, focus on projects where you were leading the project in some way and emphasize how you worked with and led the team.

But based on your LinkedIn profile I'm skeptical and a resume like this would not pass my filter. The HR person I work with would see your profile as impenetrable. If they can't figure out what you do and what you have done they are not going to send your resume to me.

Finally, can you move into a role at Comcast that gives you an official "Data Scientist" title? From what I've heard Comcast has a solid Data Science practice and it seems like they have some really interesting data. Finding that combination is really hard. Many good or potential Data Scientists end up at jobs where they are unhappy because the Data Science culture and/or data is awful.

joshvmhace 9 años

> I wish organizations would talk about what they wanted to do with their data instead.

It may seem like a daft question, but have you tried asking them? They may clam up and refuse to give you anything, but I would imagine most small companies would be willing to talk about what the role involves. It's a bit late at the end of the interview to be asking "So what would I be doing?".

You may find out that they're working on some data which is e.g. image-heavy and you happen to be an image processing expert.

lukejduncanhace 9 años

This is a great breakdown. Data Science can mean a lot of different things and I think the three roles you defined are spot on. At LinkedIn, what used to be called Data Science is now exclusively analyst roles. We have a separate org (hundreds of people) called Relevance which is part of the engineering org and is composed of scientists and engineers. Everyone in that org falls somewhere on the scientist to engineer spectrum with ML / Stats backgrounds and a bias towards ML engineers. Most of the people who interview would consider the relevance roles "Data Science" but the term within the company is still tightly coupled with pure analyst functions.

sheeshkebabhace 9 años

I wish all interviewing was like that above - typically it's - create some incoherent job description, run through a bunch of semi-random candidates, pick a random guy that fits the budget, rinse/repeat...

Candidates know that, typical in person interview has a 1/10 chance (or worse) of getting an offer from an inperson interview, and so most candidates, including good ones, would care less to spend much time on interviewing with any one employer, or do much to increase their unknown chances.

As a rusult employers don't get right candidates and then wine about lack of talent and pay for crappy tools that just sort resumes half assed way... good grief.

nickdavidhayneshace 9 años

Could you talk more about your phone screens? What criteria are you filtering on? What sorts of questions have you found to be effective at stratifying candidates?

SatvikBerihace 9 años

I'm mostly looking for people who want to learn about engineering with largeish datasets (~100gb/day for us), and have some of the prerequisite skills. Our codebase is mostly in Spark/Scala and uses functional programming idioms, so I'm looking for people who either know or want to learn how to use those. I'm also specifically trying to filter out people who mostly want a stats-heavy, machine learning heavy job, since that's not what we do.

An engineer who wants to learn data science is a great fit for us, an academic who wants to write R all day is not (though an academic who wants to learn engineering/functional programming is fine!)

Beyond that, I ask some questions about projects they've worked on, and in particular, how their approach would change if assumptions were different. Here I'm looking for the ability to reason backwards from a business goal, as opposed to somewhat blindly applying statistical techniques.

If they do well on these, we send the take-home exam. As previously noted, this is specifically designed to require relatively little knowledge but heavily test analysis skills, and lightly test programming skills. It's almost impossible to complete this exam without using Google effectively, so that's another thing I'm testing.

walshemjhace 9 años

can I ask why functional programming in particular I can see why you might want to avoid java for big data - but isn't the average ML algo more in the procedural mould?

Would not python with numpy be a better fit ? or fortran with some handwave interface code

Back (early 80;s) when I did map reduce we used PL1/G

SatvikBerihace 9 años

The most direct reason is because the current team enjoys functional programming.

From a business standpoint though, there are a few main reasons:

–Data pipelines are well modeled as functions: they take a few input datasets, return a few outputs at the end, and do a ton of processing in between

–FP idioms generally make parallelization easier, and this is very important for the datasets we're dealing with

–A strong type system like Scala's lets us prevent many runtime errors, which is quite important when your pipelines can take several hours

–It's fairly trivial to wrap a statistical/ML algorithm in a pure functional interface, even if the algorithm itself is imperative

micro_camhace 9 años

Have you had performance issues getting things to conform to functional paradigms?

For example i've found that as a pipeline gets optimized for production use it needs to preallocate all of its output space and then modify things in at each step (like a one hot encoder flipping a few bits in specific rows of a zeroed array instead of allocating new ones and copying them in).

I find it difficult to reconcile this sort of code with a "pure functions without side effects" philosophy and still have it perform an an acceptable level.

SatvikBerihace 9 años

We're mostly doing ETL on large datasets, so the code needs to parallelize well, but beyond that performance isn't really a big concern. We use ML in research, but no models in production, because the costs of increased maintenance/lost transparency generally outweigh the benefits in our use case.

In jobs that were heavy on ML, I would use high-performance tools for the models (imperative code, numeric computing packages etc.) and functional code for the ETL, which worked pretty well–no need to be dogmatic about it, a 70% pure codebase is still generally easier to reason about than a 20% pure codebase.

walshemjhace 9 años

Interesting I will have to have a proper look at Scala when I get my baby cluster up and running.

cjhankshace 9 años

Functional programming for a lot of numerical computing maps easier to mathematical notation. However, Scala is usually a worse choice than Java for numerical computing since everything is a boxed type.

kodhace 9 años

This is straight up false, why do you think Scala doesn't have primitive values? Long will be either a value or reference type as needed, despite being spelled only one way instead of two different ways in java.

cjhankshace 9 años

A 'primitive type' is one which can be directly operated on by intrinsic CPU instructions. My understanding of Scala was that all objects (such as Long, Int...) are encapsulated inside of an object.

Therefore an array of boxed types will not be memory aligned; and any vector instructions (which are very important to scientific computing) cannot be used.

Perhaps something has changed in Scala land since I last looked(??).

premium-concernhace 9 años

No, it has been worked the way GP wrote since approximately forever.

lifeonahilltophace 9 años

Can I ask what are the rough percentages of the three types of data science jobs in the industry? And is there an easy way to search for only one type of jobs (such as using some particular keywords on indeed.com)? I have some advantage only in the scientist/stats-heavy type of data science jobs and would like to better focus my job-searching effort. Thanks.

kafkaesqhace 9 años

So you're at least occasionally getting actual feedback from some of these discussions. That's quite something, actually.

BTW, as to some of that "feedback":

Honestly, I think the way you communicated your thought process and results was confusing for some people in the room.

"Okay, pal - let's put your people up in a room full of strangers (some of whom show through their body language and/or constant phone-checking that they pretty much don't really want to be there in the first place), make them answer some made-up questions (which may or may not be coherently presented; or even particularly relevant to the job description) -- combined with the background stress of being unemployed, and/or stuck in job they absolutely HATE, and can barely stand another day of -- and see how they do."

Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you.

Great -- so they're basically accusing you of being a slacker (not really into the work, only interested in what's in it for you, etc). Which is quite a presumptuous attitude to take in response to a perfectly reasonable question about the value proposition they're asking you to consider (a question for which it might be in better form to wait until the later stags of the negotiation process to bring up... but that's a very minor style point that you definitely shouldn't be dinged for).

"Quite honestly" that attitude sucks. And you don't need to feel bad about being "rejected" by people like that.

FeatureRushhace 9 años

Without context the second quote sounded to me not like an accusation of slacking but like a warning that the company is bad place to work at (poor life-work balance)...

mulmenhace 9 años

I recently asked about the "unlimited vacation" policy and received a vague answer about "establishing expectations with your manager" and then was told they went with someone else.

I thought about if I should ask or not but I honestly don't understand how such a policy could be good for employees so I wanted an idea of the culture that allows it. If that question contributed to them not giving me an offer then good riddance so I don't regret asking.

kafkaesqhace 9 años

No need to keep asking. If your get is telling you that something's not quite with this "unlimited vacation" policy -- it's because it's a sham, basically.

And everyone at the company knows it -- including and most especially, upper management. Because that's after all where it originated.

kafkaesqhace 9 años

EDIT: 'get' should have been 'gut'.

jlaroccohace 9 años

Completely agreed. An interaction like that during or after an interview would be a big red flag that it's not a place I want to work.

namelezzhace 9 años

> Honestly, I think the way you communicated your thought process and results was confusing for some people in the room.

Wow, get similar rejection lately: this is a fast paced company and the way you explained your thought process was confusing and might slow us down, so good luck with your job search.

daenneyhace 9 años

Did you ask them what was confusing and how it would slow them down. Did they tell you? Because ultimately if I'd hear that without anything to help me improve it's a bit of a catch-all "we don't know so out you go" kind of thing.

krirohace 9 años

This sounds like a pretty douchy sort of "yeah you're just not good enough, look how fast we are" jab by them.

Better not answer with "Thank you very much. I also had a feeling that some people in the room lacked the mental capacity to follow my explanation."

;)

beachbum8029hace 9 años

Reading this thread makes me want to cry. The world is filled with morons.

chronicx0hace 9 años

minimaxirhace 9 años

Many recommend setting up an online portfolio for these types of positions. But I've applied to a number of Data Analyst/Scientist jobs recently and I am immediately rejected almost every time despite highlighting my blog/portfolio (http://minimaxir.com) and my GitHub with open-source code/Notebooks for each and every post (https://github.com/minimaxir), both of which have topped HN on occasion.

Internal recruiters have hinted that my Software QA Engineer background + no CS degree implies I have no technical skill.

tom_bhace 9 años

I wonder if you are less impacted by the lack of CS degree than by your "Software QA Engineer" label.

My own experience was that my initial position as a software performance engineer resulted in a perception that I was a "tester" without technical skills despite having multiple CS credentials and published code in practitioner-oriented sources.

Overcoming recruiter biases was such a struggle that I now routinely counsel students and early career programmers to carefully consider whether job role perceptions would negatively affect their future prospects. I also tell them, when financially viable, to not take on roles where hiring orgs cannot give them a day-to-day job description that directly matches their desired career path.

This advice seems quite challenging or maybe misguided for data science careers though. It seems like just getting an entry-level data science job might require a dedicated MS in either CS or stats, with a healthy set of projects in whichever of those two subjects you didn't spend grad school working on to prove yourself . . .

bbarnhace 9 años

You're probably on to something there. Sadly, the opposite of this is true too - it's incredibly hard to find competent QA Engineers (not testers) because they all realize very quickly they can make more money as rank and file software devs than QA-side specialists, even if their role in QA is leagues more complex.

yttriumhace 9 años

I had a similar experience. My first job out of college was for a Developer Role (building testing frameworks, maintaining and building browser extensions) but the job was titled 'QA Developer' so I had a hell of a time the first time I tried to find a new job. Never mind that I wrote thousands and thousands of lines of application code, lots of recruiters would deny me on the basis that my background didn't fit.

thomasthomashace 9 años

Some companies are really dumb and only consider candidates with specific job titles on their resume. I've had recruiters from contract companies say "your experience is great but can we change your previous job title to X." Recruiter knows the client gets hung on job titles if its not exactly what they're looking for. In my experience i'll change previous job titles to fit the position i'm applying for, as most of my previous jobs I could've been titled 4-5 things.

CryoLogichace 9 años

I've seen this happen. Most companies cannot hire competent recruiters. Often recruiters have no technical background, and are unfamiliar with all but the buzzwords.

This trend will probably continue until someone decides to up recruiter pay and hire engineering background candidates for recruiting roles (if possible).

santaclaushace 9 años

Did you try dropping the exact title from your resume for a more general description of the role?

yttriumhace 9 años

Yes, but it was hard to entirely obfuscate the fact that I designed/coded testing frameworks. Part of my job was running and analyzing big batches of regression scripts, so it would have been disingenuous to pretend like it didn't happen.

user5994461hace 9 años

Replace by Software Engineer. Problem solved!

user5994461hace 9 años

The first job is the hardest to get. Keep trying, lower your expectations: Take anything you can and don't expect to be paid much.

Contrary to what the reddit folks would have you think. The ONLY thing that consistently get someone through the door is demonstrable real work experience, on real projects, in the industry (read: not academia). Side projects are not a substitute and the lack of degree is a barrier.

[deleted]hace 9 años

user5994461hace 9 años

https://www.linkedin.com/in/minimaxir/

It turns out that QA was only the tip of the iceberg. You have a job title as "QA" and a job title as "support".

You will NEVER get any development or engineering job with that. Expect 90% reject in the resume screening because you are not a developer.

If they were development jobs, replace both by "software developer".

minimaxirhace 9 años

Those were my college internships, as you can see by the dates. (and obviously I do not include them in my resume)

My most recent job is obfuscated a bit in the public eye/only visible to logged-in LinkedIn users, for good reason. (I politely request people looking into it not discuss it)

user5994461hace 9 años

> and obviously I do not include them in my resume

You remove the experiences you have and wonder why your resume doesn't pass a screening test? Must seem quite empty.

EternalDatahace 9 años

I've seen your posts on Spark. That's sad to hear.

Are you applying online or via getting coffees with people in the data science team and just talking shop? The latter seems to have a high conversion rate if you can nail it.

minimaxirhace 9 años

Mostly online; I've been too busy to network during the weekdays unfortunately, which is something I'm working to fix.

muninn_hace 9 años

Yeah. The problem is recruiters are almost universally non-technical, so they can't properly gauge your talent nor do they understand the job requirements fully.

cwilkeshace 9 años

I wouldn't limit it to recruiters ... some engineers see the word "QA" or "SDET" and think the guy isn't worthy of being a "real" software engineer, or if they do then it is at the lowest rung of the ladder. Really annoying.

In the OP's case I would change their title (if applicable) to "QA / engineer" as a) that probably paints a more accurate picture of what you were doing and b) gets around yokels that look down on QA/SDET.

NumberCruncherhace 9 años

Your portfolio sais "I can plot public data in colour". It should say "I understand and can apply in practice a couple multivariate modelling techniques". Learn until you understand why we don't blog about single decision trees.

dataframinghace 9 años

I have to disagree. The blog is very heavy on the data visualization -- but is that a bad thing? The posts effectively cover most of the data analysis timeline (collection, iterative exploration, model building, visualization) and they're well-explained and thoroughly-explored.

Others have said the same, but I'd wager the biggest issue is (a) the limited past roles that directly dealt with data science/analysis or (b) SF's narrow-minded hiring practices. Maybe a masters or data science bootcamp would go further than more blog posts?

minimaxirhace 9 años

Thanks for the optimistic descriptions of my posts. :)

kevinburkehace 9 años

> Internal recruiters have hinted that my Software QA Engineer background + no CS degree implies I have no technical skill.

They're really trying to mitigate risk that they pass you along to engineers and the engineers say "why am I looking at this person with no formal qualifications" and then the recruiter feels like they've wasted the time of the engineering team. It's stupid but insidious.

If you can I would suggest trying to bypass the recruiters and contact the engineers/hiring manager directly; ask them out for coffee and ask for advice on starting a career in data science, or show them your portfolio via email there.

dev1nhace 9 años

Send the recruiters your Reddit posts in the r/beautifuldata or w/e that subreddit is. I'm sure the recruiters spend a lot of time on Reddit. Recruiters are ridiculous.

edit: spelling

ploggingdevhace 9 años

I would suggest taking part in the Who wants to be hired monthly thread if you have not been doing that already.

I realize that most startups recruiting through HN are looking to hire for web, app or backend development, but there are a few data science jobs. Since people recognize you here on HN, you have a much better chance of landing a job.

It's unfortunate that most recruiters primarily screen based on resume.

I don't know if Triplebyte [1] helps people find data science jobs, but do check with them.

[1] https://triplebyte.com/

melvinmthace 9 años

> my Software QA Engineer background + no CS degree implies I have no technical skill

It's OK to leave stuff out on your resume if it's not relevant (and maybe even harmful) to the position you're applying for.

minimaxirhace 9 años

My Software QA Engineer position is my first, and only job post-undergrad.

cemahace 9 años

Maybe you could rephrase it. For example, give it a title "software engineer", and then a subtitle or one short paragraph explaining: "the position was officially called qa whatever and involved developing software for this and that", and then list the accomplishments.

amorphidhace 9 años

I recruited for one data science job. The hiring manager was super focused on getting someone with a PhD. Is that the norm for a position like that?

randcrawhace 9 años

Some companies know what skills they need, others don't. For those who don't know, but want a typical statistical modeler, a PhD in statistics is the safest choice, then a PhD in machine learning or MS in stats, after that, you need to look for the presence of specific skills... like deep learning for image recognition or the Hadoop stack.

megablasthace 9 años

> Software QA Engineer background + no CS degree implies I have no technical skill.

Sure, but that doesn't sound unreasonable.

minimaxirhace 9 años

>:|

chronicx0hace 9 años

_flbthace 9 años

I was coming from academia with no real competitive background in machine learning/predictive analytics/statistics when I started applying for data science positions. I was coming from a post doc in computational neuroscience, and had done ML/your run-of-the-academic-mill stats as part of my thesis, but never had formal training in it. I landed a lot of interviews (and applied for an ENORMOUS amount of positions), but it took me a while to land a really solid offer. So just keep at it.

Once I got that offer and 'data scientist' was listed as a position on my resume, I've had at least one or two recruiters reach out to me each week. Not just your 'bulk tech recruiter', but individual hiring managers/team leads from companies who had no interest in me prior to the title change.

Hell, I had a manager I had already interviewed with from another company (and got rejected) reach out to me TWICE to come back in. To be honest, I'm not entirely sure he remembered that I had interviewed there only a month earlier.

All after I had the title change on my resume. What I'm getting at, is for me, the title change really opened up doors. I figure 'data science' is such a huge buzzword now, recruiters look more for that buzz word than they do for the actual content of what you've done on your resume. I'm sure it will die down at some point.

This may be more prominent outside of silicon valley where I am, and (in my opinion without any facts to back it up), where I feel more weight is given for the title than the substance.

aeorgnoieanghace 9 años

I still feel grateful for an otherwise-pretty-terrible employer for gifting me with a 'developer' title. Regardless of how much experience or aptitude one has or can demonstrate programming or building software, titles are crucial.

graphememeshace 9 años

I don't have much Data Science job hunting experience, however I have applied to and been hunted by over 100 startups, 24 enterprise companies, and 12 large "startups".

I've had the following:

  - 106 Rejections
  - 39 Offers
  - 24 Refused
  - 15 Contemplated
  - 5 Taken

Roles Applied for:

  - Software Engineer (55rj, 8o, 6r)
  - Product Manager (5rj, 6o, 5r)
  - Senior Software Engineer (31rj, 23o, 22r)
  - Product Lead (0rj, 1o, 1r)
  - Senior Product Manager (9rj, 3o, 2r)
  - VP of E (2rj, 1o, 1r)
  - CTO (0rj, 1o, 0r)

It's been a journey. I don't keep an exact list so this is from looking at my calendar and doing some basic math. This certainly doesn't include all of the role changes within companies.

Things to note:

  - People conflate Agile & Scrum a lot.
  - Make it easy for people in the room to know what you personally accomplished during your career.
  - Make an impression, don't overdo it however.
  - Make your descriptions easy to understand.
  - Don't just list tech stacks on your resume, list achievements.
  - Stay consistent.
  - Ask questions.

Also Note:

Regardless how impressive your Github & Stackoverflow accounts might be, you will still be asked to do a stupid code challenge. It irks me. I understand the reasoning for it when you don't have a viable pool of information on the individual, but when you do... It's just disrespectful.

Few places I have been rejected by:

  Salesforce, Atlassian, Trello, Twitch, Segment, AirBnB, Twitter, OKCupid, LinkedIn, Shippo,  and many many more.

Some places I have been offered by:

  Amazon, VMWare, Oracle, Google (twice), Apple (twice), Venmo, Auth0, Discord, Youtube, Hitbox, Steam, Pusher, Cloudflare

michaelscotthace 9 años

This is a really interesting set of stats you have here. Heartening in the sense that rejection is a common and overwhelming likelihood and shouldn't be a source of dejection, but also kind of depressing in the sense that it's clear many, many companies (including some big ones) have a terrible hiring process.

graphememeshace 9 años

Yes, for example some take months with everyone loving you the entire way, for a single person to interview you and reject you without a reason.

I'm glad I had those interviews early on, so now when I see an interview start to drag on, I cut it short as it doesn't look like a priority on their end.

strictneinhace 9 años

> "Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you."

I had a very similar experience. Job offer was basically on the table and then they balked because I mentioned that I had another offer (at a larger company, which they seemed shocked/annoyed with) and I had a question about parking at their new offices. The current offices had free parking, so I had simply asked if the new ones would too. The response was very cold (must have a been the source of internal strife, I guess?). An hour later I got a call saying they were no longer interested.

dsr_hace 9 años

It is always the case that you are better off learning that the company is dysfunctional before your acceptance than after.

It still hurts, of course.

komali2hace 9 años

Both of your points I find extremely surprising. I've found that mentioning that I have other offers makes me a more competitive pick (this reflects my experience when I was a recruiter as well), and that's very weird that they balked at your question about parking. Honestly you probably dodged a bullet.

strictneinhace 9 años

That's been my takeaway too. Company was purchased by a huge private equity firm last fall (KKR), so I probably would have been out of there by now regardless.

BickNowstromhace 9 años

Mentioning that you had another offer can be tricky. Did they ask you if you were seeing other companies? Did you just put it out there? Did you put them on a deadline?

Mentioning that you have another offer is seen as a negotiating tactic. Then they have to compete/bid against the other offer, and you are removing some of their leverage/power. A strong reaction to such a move can be to cut off the interview, so to regain back their power.

PakG1hace 9 años

Yes, you're correct. But it's best to walk away from companies who try to own you to that level. If a good company respects and wants you, they are completely OK with giving up some of their leverage in order to gain your trust as well as they can.

strictneinhace 9 años

I think they may have asked me, although, to be honest, I'm not 100% how it came up. It wasn't me putting them on a deadline though, I just try to be pretty open about things in general.

ryandrakehace 9 años

I've had a company bail on me when I asked about what their policy was on working on open source / side projects on weekends! You never know. Different companies get squirrelly about different things.

leblancfghace 9 años

Just to play devil's advocate here, in many cases the reasons stated for refusal are only one part of it.

In his case, the author doesn't seem to have had an offer on the table. Perhaps the interviewer felt as though the candidate was trying to negotiate too much, too early in the process. Perhaps the interview didn't go over all that well?

draw_downhace 9 años

Wow!

motivichace 9 años

This is a bit off-topic but one thing I'm curious is how the author manages to interview with these companies while holding down a full-time job and keep up-to-date with the latest developments. Interview for a data science position is typically a drawn-out process, with multiple rounds of interviews and possibly take-home projects. I found them to be very time and energy consuming.

To share my story, I also had a difficult time transitioning into a data scientist role after leaving academia (pure mathematics), and I always thought the root cause was my lack of experience and competency. So instead of keeping on applying, I spent over a year just to sharpen up my skills. It paid off in the end.

How can one develop his/her skills and cultivate expertise if one is job-shopping all the time (possibly aimlessly)?

eanzenberghace 9 años

I think being a good DS requires focused learning. For example, when I started out, I didn't know much about neural networks and their different architectures. I tend to find time outside work to go over papers, thesis, or watch lectures on youtube to keep myself up to date so that now I can describe cnn and rnns to tech and non-tech people.

motivichace 9 años

A better but more difficult approach is to distinguish oneself and have the companies go after you.

bsg75hace 9 años

> Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you.

Dodged a bullet on that one:

- PTO is a touchy subject?

- They are looking at more than one candidate, why would any candidate limit themselves to one potential employer?

hudibrashace 9 años

Yeah, that's just weird. Paid time off is part of the total compensation package; it would be strange if a candidate didn't ask about it.

cwilkeshace 9 años

And if they are touchy about it probably also means that they won't pay it out if you leave ... that's the law in California but not in other states like Washington.

I was shocked to hear that, apparently it is somewhat common in retail jobs not to get paid out. Happened to me at a company that claimed to be more of a technology company. Maybe they don't think word gets around. Or that smarter employees will just take a lot of time off before quitting, which does a lot more harm to the company than if they just paid out the time owed to the employee.

preordainedhace 9 años

The too many jobs knock I can understand...I'm just plain old Midwest developer guy, so maybe I don't understand the nuances of a market like Silicon Valley or the like that is inundated with software jobs. Still, when I read HN I often get the impression that there is more focus on getting that next great offer than actually doing a given job...so much so that the ability to be perceived as valuable at interview/screening time seems like the mostly highly focused on or developed skill for some people. It seems like there are many professional interviewees out there these days.

I have a hard time understanding. At least for what I've seen/done, it would take about a year of experience to be of any real value. I'd say you start seeing real dividends from an employee near the three year mark. I think the primary exception would be where you have a big gaping hole in an organization...like building a data science program or something from the ground up. But if you've got software and customers long long past the 1.0 stage to support, and someone is going to (someday) understand/contribute to the core? Think about Google's monolith or the Linux kernel...sure most software isn't that mature/grand, but there are many projects out there that are closer to that than greenfield. And it's not just the code, it's the developed relationships/rhythm with coworkers and customers.

Maybe I've explained it to myself, I don't know. The difference may be startup companies or projects versus mature ones. At least in the latter case, it seems to me that if a company retains an engineer for less than 2-3 years, they've almost certainly lost on that investment.

emplynxhace 9 años

You may well be right.

alain94040hace 9 años

You never really know

That's actually the only correct answer. Having been on both sides of the table for many years, I can pretty much guarantee that whatever reason the candidate is given is nowhere close to the actual reasons.

There may not be any specific reason why we didn't pick you, but we'll give you a tiny sample anyway. So you think that's the reason - it's not.

In other cases, we have a strong reason not to pick you, but it's embarrassing so we feed you a bogus reason instead.

And in more cases than I'd like, there is no problem on your side, we are having internal issues that we can't reveal anyway. Any reason you hear from us in that case is completely irrelevant.

By the way, when you evaluate people in interview, you really need to figure out a vector: where they are today in terms of knowledge and experience, but also in what direction they are going (fast learner or not, high potential, etc.). Which is why sometimes you can hire someone who has less relevant experience, but you think they'll learn fast and are very smart, and sometimes you are looking for the perfect match to the current position, but don't care too much whether they can pick up new stuff or not.

bbarnhace 9 años

HR Departments have also groomed hiring managers not to ever give a reason why they say no. Every reason you provide is grounds for speculation and potential law suits.

[deleted]hace 9 años

CSDudehace 9 años

At least you have been told the reasons, even if they are true or not. I recently had many tech/behavioral interview with team and passed, but after a review with VP of X, saying we decided not to move forward is way worse than this. I still keeep wondering 'What is wrong with me?' even after years of interviews. I have some suspicions, but never a promising answer. If I oneday found a company, first company policy would be to tell the candidates to tell why they were not hired in a polite way.

jedberghace 9 años

> first company policy would be to tell the candidates to tell why they were not hired in a polite way.

And your lawyers would strongly advocate against that.

In most cases they can't/won't tell you because it could open up legal liability. If they say a reason for you, but then hire someone else to whom that reason also applies and you find out, you could sue them. Or you could find a way to twist it into something against a protected class. Too many ways for the company to get screwed.

That's why they all say "we've decided to go another way" or something equally generic.

yuhonghace 9 años

This is part of why I dislike employment anti-discrimination laws. They probably worked better for manual labor jobs. Of course, there is also firing where the laws vary by state. I think CA is one of the most strict, right?

[deleted]hace 9 años

FireBeyondhace 9 años

Even just a little 'context' can help. I had a candidate ask me after the last position I held interviews for "Can you tell me what to improve? I was honestly surprised I didn't get the position, given how well our interviews went".

And he was right, too.

But I broke it down as follows. We received 300+ applications, we filtered down to 100 that were worth even reading in detail. From those we took 50 that we discussed/scored as a group, and came up with 15 people we wanted to interview. Of those we interviewed 10, and of those we interviewed 5 a second time, and 3 a (brief) third time.

"So yes, you did well, and we'd like to keep your name and reach out" (said sincerely). But for him knowing that he didn't screw something up, there was just an even better candidate and he was realistically still in the top 1-2% of applicants was confidence-boosting.

kolinkohace 9 años

How about talking to a therapist / counsellor / coach that specialises in professionals?

Even one or two sessions can help you a lot - they are trained in pinpointing "soft" issues.

emplynxhace 9 años

The reasons are rarely shared unsolicited. It's usually because I asked (no harm in that) or knew someone on the inside.

smilekzshace 9 años

My $0.02: Headcount is a major reason and you totally don't have to feel sad about it.

pascalxushace 9 años

The real reason you were rejected, or anyone applying in a highly competitive field: an oversupply of qualified candidates.

skahace 9 años

In this particular case (data science) it is more an oversupply of candidates (qualified and not), plus difficulties defining and measuring "qualified", plus buzz.

It's a difficult enough field to hire in when you understand what it is (and isn't) - and lots of companies are trying to do it with far more vague goals.

achompashace 9 años

I think this is closer to reality. Tim is not competing with many folks like him -- he's a knowledgable, experienced, and capable data scientist with significant infrastructure experience, and a net positive to any team he joins. Some problems possibly originating from the company perspective

(1) They are inundated with applications folks of all sorts of backgrounds: engineering, finance, academia, marketing, BI/analytics, etc.

(2) They still haven't figured out hiring. To be fair, no one really has figured it out. Jeff Kolesky recently covered this as part of an excellent blog post. [0]

(3) In addition to the typical variance in engineering interview processes, we now introduce variance in the definition of data science across companies, which just complicates things further.

(4) Basically everything else Tim mentioned in his post: role or goals aren't clearly defined, remote data science is an unknown, etc.

[0] http://kolesky.com/datums/job-search/

throw_away_777hace 9 años

Why do you think candidates for data science jobs aren't qualified relative to other positions?

yummyfajitashace 9 años

I've probably interviewed about 70-100 such people in the past year and a half. Exactly 1 such person was qualified (I hired him). The issue in my view is the following: people who know both statistics and computer science are extremely rare.

People who actually understand statistics are rare. I can probably weed out 1/3 to 1/2 of candidates simply by asking what a p-value is, or what precision/recall are (this includes people who said they worked in search).

Of the ones who know basic stats, most are neither good at nor interested in programming. They just want to use existing libraries to crunch numbers in a Jupyter notebook, then hand that off to the developers.

Finding a person who can come up with a predictive model, understand what they did, optimize it without breaking it's statistical validity and deploy it to production is very hard.

(If you can do this, I'm hiring in Pune and Delhi. Email in my profile.)

halflingshace 9 años

Ignoring what a p-value is does not mean that you don't know statistics. p-tests are not some inherent statistical property, they're just a useful model for significance. People coming from a CS background most likely didn't have to deal with p-values, but they can still be good at linear algebra or bayesian statistics.

(not sure I can defend somebody that does not know what precision/recall are)

Bootvishace 9 años

Regarding precision/recall, I've a background in financial econometrics and this is the first time I encounter the terms.

kenjacksonhace 9 años

I think the problem is that certain subfields use different terminology to mean similar or identical concepts. For example, while I'm in software, I tend to hear the terms sensitivity and specificity. They are historically medical terms. They aren't identical to recall/precision, but I think you can derive one set from the other.

antogninihace 9 años

The fundamental thing to know is the confusion matrix. There are about a dozen terms for various descriptors of the matrix, but they all can be calculated if you know the confusion matrix. The Wikipedia page has a great table to describe them all:

https://en.wikipedia.org/wiki/Confusion_matrix

You can see from that that sensitivity and recall are the same thing, but specificity and precision are not.

Bootvishace 9 años

That certainly seems likely and a good thing to keep in mind when you're giving or taking an interview!

jknoepflerhace 9 años

It's literally the first thing you learn in data science / machine learning coursework about evaluating model performance. It would probably be better to ask the candidate to whiteboard a set of metrics for evaluating model performance rather than ask for the definition of a pair of words, but the concept is practically the for-loop of data science.

Edit: note that I'm not saying you need this to add roi as an analyst for a business!

achompashace 9 años

For those downvoting this comment -- it's absolutely true that model performance is discussed at length early in ML courses (usually in the context of the bias-variance tradeoff).

My only quibble would be that precision + recall are one set of evaluation metrics applicable to classification tasks. Modelers can absolutely use other loss functions.

Additionally, precision/recall do not map nicely to regression problems, so people use other metrics (RMSE, MAE, etc.).

Bootvishace 9 años

I haven't taken a lot of data science classes but I'm not sure that's true. If you start with linear regression the mean squared error would make more sense. I actually searched through "The Elements of Statistical Learning" and the word 'recall' is not used in this sense at all.

mjnhace 9 años

The jargon does vary by subfield and community, along with the actual measures used (sometimes it's just a different name, but sometimes practices are different as well). Precision/recall are terms from information retrieval that migrated into the CS-flavored portion of machine learning, but are not as common in the stats-flavored portion of ML, in part because some statisticians consider them poor measures of classifier performance [1]. Hence they don't show up in the Hastie/Tibshirani/Friedman book you mention, which is written by three authors solidly on the stats side of ML. It does occasionally mention some equivalent terms, e.g. Ctrl+F'ing through a PDF, I see that in Chapter 9 it borrows the sensitivity/specificity metrics used in medical statistics, where sensitivity is a synonym for recall (but specificity is not the same thing as precision). It looks like the book more often uses ROC curves, though, which have their own adherents and detractors.

[1] This paper is the one that most often gets cited as background by people who don't like recall/precision as metrics: http://dspace2.flinders.edu.au/xmlui/bitstream/handle/2328/2...

mehaveaccounthace 9 años

People don't pay for linear regressions. They pay for discrete things: what is my best option among my three clear courses of action. Linear regression can be a tiny piece of a larger argument in favor or against one option or the other, but that alone doesn't make money.

Bootvishace 9 años

That's obvious but not at all what I responded to in my post.

I responded to the claim that ML courses start with the definition of precision and recall. In my admittedly limited experience those courses start with linear regression and mean squared errors. After that, there is so much generalization possible and that doesn't include precision/recall.

You make money by solving someone's problems, making money by stating definitions is only done on TV quizzes.

[deleted]hace 9 años

halflingshace 9 años

That's OK. The article was talking about somebody interviewing for a search-related position (where precision and recall are usually what you are optimizing for). I guess they might be called differently in econometrics?

yummyfajitashace 9 años

I usually tailor my questions to whatever the person said they did. If they say they did search, I ask about precision recall. If they discuss hypothesis testing, I'll ask about p-values or other approaches.

I'd happily take a Bayesian answer if they preferred that, but that hasn't happened very often.

x0x0hace 9 años

I can't imagine how you could learn Bayesian stats w/o starting with a basic inference class, which will cover p-values at length.

halflingshace 9 años

I did Bayesian stats without ever discussing p-values. Not al classes discuss them. (my background is ML, where p-values are not as useful as say in biology or any field relying on experiments with control groups)

dandermotjhace 9 años

You're certainly not going to study Bayesian statistics without knowing (or at least having studied) what a p-value is.

AstralStormhace 9 años

Actually p-values are used way less often in Bayesian statistics than frequentist ones. The latter rely on statistical tests more.

Bayesian stats tend to use likelihood ratios or Bayes factors instead of p-values for hypothesis testing.

The trick in all cases is that you're comparing to expected results given some prior distribution. Most people use a dumb prior (e.g. Gaussian) and then they're confused when the numbers make no sense as data is multimodal or heavy tailed, thus mismodelled.

dandermotjhace 9 años

I studied statistics - my point was that statistics is taught in a linear manner, starting with distributions and hypothesis testing (p-values) and then move onto more advanced treatments like Bayesian stats.

maus42hace 9 años

That happens in statistics programs. However, I have a ML-heavy minor in CS, and based on the ML course contents at our CS dept I've seen, I'm not sure if the all their CS majors go through the the full canonical statistics curriculum, nor that they were intended to. At least the ML courses had quite much introductory probability and statistics as far as ML applications were concerned, so I understood the implication was they didn't assume that the students would have already done the similar stuff in statistics (though it certainly helped), and I can't remember a single mention of p-value there.

And then there's this, that even if your intro to probability course everywhere covers the classic statistics with p-values and hypothesis testing and frequentist confidence intervals and so on, you are not necessarily going to use them that much. I calculated some p-values and other tests with R for some example datasets a couple of years ago and never seen them since in coursework, everything we've done after that has been more or less fully Bayesian. The concepts are still fresh[1] in my mind mostly because I read some statistics blogs, such as Andrew Gelman's [2]. The irony is that Gelman does not exactly love frequentist framework, he just mentions its concepts often enough.

[1] or not totally forgotten

[2] http://andrewgelman.com/

dwpdwpdwpdwpdwphace 9 años

I'm surprised people bother spending so much energy looking for someone who is both a statistician and a computer scientist knowing they are so rare. There are so many more statisticians who can at least communicate and work effectively with developers and vice versa. Why not just compose a team? I feel like just like other professionals have assistants, statisticians should have them too, and they'd be focused on the computer science and deployment of the applied statistics.

skahace 9 años

This is a classic problem that shows up equally with lots of related areas: numerical work, statistics, ML, signal processing etc.

"just compose a team" sounds easy, doesn't it? Unfortunately there are lots of failure modes involving different parts of the team not really understanding what each other are trying to do, let alone what they are doing, and subtle errors getting by people who don't know what to look for. So, you can find such teams and some of them work well but a lot of them don't.

So an alternate is to try and find or create domain experts who mix all the appropriate skills, but this is hard and in the extreme case involves chasing down unicorns.

Companies and industries flop back and forth between preferring different approaches - right now a lot of people are talking about "data scientists" as one of the latter, but it will likely change over time as it always does.

It's a hard problem, and it shows.

dwpdwpdwpdwpdwphace 9 años

As an engineer, I usually know better than to use the phrase I hate to hear: "why don't you just..."

pbhjpbhjhace 9 años

Surely "Why don't you just... ?" is an exceptionally good phrase to use. In practice people mean "Just do ... !", which is very different. The why question, however, gets to the heart of an issue, it's a short hand for "The obvious solution appears to be that ... but I imagine you tried that and have a reason not to do things that way, what are those reasons?". It's a direct learning-centred enquiry that ekes out the kernel of complexity of a situation relying on the wisdom of the person it's aimed at.

So why don't you just use the phrase "Why don't you just ...?"?

oliviervahace 9 años

I couldn't agree more. I just got hired to 'productionize' some proof of concept developed by data scientists in jupyter notebooks. The first thing I did was hiring a Python developer (no data science experience) to start cleaning up the code and a devops to put the infrastructure in place. Second step: I went to the data science department and sat down with them and I taught them how to program properly: test driven development, version control, code reviews (git pull requests) and continuous integration. They all have PhD's so it's not that they would have any trouble learning anything new. They thought it was great. Result: all their new code now directly goes (via code review and CI unit/end2end testing) into pre-production and after sign off from the product owners into production (quite often the same day). I just do not understand why instead of trying to find the perfect person for the job people don't just hire someone to teach them how to do the programming part of their job properly. Good teams are cross functional, diverse and have a strong focus on transferring knowledge.

SatvikBerihace 9 años

This is a good question. I've tried both approaches, and currently favor going after the rare multi-skilled hire. In general, I have seen many cases where one person who has a small-medium amount of experience/ability in both is a lot more productive than two specialists.

> There are so many more statisticians who can at least communicate and work effectively with developers and vice versa.

Not in my experience. You need to design your data infrastructure to promote easy analysis, and you need to design your models to scale well according to the amount of data you're working with. There are also many cases where a project will require mostly engineering work for a while, and then mostly analysis/statistics work–there are ways to handle this with specialists of course, but there's generally a significant switching cost.

Also people with a combination of statistics & programming aren't that rare–IMO it's more that employers tend to search for both degrees, when instead you should be trying to evaluate the skills directly.

SatvikBerihace 9 años

I have similar experiences. To add some color: I find that for data science tasks, someone who knows statistics & can program is much, much more productive than someone who only knows one. Part of that is because data science has to do their own product management–the question you ask next changes quite rapidly depending on the results of a single query.

That said, most companies should probably be hiring data engineers rather than data scientists–for most "data science" jobs I've seen, almost no statistics is actually necessary/useful.

drxzclhace 9 años

This is also very true! Usually we hire AI/stats folks and do a heck of a lot of training to get them up to speed on the development side of things. You can do it the other way around, but math is a lot harder to pick up outside of formal education than computer stuff.

johan_larsonhace 9 años

When you say "know statistics", how high are you placing the bar? Lots of people are forced to take one stats class (for non-specialists) in college, and it goes up from there.

skahace 9 años

Two main factors make data science stick out a little for me right now, although it isn't unique.

One is that there is buzz & excitement around "data science" right now. Nothing specific to this area, but in my experiences this creates a large number of under- or un-qualified applicants. It also creates an environment for companies to desire to hire a role they are not well qualified to hire for. It is really difficult to hire well for roles you don't understand well.

The second thing is that extremely few people are actually ready for this sort of job straight out of an academic program. A related Ph.D. or post doc plus a few years solid training in industry can make you a great candidate, but the academic work alone usually isn't even remotely close. There is confusion about this among both candidates (don't know what they don't know) and hiring managers (don't know what they are actually looking for).

Add to that an oversupply of academic credentials relative to academic jobs and you have a problem. If you are a large company with a well defined data science program and a defined "entry level" data science role, if you take skill development and training seriously and have the senior staff for it, well then you are fine taking strong academic candidates and turning them into talented data scientists. If you are a less experienced company looking for scientists to solve a problem you don't fully understand, you may be in for a pretty rough ride.

drxzclhace 9 años

Not OP, but I think many companies aren't qualified to judge who is and who isn't a qualified candidate, at least for the first hires. This turns the whole thing into a "market for lemons".

I've helped a few organisations solve this bootstrap problem by helping out with candidate selection and interviews, but many other just don't ask for help.

emplynxhace 9 años

But that offers no explanation why some are hired and some aren't.

cortesofthace 9 años

Because SOMEONE has to be hired?

Let's supposed, for a moment, that the hiring was completely random. 100 people apply, they pick a random name out of a hat, and hire that person.

Your name might never be chosen.

ryandrakehace 9 años

But, wait! I thought there was a "shortage of tech workers."

x0x0hace 9 años

Which must necessarily be urgently addressed by open immigration for anyone who can write code! The future of silicon valley demands it!

CalChrishace 9 años

Companies often use interviews as a time to figure out what they're really looking for.

For startups, this transcends data science. It might be the one time that week they focus on that need.

Networking is still king.

Exactly and this also argues against wanting to get hired to work remotely.

102142592235adhace 9 años

>Exactly and this also argues against wanting to get hired to work remotely.

True, though not unheard of. I worked at a startup that hired remote devs via networking of that nature. It seemed to work well for them. Eventually the entire team migrated to the same area but things operated smoothly remote for the time being.

emplynxhace 9 años

Not sure networking is contrary to working remotely. Opportunities for networking online are far bigger than in-person.

CalChrishace 9 años

You may be talking about contracting and the OP is talking about hiring. Moreover while the opportunities for networking online are countably infinite they can be thin opportunities. I know Joe from HN doesn't carry the same weight as I know Roger from Google. It carries weight just not nearly as much.

eanzenberghace 9 años

My analytical thought process on DS interviews:

- Signal is still quite low among noise, even with long multiple interviews, take-home homework, coding challenges, etc. Most relevant data is still hidden and takes months-years to come out.

- Companies seek to minimize false-positives much more than minimizing true-negatives.

- It's a numbers game from both ends because the probabilities are low, due to above 2 points.

FireBeyondhace 9 años

Woe betide the company that asks for a take home project, especially one that isn't just 50 lines of an algorithm (I'm talking one that I recall that asked for parsing Apache logs from a stream, displaying moving averages for URLs, aggregates, having high water mark "alerts" and resetting when rolling averages dropped below that, some unit tests and docs...

And doesn't bother to respond after you submit. I'm certainly not the world's best coder, but my solution met all those requirements in a reasonably elegant manner, and took several hours (10-20?) to make comprehensive.

That gets you on my shit list, and people I know get to hear about it, too.

eanzenberghace 9 años

Same FireBeyond. Even worse, I've gotten feedback on a take-home project where the solutions had errors in it and I had the correct response! This, btw, was from one of the huge tech-companies in SF. Since it was the recruiter who went through the solutions, I couldn't nor cared to correct their mistakes :)

donovanmhace 9 años

Is there a typical path for getting into Data Science? Posts like this make it seem like it's the wild west

dbeckerhace 9 años

I've had a few DS jobs. And I've been a hiring manager at two of those jobs. I can confirm that it's roughly the wild west still.

emplynxhace 9 años

It's a Wild West.

autokadhace 9 años

I can't even get a call back for an interview lined up. After hearing people get tons of unsolicited mails on linkedin from recruiters (while I got none), I tried adding more words in my linkedin to show up on searches, but still nothing.

Does anyone have any suggestions? Here is my linkedin account: https://www.linkedin.com/in/karl-dailey-02557b65

rubayeethace 9 años

"Networking is still king"

My takeaway from it.

wgynhace 9 años

I think that just "networking" would be the wrong takeaway, though. The author[0] is pretty active on Twitter / the blogosphere and, even though I don't follow him personally, his writing has popped up on my feed a number of times. He's likely meeting people and having interesting conversations ("networking") because he has interesting things to say.

[0] https://twitter.com/tdhopper

mindcrimehace 9 años

Not just blogging, but Tim is also active in terms of attending / speaking at various industry conferences and what-not. Whether he is doing it simply to "build his brand" or because he genuinely enjoys it, or both, is something I can't speak to. But I think he's definitely "meeting people and having interesting conversations".

emplynxhace 9 años

Hi Phil!

mindcrimehace 9 años

Oh no, I've been outed!

ENGNRhace 9 años

Clicked hoping he was going to do data science on his failed interview data

segmondyhace 9 años

I looked at this resume. it looks good but his job history is a major concern. Of course, some might say why even invite him to an interview, perhaps they gave him a chance only to discover in the interview process that he's really impatient and not willing to stick around for long?

There is a ramp up time for new hire, which could be a couple of months. So durations of a year or less doesn't look good. I personally like to see minimum of 2 years for each job. Of course, too long can be a concern too unless they really grew in their role.

I agree with his conclusion, Network is king, but I also believe in listening to the universe. If everyone is "wrong", maybe you are the problem. If lots of people don't wish to hire you after you have solid experience in the industry, something is wrong with you and you are being stubborn by refusing to recognize it and fix it.

[deleted]hace 9 años

mattfrommarshace 9 años

Ok, this is a little worrying. I've read through the all the conversation here but is it really that difficult to get jobs for people who already are involved in Data Science? Here I was thinking if I do a course on Data Science fundamentals of computer science online, I would on tract of getting a computer science/data science job.

My future really rests upon this. I have a degree in mechanical engineering with irrelevant experience. My only regret is not doing bachelors in computer science.

I wouldn't lie if all the post got me little tensed and made me thinking, "what am I doing with my time". About to finish introduction to Computer Science using Python on Edx.

carterehsmithhace 9 años

That sucks.

That said, what is this?

a) "I cannot do X" b) "But here is some advice on how to do X"

I mean... why would anyone listen to the advice that is proven to lead to failure? Serious question.

[deleted]hace 9 años

LeanderKhace 9 años

Well, it also seems like he applied to a lot of jobs (he worked for 5 companies since 2012, for many only under a year!). He must be constantly looking for a new job.

I know switching jobs is common here, but i would think that sticking at least 1 to 2 for a job would be normal (assuming it works out).

megablasthace 9 años

> you rarely truly know why you were turned down.

Bit of a silly article. You could say this about anything. And I don't see what the problem is with never being contacted, if someone doesn't want me for whatever reason, I don't really want to hear from them again.

emplynxhace 9 años

anotheryouhace 9 años

While I find it reasonable, I would still not ask about vecation policies in the interview, but rather when they decided they want me and we go through the details. Why would you ask before? (honest question)

emplynxhace 9 años

I actually had an offer in hand, and it was rescinded after I asked about PTO.

anotheryouhace 9 años

Than it's good you weeded that one out :)

maxxxxxhace 9 años

Why not?

anotheryouhace 9 años

Not to give them any reason to doubt my working moral.

maxxxxxhace 9 años

If asking about vacation policies makes them doubt your working morale I would recommend to run quickly. Is asking for money offensive to them too?

anotheryouhace 9 años

It just came to my mind that I'm spoiled by the german vacation standards. It probably makes a lot more sense elsewhere.

edblarneyhace 9 años

"Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you."

You wouldn't want to work there.

" but the team has decided to keep looking for someone who might have more direct neural net experience."

Fair enough - but this has to be slim pickins. How many AI jobs are there out there? Realistically, very few.

draw_downhace 9 años

> "Honestly, I think the way you communicated your thought process and results was confusing for some people in the room."

We got confused, and then we stopped. The end.

gillianlishhace 9 años

ironically, the unwashed masses that data science has 'targeted' for 'optimization' go through the exact same process and feel the exact same emotions.

apply for 100 jobs, get zero response, and zero reason why.

be told illogical things.

be told out right lies.

welcome to capitalism. welcome to the workplace that is run entirely by data. (or by people who only care about data). welcome to the future of humanity.

[deleted]hace 9 años