Tips for Hiring a Data Scientist into a Tech Company(blog.infer.systems)
blog.infer.systems
Tips for Hiring a Data Scientist into a Tech Company
https://blog.infer.systems/2020/01/10-tips-for-hiring-data-scientist-into.html
23 comments
You could say the same about recommending someone getting a computer science degree. You may be setting yourself up for disappointment if you’re only in it for the job you can get afterwards, since the work that you’d be doing in the job would at least somewhat depend on the work that needs to be done.
Having worked in data science, my advice would be to hire people who has coding ability.
Who can write working and reviewable code.
Not because statistical knowledge is not important, but because coding skills are often overlooked. I see no role in data science that shouldn’t need at least a junior software engineer level of coding skill.
Who can write working and reviewable code.
Not because statistical knowledge is not important, but because coding skills are often overlooked. I see no role in data science that shouldn’t need at least a junior software engineer level of coding skill.
As a data scientist, I agree with this. On my team, the data scientists with the largest impact seem to be the ones who are sort of hybrid software engineers / data scientists. I’ve noticed these people tend to swap between the two titles occasionally (re-interviewing within the company for one role or the other).
I think if I had my own company I’d create a position like “scientific software engineer”, which would basically be a SWE with a scientific or statistical background. It would de-emphasize pure analysis in favor of directly applied research in the form of production code. There are some similar titles out there (research scientist, ML engineer), but they don’t quite capture exactly what I have in mind. “Applied scientist” would probably be the closest.
I think if I had my own company I’d create a position like “scientific software engineer”, which would basically be a SWE with a scientific or statistical background. It would de-emphasize pure analysis in favor of directly applied research in the form of production code. There are some similar titles out there (research scientist, ML engineer), but they don’t quite capture exactly what I have in mind. “Applied scientist” would probably be the closest.
I've done this at a couple of places, creating "scientific developer" positions. It works reasonably well but has some gotchas. The biggest one is identifying people who are truly interested in developing some depth in both areas.
I agree. Last time I hired a data scientist, we got someone who could code but who'd write very much suboptimal code that worked for his exploration and as a proof of concept, but not much more.
That worked out fine: we paired him with a relatively junior developer that worked with him on translating his proof of concept code into something more suitable for production. I think both of them learned quite a lot of coding and data science respectively from each other, and output was good.
But if our data scientist hadn't been able to produce something that we could run as a starting point, the gap would have been too big.
That worked out fine: we paired him with a relatively junior developer that worked with him on translating his proof of concept code into something more suitable for production. I think both of them learned quite a lot of coding and data science respectively from each other, and output was good.
But if our data scientist hadn't been able to produce something that we could run as a starting point, the gap would have been too big.
What's a concrete example of production DS code? Does it mean that the code handles edge cases, uses logs, is well documented, etc or what exactly? I'm a DS student and don't know what everyone means by "production" code.
It's pretty much the same as production software code.
I'm not talking about code to visualize data and present in some form of media. They are part of DS for sure but more for BI and analysts.
For instance, in my previous experience with data science we've built production pipeline to ingest, digest, create model, serve model on some API. The model updates itself every so often, all without human intervention. Code powering something living in production like that would be production DS code imo.
I'm not talking about code to visualize data and present in some form of media. They are part of DS for sure but more for BI and analysts.
For instance, in my previous experience with data science we've built production pipeline to ingest, digest, create model, serve model on some API. The model updates itself every so often, all without human intervention. Code powering something living in production like that would be production DS code imo.
The other replies here, and your own suggestion all makes sense. Often it's also performance and memory use. The prototype code didn't tend to take into account which shortcuts etc. we needed to take to run things fast enough on hardware of more than a decade ago with the sheer amount of data needed in production.
Which was fine - clarify of the methods was helpful in order for the rest of us to understand what he was trying to do.
Which was fine - clarify of the methods was helpful in order for the rest of us to understand what he was trying to do.
imo as someone who does ds & analytics and has deployed prod models: meets business-defined specs, is maintainable, requires little to no support. everything else falls underneath
tbh i dont think this is something you learn in school, or is expected of new grads
tbh i dont think this is something you learn in school, or is expected of new grads
the fact is all the knowledge that a data scientist has should already be discoverable by any decent programmer. Basically college level statistics is required to be a data scientist. If your engineer cant handle that I hope they are more frontend than backend.
Data science does have particular algorithms and analysis graphs that are unique to the field but the underlying math is set theory and statistics.
"Data scientist" is a fancy title for "smart math guy". But math without application gets you no where. You need someone to put it into the machine.
Data science does have particular algorithms and analysis graphs that are unique to the field but the underlying math is set theory and statistics.
"Data scientist" is a fancy title for "smart math guy". But math without application gets you no where. You need someone to put it into the machine.
Depends. If it’s a spam classifier, yeah I agree. If it’s estimating causal models for some type of economic analysis it will be harder for a self taught SDE to compete with someone with academic research experience dealing with messier social sciencey economic data.
Actually spam classifiers are not easy at all.
First, the data is very unbalanced.
Second, you would need to retrain the model often. So you would need to understand statistical methods for comparing text level/word level distributions. This is not a college-level stat.
Third, you would need deep knowledge in NLP, feature engineering and algorithm selection.
First, the data is very unbalanced.
Second, you would need to retrain the model often. So you would need to understand statistical methods for comparing text level/word level distributions. This is not a college-level stat.
Third, you would need deep knowledge in NLP, feature engineering and algorithm selection.
I didn't say it's easy, I said it's something more amenable to an SDE self-learning.
[deleted]
I think the point is that the person with academic research experience dealing with messier social sciencey economic data would/should have basic junior SDE level programming skill.
If not, say that his research is conducted in excel, I'd still rather have the former.
If not, say that his research is conducted in excel, I'd still rather have the former.
Yeah. I agree with you, as I have both the academic research skills and the SDE level programming skills as a DS (by only doing a masters I was able to get out of academia earlier and develop some SDE skills instead of writing a dissertation). But every once in a while we hire someone who is in their early 30s and just finished academia, and despite sucking at engineering, can blow everyone away scientifically. I think a well crafted 2-pizza team can support 1 or 2 of those people.
> 2-pizza team
Unrelated, but since you spoke of this, I just want to tell you whoever came up with this term severely underestimated our teams appetite.
Unrelated, but since you spoke of this, I just want to tell you whoever came up with this term severely underestimated our teams appetite.
This thinly veiled advertisement gives me almost no confidence that the posting firm has ever met a data scientist much less possesses expertise in how to effectively recruit and build a function for them at a small tech company.
The first things to understand is if you are hiring for a DS role to support product development/understanding, growth or marketing, advanced BI, core product features (eg ML in your offering) etc. What VP your DS would report to in a 500 person version of your org chart makes a huuuge difference in understanding what skills and aptitudes you are looking for.
Design the position and responsibilities now and in 2 yrs for the ideal candidate, then design your hiring strategy.
The first things to understand is if you are hiring for a DS role to support product development/understanding, growth or marketing, advanced BI, core product features (eg ML in your offering) etc. What VP your DS would report to in a 500 person version of your org chart makes a huuuge difference in understanding what skills and aptitudes you are looking for.
Design the position and responsibilities now and in 2 yrs for the ideal candidate, then design your hiring strategy.
> you might find that you are not looking for a Data Scientist at all, perhaps you are in fact looking for a Data Engineer or a Data Ops person
Strong agree
Managers who haven't been through this process once will hire a smart stats person who can't operate without a full-time engineer pulling production data.
Which can be crippling for small teams. The alternative is to make your PM do their own data work or promote a product-minded engineer into this role.
Strong agree
Managers who haven't been through this process once will hire a smart stats person who can't operate without a full-time engineer pulling production data.
Which can be crippling for small teams. The alternative is to make your PM do their own data work or promote a product-minded engineer into this role.
Your PM can make you good looking charts and do basic summary statistics (mean, median, percentiles, etc.) I would call that business analyst work. It's probably good enough in a lot of situations. But if you actually need statistical analysis, there's no substitute for a statistician. Even working scientists in the hard sciences routinely screw up their stats and should be (but mostly aren't) consulting with a staff statistician for their papers.
This is a meaningless checklist; the only actual useful advice is to get the word out at meetups and such. The way you hire a data scientist is much the way you hire into any other team.
If you have a data scientist who is worth anything, he should be given responsibility for building the DS team. If you don't: you probably shouldn't hire one. If you must: hire one with experience, if necessary, as a consultant to build your DS team.
Putting physicists at the top rank is also a bad piece of advice; people with experience are at the top rank for DS. If it's pure fresh meat, you're better off with applied math people than physics people (and I am physics people).
If you have a data scientist who is worth anything, he should be given responsibility for building the DS team. If you don't: you probably shouldn't hire one. If you must: hire one with experience, if necessary, as a consultant to build your DS team.
Putting physicists at the top rank is also a bad piece of advice; people with experience are at the top rank for DS. If it's pure fresh meat, you're better off with applied math people than physics people (and I am physics people).
Most companies vary wildly in expectation for these roles. You might be asked to spend most of time data engineering (like making ETL pipelines in one), or build a bunch of basic ML models , or spend all your time doing SQL queries and building dashboards in another.
It’s silly and would never recommend anyone pursue that path as a lifelong career. Learn to code if you don’t know already, and just go into a software role that is data or ML focused.