Potemkin Data Science (2020)(mcorrell.medium.com)
mcorrell.medium.com
Potemkin Data Science (2020)
https://mcorrell.medium.com/potemkin-data-science-fba2b5ba5cc6
64 comments
I agree with this. I think you really need to be a good data engineer to be a good data scientist. Now, these terms are different roles at different organizations.
I think you need to have a well developed software engineering skillset. You need to have some experience with database systems and query optimization. Of course you need a strong background in statistics and mathematics. Often these skillsets are split across teams and departments, which tends to slow the process down. It takes a long time to develop these skills, but if you have them, you are very valuable.
Many DS and sometimes companies make a mistake of undervaluing the software engineering skillset. However, if your data comes from the real world, it will be messy, it will need to be organized. If you don't know how to do that, then you aren't going to build your models. If you are bad at querying your data, then you are going to build your models much more slowly.
I think you need to have a well developed software engineering skillset. You need to have some experience with database systems and query optimization. Of course you need a strong background in statistics and mathematics. Often these skillsets are split across teams and departments, which tends to slow the process down. It takes a long time to develop these skills, but if you have them, you are very valuable.
Many DS and sometimes companies make a mistake of undervaluing the software engineering skillset. However, if your data comes from the real world, it will be messy, it will need to be organized. If you don't know how to do that, then you aren't going to build your models. If you are bad at querying your data, then you are going to build your models much more slowly.
Do you think it makes sense for someone to combine these skills? My (unexamined) belief is that its better to have them split out. My reasoning:
1. Data science is often low latency work, running investigations to provide insight for strategic decisions. This is more of a political/comms role - and the cadence/requirements of the role tend to be counter to the long periods of uninterrupted focus that an engineer needs.
2. DE work is foundational, but once it's built, you probably want someone who is fast at adhoc analysis. (Unless you want to push into production - but that's a 3rd skillset, IMO)
3. It's hard to be both user and designer; having two brains has saved me from many mistakes related to bias and assumptions that I make. The DS can function almost as a product manager, synthesizing information and focusing on deliverables (the "what"). The DE can focus on the how.
Would love to hear your thoughts - this is an area that I'm very interested in.
1. Data science is often low latency work, running investigations to provide insight for strategic decisions. This is more of a political/comms role - and the cadence/requirements of the role tend to be counter to the long periods of uninterrupted focus that an engineer needs.
2. DE work is foundational, but once it's built, you probably want someone who is fast at adhoc analysis. (Unless you want to push into production - but that's a 3rd skillset, IMO)
3. It's hard to be both user and designer; having two brains has saved me from many mistakes related to bias and assumptions that I make. The DS can function almost as a product manager, synthesizing information and focusing on deliverables (the "what"). The DE can focus on the how.
Would love to hear your thoughts - this is an area that I'm very interested in.
I think if you have the desire and the time to combine these skills, then it's a good idea to do this. If you enjoy building and optimizing pipelines and if you enjoy doing creating analytics, stats and building models, you will be very much in demand and it will be pretty lucrative. Most people don't do this, because it takes a long time, and they enjoy one over the other.
1. This really depends on the DS role at a company. You are describing a company where DS is highly valued and has a lot of power. They probably have representation with a VP or in the C suite. Some companies are like this.
2. DE should be foundational, and they should also be able to push to production. You might have another team which works on setting up new platform environments, but DE better be comfortable pushing and debugging issues in production. It's helpful for DE to be able to perform exploratory analysis on a dataset. It's easy to introduce bugs, if DE can catch this, then it really improves the velocity. If they can't do this, then you need to wait for DS or somebody to act like QA for a dataset. You can find cheaper DEs who are good at shoveling data around, but they don't know what a probability distribution is.
3. It's helpful for DS to have some domain knowledge. If they have that, and DS is highly valued at the company, then it makes sense for the DS role to contain a Product Management component.
I'm glad you are interested in my thoughts on this. If you have any more follow up questions, feel free to ask.
1. This really depends on the DS role at a company. You are describing a company where DS is highly valued and has a lot of power. They probably have representation with a VP or in the C suite. Some companies are like this.
2. DE should be foundational, and they should also be able to push to production. You might have another team which works on setting up new platform environments, but DE better be comfortable pushing and debugging issues in production. It's helpful for DE to be able to perform exploratory analysis on a dataset. It's easy to introduce bugs, if DE can catch this, then it really improves the velocity. If they can't do this, then you need to wait for DS or somebody to act like QA for a dataset. You can find cheaper DEs who are good at shoveling data around, but they don't know what a probability distribution is.
3. It's helpful for DS to have some domain knowledge. If they have that, and DS is highly valued at the company, then it makes sense for the DS role to contain a Product Management component.
I'm glad you are interested in my thoughts on this. If you have any more follow up questions, feel free to ask.
I agree with the fact that they need to be treated as separate roles, although a lot of organizations just don't understand this well enough and you end up having data engineers doing data science, or data scientists doing data engineering, or both, maybe without the teams ever really talking to each other.
However I think that's a somewhat different issue from the one under discussion here. The problem here is that data scientists can end up wasting time trying to apply fancy techniques, when the real impact would come from basic data analysis using old-school statistics.
Interestingly, I have never encountered this in my personal experience. I don't personally know any data scientist who would rather use in neural network win something simpler would suffice.
Also, I disagree with the idea that data scientists don't need long periods of concentration. If anything, data science is more of a "creative" enterprise, whereas coding tends to be somewhat methodical and mechanical. For me at least, that means long periods of time spent trying different things and contemplating, which requires just as much deep focus as writing a complicated piece of software.
However I think that's a somewhat different issue from the one under discussion here. The problem here is that data scientists can end up wasting time trying to apply fancy techniques, when the real impact would come from basic data analysis using old-school statistics.
Interestingly, I have never encountered this in my personal experience. I don't personally know any data scientist who would rather use in neural network win something simpler would suffice.
Also, I disagree with the idea that data scientists don't need long periods of concentration. If anything, data science is more of a "creative" enterprise, whereas coding tends to be somewhat methodical and mechanical. For me at least, that means long periods of time spent trying different things and contemplating, which requires just as much deep focus as writing a complicated piece of software.
It depends a bit on the definition of the roles, however this is basically what I am arguing against.
I think you are right that there are organizational trade-offs, especially in day-to-day operations.
However, I would claim that if DE and DS are divorced enough, there is essentially no way that DS work can be done correctly and it would be up to chance whether the DS division in your firm actually does useful work, or does all the things that the article bemoans.
Much frustration with Data Science (also as indicated in the article) arise because DS is seen as an ad-hoc creative enterprise to squeeze "insights" out of an existing data-repository and pipeline. Instead, I'd argue, that anything that touches data needs to be handled by both a DE and a DS (not necessarily the same person).
There are fundamental reasons for this. If you really want to draw insights from data, then you can not simply take some data set and go at it. From a standpoint of statistical identification, data and model go hand in hand. It's not a model that is identified, it's a model on a specific statistical process (that results in a set of data). And frequently, thinking about how to generate and process the data is much more important than the choice of which model to run.
In other words, DS need to be involved from the moment the data starts to exist as such, in particular in the moment the data is collected or generated.Otherwise, DS are working blindfolded - which is really the norm right now.
Think of it this way: if the DS process in your company starts with the tabular data that goes into the model, then there is essentially no way that the process of creating a statistical model can be right. If the outcome works, it is as much chance as it is skill.
If the work of DS is defined by starting with some mysterious yet usable set of input data, collected somewhere else "in the machine" and squeezed together for the DS's convenience, then I highly doubt that DS will be a popular job description in the next ten years!
Again, this does not mean DS and DE should be the same person. But since many points about data are as important as they are subtle, an organizational disconnect is at least risky.
Edit:
Currently the endorsed process is to build a model, eventually be surprised by "data drift" and then rebuild everything.
In my view, this is all backwards. Sure, data changes and one needs to iterate. But this is something the DS team should have been aware of beforehand (in most cases). What I mean here is that the surprise factor conclusively shows that no one really thought about crucial properties of the data while using it. From a statistical standpoint, this casts great doubts on these sort of analyses and models. If you are surprised by data drift, at least to the degree that it seems totally unanticipated, most likely you spent too little time on data and too much time on model.
We all know examples of this. You get "some" data collected on marketing impact on young adults and you find out that doing X does Y. And then later on, the whole thing stops working. NOW you begin an error analysis and find out that data was collected in some semi-random fashion from phone users in the US, whereas now you have new audiences from Twitch (or whatever) in SE Asia. The problem here is not the model. The problem is also not that you need to iterate. The problem is that through the opaque process of data generation no one knew, could know, or cared to know what exactly the properties of the data were.
Right now we are using an ad-hoc approach. Either, because nobody really knows how to characterize the data and what implications for inference this has (for example because statistical issues are dense, poorly communicated and consequently rarely taught in anything ML and DS), or because the organizational disconnect from data collection makes it impossible to reason about these issues.
I think you are right that there are organizational trade-offs, especially in day-to-day operations.
However, I would claim that if DE and DS are divorced enough, there is essentially no way that DS work can be done correctly and it would be up to chance whether the DS division in your firm actually does useful work, or does all the things that the article bemoans.
Much frustration with Data Science (also as indicated in the article) arise because DS is seen as an ad-hoc creative enterprise to squeeze "insights" out of an existing data-repository and pipeline. Instead, I'd argue, that anything that touches data needs to be handled by both a DE and a DS (not necessarily the same person).
There are fundamental reasons for this. If you really want to draw insights from data, then you can not simply take some data set and go at it. From a standpoint of statistical identification, data and model go hand in hand. It's not a model that is identified, it's a model on a specific statistical process (that results in a set of data). And frequently, thinking about how to generate and process the data is much more important than the choice of which model to run.
In other words, DS need to be involved from the moment the data starts to exist as such, in particular in the moment the data is collected or generated.Otherwise, DS are working blindfolded - which is really the norm right now.
Think of it this way: if the DS process in your company starts with the tabular data that goes into the model, then there is essentially no way that the process of creating a statistical model can be right. If the outcome works, it is as much chance as it is skill.
If the work of DS is defined by starting with some mysterious yet usable set of input data, collected somewhere else "in the machine" and squeezed together for the DS's convenience, then I highly doubt that DS will be a popular job description in the next ten years!
Again, this does not mean DS and DE should be the same person. But since many points about data are as important as they are subtle, an organizational disconnect is at least risky.
Edit:
Currently the endorsed process is to build a model, eventually be surprised by "data drift" and then rebuild everything.
In my view, this is all backwards. Sure, data changes and one needs to iterate. But this is something the DS team should have been aware of beforehand (in most cases). What I mean here is that the surprise factor conclusively shows that no one really thought about crucial properties of the data while using it. From a statistical standpoint, this casts great doubts on these sort of analyses and models. If you are surprised by data drift, at least to the degree that it seems totally unanticipated, most likely you spent too little time on data and too much time on model.
We all know examples of this. You get "some" data collected on marketing impact on young adults and you find out that doing X does Y. And then later on, the whole thing stops working. NOW you begin an error analysis and find out that data was collected in some semi-random fashion from phone users in the US, whereas now you have new audiences from Twitch (or whatever) in SE Asia. The problem here is not the model. The problem is also not that you need to iterate. The problem is that through the opaque process of data generation no one knew, could know, or cared to know what exactly the properties of the data were.
Right now we are using an ad-hoc approach. Either, because nobody really knows how to characterize the data and what implications for inference this has (for example because statistical issues are dense, poorly communicated and consequently rarely taught in anything ML and DS), or because the organizational disconnect from data collection makes it impossible to reason about these issues.
My perspective on this as a data scientist:
Data science as a field (and as a job title) is broad and generally splits into
- research
- software engineering and data work
- product/business decision making
With many data science jobs being some blend of the three instead of staying purely in one category. And you can split each category even further. The point is that there is so much breadth that people can have the same title and completely different job descriptions. You need a disclaimer about this pretty much every time you discuss anything data-science related.
The article focuses exclusively on the third category of data science (as does most of the criticism of data science). So to understand the article as criticism towards the field, you need to take two things into account
- what proportion of all data science jobs are predominantly in the third category or involve any influence on business/product decisions (anectodally, not that many)
- how many of those jobs and teams actually exhibit the dysfunctions listed in the article
I don't have any numbers to answer these questions.
Personally, I'm interested in the engineering type of data science jobs (ML engineering, scientific computing and other adjacent roles). I wouldn't want to be in a position where I have to advise on decisions, or come up with justifications for someone's business/product choices, or continually have to come up with excuses for my existence in the company. In my current job search I have a preference for positions titled "software engineer" just to avoid all of this ambiguity. It's also very tempting to switch to being a backend developer or something similar and drop the whole data science thing altogether.
Data science as a field (and as a job title) is broad and generally splits into
- research
- software engineering and data work
- product/business decision making
With many data science jobs being some blend of the three instead of staying purely in one category. And you can split each category even further. The point is that there is so much breadth that people can have the same title and completely different job descriptions. You need a disclaimer about this pretty much every time you discuss anything data-science related.
The article focuses exclusively on the third category of data science (as does most of the criticism of data science). So to understand the article as criticism towards the field, you need to take two things into account
- what proportion of all data science jobs are predominantly in the third category or involve any influence on business/product decisions (anectodally, not that many)
- how many of those jobs and teams actually exhibit the dysfunctions listed in the article
I don't have any numbers to answer these questions.
Personally, I'm interested in the engineering type of data science jobs (ML engineering, scientific computing and other adjacent roles). I wouldn't want to be in a position where I have to advise on decisions, or come up with justifications for someone's business/product choices, or continually have to come up with excuses for my existence in the company. In my current job search I have a preference for positions titled "software engineer" just to avoid all of this ambiguity. It's also very tempting to switch to being a backend developer or something similar and drop the whole data science thing altogether.
Hi beforeolives—
I like your breakdown, and I've observed similar things in my experience as an engineering focused data person! I've had many discussions with my colleagues about how to manage effectively these different blends of roles and skills.
I'm looking for someone for an engineering type of data role right now. Is there a way to get in touch with you about it?
Our product helps companies listen to their customers by unifying natural language feedback across various channels, applying signals using various natural language modeling techniques, then aggregating them to help teams deliver better outcomes using more relevant information.
Hope to hear from you (brandon at frame.ai) :)
edit: forgot to share agreement for your breakdown
I like your breakdown, and I've observed similar things in my experience as an engineering focused data person! I've had many discussions with my colleagues about how to manage effectively these different blends of roles and skills.
I'm looking for someone for an engineering type of data role right now. Is there a way to get in touch with you about it?
Our product helps companies listen to their customers by unifying natural language feedback across various channels, applying signals using various natural language modeling techniques, then aggregating them to help teams deliver better outcomes using more relevant information.
Hope to hear from you (brandon at frame.ai) :)
edit: forgot to share agreement for your breakdown
Meh, in the real world, the boundaries aren't so clear.
In most organizations, data science or analytics will likely play all these Potemkin-ish roles to varying degrees, while also adding real value to the business.
Sure, you may do some "decision laundering" for an executive, but that executive may then go on to help champion data science and develop a data-driven culture because by doing him or her a solid every now and again, you build a relationship. Favor trading may seem dirty or impure to a lot people, but it's also how a lot of great work gets done.
And sure, your org will likely use data science as marketing with a healthy does of McKinsey-inspired corporate BS thrown in. This does not and generally will not preclude the organization from also using data science productively and appropriately.
Anyone who thinks you can rise above petty politics to do some sort of "pure" data science is going to find themselves, frustrated, ineffective and burnt out.
In most organizations, data science or analytics will likely play all these Potemkin-ish roles to varying degrees, while also adding real value to the business.
Sure, you may do some "decision laundering" for an executive, but that executive may then go on to help champion data science and develop a data-driven culture because by doing him or her a solid every now and again, you build a relationship. Favor trading may seem dirty or impure to a lot people, but it's also how a lot of great work gets done.
And sure, your org will likely use data science as marketing with a healthy does of McKinsey-inspired corporate BS thrown in. This does not and generally will not preclude the organization from also using data science productively and appropriately.
Anyone who thinks you can rise above petty politics to do some sort of "pure" data science is going to find themselves, frustrated, ineffective and burnt out.
> but that executive may then go on to help champion data science and develop a data-driven culture
That's not an end in itself, though, is it?
That's not an end in itself, though, is it?
Yeah, financial consulting firms basically played this game for a long time; but they know the game, always give a report that satisfies the person who brought you in. Luckily most "data scientists" also learn pretty quick who butters their bread and 'correct' their results accordingly. Those who aren't 'team players' tend to not get brought into meetings with the higher ups, and eventually move onto real jobs.
this doesn't sound like science, or data science.
sounds more like consluting services
Did you read the article? This was essentially it's point about most corporate DS operations. Science is boring and takes a long time, but building a regression model on an excel sheet is easy and doesn't burn too many brain cells
Overall pretty great points. I have been inside a Fortune 100 company that had a totally backward Data Science team that was running to crazy levels of complexity with no known goal in sight.
One thing that is worth pointing out is that the deficiency in identifying goals before running after impressive technological complexity as a status symbol is not restricted to just Data Science. I see it all the time with technologies like "Bluetooth". Let's put Bluetooth in everything. We don't know why but it seems impressive so let's just do it.
One thing that is worth pointing out is that the deficiency in identifying goals before running after impressive technological complexity as a status symbol is not restricted to just Data Science. I see it all the time with technologies like "Bluetooth". Let's put Bluetooth in everything. We don't know why but it seems impressive so let's just do it.
[deleted]
> Former Florida Department of Health data scientist Rebekah Jones is one such example: she was fired for apparently refusing to doctor the COVID data being presented to the public by the state government
An unfortunate choice of example in hindsight
An unfortunate choice of example in hindsight
I guess you're referring to recent reporting that she might have made up some of her earlier claims, e.g. that she was being asked to "delete" deaths from the system: https://www.nationalreview.com/2021/05/rebekah-jones-the-cov.... Does look like she might not be the hero some people think she is, but maybe let's wait a bit on other reporting before vilifying her.
What other reporting should we wait for? Looks pretty damning to me.
Any other news source than the one that rejects climate change and originated the Obama birther movement? :-)
Given there are no major non-partisan news sources anymore (at least in mainstream infotainment press, I'm not speaking of narrow specialized industry publications), and partisan ones are completely OK with suppressing or distorting information that does not fit the current partisan narrative, your request may just be impossible. Any political controversy that site A finds useful would be touted by site A's press, and suppressed or declared insane conspiracy theory by site B's press. And vice versa - if site A's press says something is wrong, and it'd be useful for site B if it were true, then site B would report about it. I wish there were some media outlets that you could use as a measuring stick and say "if they report on X then X must be real, and if they don't, then no way X is real" - but I think if those existed, they died out.
Fortunately for me, I dont practice the "falsus in uno, falsus in omnibus" method of news analysis and so therfore, the previous stories from this outlet are irrelevant. What makes the follow-up story important is that it suddenly occurred to me that the original reports about Jones depended primarily on her word as I recall. That recollection is enough for me to dismiss the story as uncorroborated. MY standard, not the source link posted here.
So many anecdotes in business writing are uncorroborated, that's hardly enough to make it "an unfortunate example in hindsight" is it? Unless you believe that it has now been proven beyond reasonable doubt that she faked the whole thing, but in that case, fortunately for me, I take the reporting of a single questionable source as evidence but not as unassailable truth. But to each his own epistemic preferences.
Tangentially related to the article, but it mentions the Rebekah Jones story, which has been thoroughly dissected here: https://www.nationalreview.com/2021/05/rebekah-jones-the-cov...
I tried to read through this hit-piece "article" but it was too painful. Are there any facts in there that you would like to highlight, or is it just a 1000-word list-of-names that Rebekah should be called by?
I mean if you refuse to read an article, you can't expect to learn much. You really should give it a read because the story is quite bizarre:
- She was a low level GIS developer working on the state dashboard
- After repeatedly posting internal data on personal social media accounts, she was asked to stop and her responsibilities were changed
- She escalated by intentionally sabotaging the rest of the team's work, locking other employees out by removing their security credentials, and by moving data out of the system, crashing the dashboard
- Even after discovery, she was not terminated, and an arrangement was established to have her go through consulting with management
- She then sent an email to everyone working on the dashboard falsely asserting she was being fired for not manipulating the dashboard, which was picked up by the press immediately. In response, she was actually terminated within a couple days.
- Her story, early on, waffled on whether the dashboard was accurate or not, and at times she asserted it was indeed transparent and accurate. Over the last year the story has evolved to the point where she now has a long list of individuals and organizations who apparently have conspired to falsify data.
- She claimed that she was raided by police last year to confiscate her laptop containing data the state didn't want getting out, during which the police waived a gun at her husband and children. However, it turned out she was related entirely unrelated to her work, and they were raiding the house because she had stolen the data of 19,000 employees of the Florida Department of Health from her home, and they traced the IP address there. She also took a video of the encounter in which she waived a sign saying "Biden Hire Me". And bodycam footage from the officers disproved her claims of guns being waived at her family.
- Using her fame and story about being a "whistleblower", she has raised hundreds of thousands of dollars online
- She continues to maintain her own dashboard, but admits she produces it using the same data source that the official state of Florida dashboard does. The difference in what they show is in their methodology for processing the raw data, but other epidemiologists and data scientists claim she clearly overcounts things like deaths and cases due to obvious errors, like counting both PCR positive tests and positive anti-body tests, which would include many vaccinated individuals and double count individuals who take both tests, as well miscalculating "excess deaths".
Apparently she also has long history of felonies/misdemeanor charges including a trial ongoing since 2019 in which she is accused of stalking her previous boyfriend. Hard to say how relevant it is to the dashboard fiasco, but at the very least makes it seem like she is an unstable individual.
At any rate, in addition to the unrelated trials, she is now awaiting trial for the data breach, as a judge signed a warrant for her arrest after reviewing the evidence seized in the aforementioned police raid.
- She was a low level GIS developer working on the state dashboard
- After repeatedly posting internal data on personal social media accounts, she was asked to stop and her responsibilities were changed
- She escalated by intentionally sabotaging the rest of the team's work, locking other employees out by removing their security credentials, and by moving data out of the system, crashing the dashboard
- Even after discovery, she was not terminated, and an arrangement was established to have her go through consulting with management
- She then sent an email to everyone working on the dashboard falsely asserting she was being fired for not manipulating the dashboard, which was picked up by the press immediately. In response, she was actually terminated within a couple days.
- Her story, early on, waffled on whether the dashboard was accurate or not, and at times she asserted it was indeed transparent and accurate. Over the last year the story has evolved to the point where she now has a long list of individuals and organizations who apparently have conspired to falsify data.
- She claimed that she was raided by police last year to confiscate her laptop containing data the state didn't want getting out, during which the police waived a gun at her husband and children. However, it turned out she was related entirely unrelated to her work, and they were raiding the house because she had stolen the data of 19,000 employees of the Florida Department of Health from her home, and they traced the IP address there. She also took a video of the encounter in which she waived a sign saying "Biden Hire Me". And bodycam footage from the officers disproved her claims of guns being waived at her family.
- Using her fame and story about being a "whistleblower", she has raised hundreds of thousands of dollars online
- She continues to maintain her own dashboard, but admits she produces it using the same data source that the official state of Florida dashboard does. The difference in what they show is in their methodology for processing the raw data, but other epidemiologists and data scientists claim she clearly overcounts things like deaths and cases due to obvious errors, like counting both PCR positive tests and positive anti-body tests, which would include many vaccinated individuals and double count individuals who take both tests, as well miscalculating "excess deaths".
Apparently she also has long history of felonies/misdemeanor charges including a trial ongoing since 2019 in which she is accused of stalking her previous boyfriend. Hard to say how relevant it is to the dashboard fiasco, but at the very least makes it seem like she is an unstable individual.
At any rate, in addition to the unrelated trials, she is now awaiting trial for the data breach, as a judge signed a warrant for her arrest after reviewing the evidence seized in the aforementioned police raid.
Any story that drenches it's premise in hyperbole like this can be immediately discounted in my view, as there's an almost zero chance that I'm reading a dispassionate objective overview of established facts. The article reads like random gutter journalism.
> But it’s not true. Indeed, it’s nonsense from start to finish. Jones isn’t a martyr; she’s a myth-peddler. She isn’t a scientist; she’s a fabulist. She’s not a whistleblower; she’s a good old-fashioned confidence trickster.
> But it’s not true. Indeed, it’s nonsense from start to finish. Jones isn’t a martyr; she’s a myth-peddler. She isn’t a scientist; she’s a fabulist. She’s not a whistleblower; she’s a good old-fashioned confidence trickster.
While I agree on your basic premise, this might be one of those times where a horrible article makes valid points and is correct in its assessment of the situation.
I think this is one of the greatest flaws of current journalism: By increasing the "feel good value" for its intended audience and confirming their respective biases, an author can completely disenfranchise readers coming from a different point of view.
You and me were both repulsed by the sweeping statements and lack of nuance and, since time is precious and the article looked like yet another propaganda tool, you rejected its value. Rightly so.
Yet in it where description that were correct and pointed to a deeper issue of whom we believe and why.
So by writing for just a section of a possible audience any discussion is impossible.
I won't read the National Review (just this article) and thus their content will be irrelevant to any good discussion I'll have in the future.
I think this is one of the greatest flaws of current journalism: By increasing the "feel good value" for its intended audience and confirming their respective biases, an author can completely disenfranchise readers coming from a different point of view.
You and me were both repulsed by the sweeping statements and lack of nuance and, since time is precious and the article looked like yet another propaganda tool, you rejected its value. Rightly so.
Yet in it where description that were correct and pointed to a deeper issue of whom we believe and why.
So by writing for just a section of a possible audience any discussion is impossible.
I won't read the National Review (just this article) and thus their content will be irrelevant to any good discussion I'll have in the future.
You are not wrong that its over the top to use that kind of language, but if you don't actually read the article to see if the author indeed making an argument without any established facts, you aren't making any more of an attempt at objectivity as a reader than the author is.
Jesus I feel violated for even remotely believing in that person's story before.
I read this piece and kept reading hoping that they would mention the real reason for this and it didn’t happen.
The vast majority of data scientists are terrible at their jobs. They lack fundamental skills on how to extract business value from data and share that value with the organization.
The fact of the matter is the majority of organizations don’t need FANG style data science with esoteric models and pipelines.
It’s very similar to the current software developer trend of over/pre optimizing or resume driven development that is ravaging the industry today.
The vast majority of data scientists are terrible at their jobs. They lack fundamental skills on how to extract business value from data and share that value with the organization.
The fact of the matter is the majority of organizations don’t need FANG style data science with esoteric models and pipelines.
It’s very similar to the current software developer trend of over/pre optimizing or resume driven development that is ravaging the industry today.
Tech is fashion driven. Ten years ago it was Hadoop/SVM/MongoDB, five years ago Spark/CNN/Neo4j. If today you appear at the door with Nomad/MCMC/Riak, you will be looked down. We are expecting something like K8s/Prophet/Snowflake.
Somewhere in the same company might be someone making the same as you with a couple of SQL/Qlik courses in their well groomed head and a handful of ironed shirts in their closet. But we will look down at them because dressing well is for sheep and SQL is like, 100 years old.
Somewhere in the same company might be someone making the same as you with a couple of SQL/Qlik courses in their well groomed head and a handful of ironed shirts in their closet. But we will look down at them because dressing well is for sheep and SQL is like, 100 years old.
I've worked in what you could call "data science" consulting for about 6 years. I also know data scientists that work in-house in data centric businesses, and have some acquaintance with in-house data science groups in bigger traditional businesses.
In all honesty, almost all firms that are hiring consultants to do data science projects (based on the sample I've been exposed to) fall firmly into the situations the article describes. It's all about appearance and justifying existing decisions or plans, and even the suggestion of changing something swamped out of existence by the bureaucratic immune system of these orgs.
On the other hand, I see (especially in advertising but also e.g. fintech and others) cases where an internal data science team really does drive the agenda, and thing like AB testing are used for real in order to make changes that result in more money.
Lastly, the in-house team at a legacy company. Although I have less experience here, the ones I've seen closely mirror "innovation" or transformation groups that are supposed to come up with new initiatives or changes but inevitably are ignored because they have no links to operations.
So over all, the article resonated. As a data scientist , there is meaningful work at companies where data is closely linked to operations, mostly in "new" businesses. As a consultant, I find it interesting to understand legacy businesses and their data, and learn about what potential upside there is, even if I know they won't listen to me.
In all honesty, almost all firms that are hiring consultants to do data science projects (based on the sample I've been exposed to) fall firmly into the situations the article describes. It's all about appearance and justifying existing decisions or plans, and even the suggestion of changing something swamped out of existence by the bureaucratic immune system of these orgs.
On the other hand, I see (especially in advertising but also e.g. fintech and others) cases where an internal data science team really does drive the agenda, and thing like AB testing are used for real in order to make changes that result in more money.
Lastly, the in-house team at a legacy company. Although I have less experience here, the ones I've seen closely mirror "innovation" or transformation groups that are supposed to come up with new initiatives or changes but inevitably are ignored because they have no links to operations.
So over all, the article resonated. As a data scientist , there is meaningful work at companies where data is closely linked to operations, mostly in "new" businesses. As a consultant, I find it interesting to understand legacy businesses and their data, and learn about what potential upside there is, even if I know they won't listen to me.
My impression is that the article mostly describes data analysts, not data scientists, though of course job titles are fuzzy and variable. IME data scientists pretty much universally at least attempt to work on things that directly and demonstrably increase revenue or reduce expenses. That experience is colored by the fact that I work in a field (insurance) where statistical analysis is core to the product, but my impression is that it's true more generally - if you're a data scientist in e.g. conversion optimization at a big company you probably have no trouble justifying your salary, and even the relatively unsuccessful data science teams I'm familiar with tend to work on things like staffing level optimization that are at least plausibly a value-add for the business.
The more interesting line of argument to support data science being a bullshit job is that all of these lucrative use cases are just a tragedy of the commons. Insurers in aggregate did just fine when pricing was based on actuaries using simple approaches to "square the triangle" and then adjust rates based on rules of thumb, the only reason they can't always do that today is that everyone else has data scientists. And if you don't have a good recommender system, your customer will just buy somewhere else (or "worse", avoid an unneeded impulse buy).
Of course, it's not strictly zero-sum. Insurance pricing is "fairer" now, which is probably a good thing, and data science is enabling all sorts of obviously positive-sum products like wearable health monitoring and car safety features (to say nothing of self-driving eventually). Other use cases like "engagement" optimization are probably negative-sum. But in general my point is that creating millions of dollars' worth of business value per year, as many (most?) data scientists do, doesn't preclude you (or me) from having a bullshit job.
The more interesting line of argument to support data science being a bullshit job is that all of these lucrative use cases are just a tragedy of the commons. Insurers in aggregate did just fine when pricing was based on actuaries using simple approaches to "square the triangle" and then adjust rates based on rules of thumb, the only reason they can't always do that today is that everyone else has data scientists. And if you don't have a good recommender system, your customer will just buy somewhere else (or "worse", avoid an unneeded impulse buy).
Of course, it's not strictly zero-sum. Insurance pricing is "fairer" now, which is probably a good thing, and data science is enabling all sorts of obviously positive-sum products like wearable health monitoring and car safety features (to say nothing of self-driving eventually). Other use cases like "engagement" optimization are probably negative-sum. But in general my point is that creating millions of dollars' worth of business value per year, as many (most?) data scientists do, doesn't preclude you (or me) from having a bullshit job.
One of the problems with data science is how black box it is compared to other corporate/software development functions:
For many roles and positions in a company, high and low, you can tell whether the person is doing a good (or at least competent) job. A sales rep can show leads he/she has followed up or converted, and $ made. A coder can point to functionality created and lines written. A CFO can point to deals done, documents prepared, etc.etc.
With data science, sometimes you just have absolutely no idea whether the person is good or not.
"This problem can't be solved". You just don't know whether it's true or not.
"These are amazing insights". Do they have any practical value that can turn into $?
Data science is from some perspectives the most black box part of a modern organization. And so much hype is put into it, so many people paid sky-high salaries in the belief that it is producing something.
Is it usually? Not sure.
For many roles and positions in a company, high and low, you can tell whether the person is doing a good (or at least competent) job. A sales rep can show leads he/she has followed up or converted, and $ made. A coder can point to functionality created and lines written. A CFO can point to deals done, documents prepared, etc.etc.
With data science, sometimes you just have absolutely no idea whether the person is good or not.
"This problem can't be solved". You just don't know whether it's true or not.
"These are amazing insights". Do they have any practical value that can turn into $?
Data science is from some perspectives the most black box part of a modern organization. And so much hype is put into it, so many people paid sky-high salaries in the belief that it is producing something.
Is it usually? Not sure.
> I have tried to make my red flags and categories vague enough that they can apply to almost any organization <...>
Trying to shoehorn something complicated into a set of broad statements does not help to address the underlying issues as they are not specific enough hence not actionable.
What is more "Data science" seems to be an ambiguous definition. Besides it is intrinsically an afterthought as data does not exist until the company collects it. I would also argue it is "a thing" only in large orgs or globally successful software; small companies do not posses datasets worthy of exploration.
Specifics would help as otherwise it feels like we're talking about technical debt.
Trying to shoehorn something complicated into a set of broad statements does not help to address the underlying issues as they are not specific enough hence not actionable.
What is more "Data science" seems to be an ambiguous definition. Besides it is intrinsically an afterthought as data does not exist until the company collects it. I would also argue it is "a thing" only in large orgs or globally successful software; small companies do not posses datasets worthy of exploration.
Specifics would help as otherwise it feels like we're talking about technical debt.
> small companies do not posses datasets worthy of exploration
Or with a cash flow for a permanent position. Tech workers are used to be in companies with healthy gross margins and revenue where they make sense (theoretically at least). But even it tech I would not be surprised if there are teams of DS/SWE people in Uber or Doordash helping their companies to lose money faster.
Or with a cash flow for a permanent position. Tech workers are used to be in companies with healthy gross margins and revenue where they make sense (theoretically at least). But even it tech I would not be surprised if there are teams of DS/SWE people in Uber or Doordash helping their companies to lose money faster.
Agreed - this article felt more like it was trying to appear clever and insightful without actually saying anything of insight. Okay, so data science in many organisations is dysfunctional: many people have already made that claim without resorting to waxing lyrical with historical analogies. What next?
“Data science, often only a step or two removed from dashboard generation”
I’ve been trying to learn data science and machine learning. However, data science as a field seems to have already fallen out of favor.
I’ve been trying to learn data science and machine learning. However, data science as a field seems to have already fallen out of favor.
Data science been criticised for over 10 years now, sometimes rightfully and sometimes not. As far as the article - imagine that someone pointed out a bunch crappy software practices and poor outcomes, they put the term "software development" in the title and never told you that the criticisms only apply to a subset of all software jobs and teams. You would then think that software development has fallen out of favour too.
[deleted]
As far as I can tell from the outside it looks like it's not necessarily falling out of favor but fragmenting into different or more specialized roles. Maybe someone who knows more can correct me.
Except that dashboard generation is exactly what "data scientists" do.
(In the same way that "DevOps" is just a fancy name for a "sysadmin with some light scripting skills".)
(In the same way that "DevOps" is just a fancy name for a "sysadmin with some light scripting skills".)
I'm a DS by title. I don't do any dashboards. Some people on the team collaborate on them occasionally but its generally deciding which metrics are tracked along with creating/maintaining them while the actual dashboard design and creation is done by the frontend team.
The thing is that Data department have essentially swallowed the former analytics departments, and many people who have done business intelligence/business analytics now seem to fall into the data science umbrella.
This is part of the reason the term now refers to different things depending on who you talk to or which team they are a part of. Look at job postings, its extremely difficult to understand what the actual job entails these days.
The thing is that Data department have essentially swallowed the former analytics departments, and many people who have done business intelligence/business analytics now seem to fall into the data science umbrella.
This is part of the reason the term now refers to different things depending on who you talk to or which team they are a part of. Look at job postings, its extremely difficult to understand what the actual job entails these days.
The job title is too broad to generalise in this way. Many data scientists never touch a dashboard at all, for many others it's a very small part of their job.
Regardless of how significant the data science insights are, in the end the way you communicate them is through some sort of dashboard.
No, not always. And "insights" might not even be a part of your job as a data scientist. (I wrote a longer comment about this in the thread, I don't want to repeat it here)
What if you are building a data product? I'm not communicating anything on a lot of the things I'm building, because they are simply components of a larger software program. I haven't touched a dashboard in years.
DevOps is a set of practises that practically involve frequent deployments, fast cycle times, low release overhead, short feedback paths, quick time-to-restore, and so on.
It's an extension of lean manufacturing to software development, where the focus is on continuously pushing tiny packets of functionality to customers (instead of occasionally pushing huge bundles of features), to react quicker to changing demand and mistaken assumptions.
It is in no way "a sysadmin with scripting skills" -- the name for which, is, by the way, just sysadmin. Sysadmins know how to write Perl and shell and Python and have done so since forever.
It's an extension of lean manufacturing to software development, where the focus is on continuously pushing tiny packets of functionality to customers (instead of occasionally pushing huge bundles of features), to react quicker to changing demand and mistaken assumptions.
It is in no way "a sysadmin with scripting skills" -- the name for which, is, by the way, just sysadmin. Sysadmins know how to write Perl and shell and Python and have done so since forever.
I know the sales pitch, spare me.
> ...the name for which, is, by the way, just sysadmin
No, because some time in the past "IT support" (you know, the guys setting up Windows and Wi-Fi routers in your office) got upgraded in name to "sysadmins", and so the old "sysadmins" who knew Unix and how to code some Perl needed a new, untarnished name. Hence, "DevOps".
> ...the name for which, is, by the way, just sysadmin
No, because some time in the past "IT support" (you know, the guys setting up Windows and Wi-Fi routers in your office) got upgraded in name to "sysadmins", and so the old "sysadmins" who knew Unix and how to code some Perl needed a new, untarnished name. Hence, "DevOps".
I once was in a "DevOps" role with the responsibility of getting devs' "works on my machine" code to build in CI.
Fun times trying to get code you didn't write to compile without knowing any context.
Anyways, that sort of turned me off trying to move into programming and so I went back to mechanical engineering after less than a year. Maybe I should have stuck it out longer.
Fun times trying to get code you didn't write to compile without knowing any context.
Anyways, that sort of turned me off trying to move into programming and so I went back to mechanical engineering after less than a year. Maybe I should have stuck it out longer.
You were on the DevOps team! They did what ops did!
Data science is the transformation of a set of data resources into a representation of a domain of activity that has integrity and utility. The curation, cleaning, integration & analysis of data to create pictures of what is going on won't go out of favour until there are no more pictures of this sort to be made!
Why cant software engineers just do the data science role?
morally ambiguous act of consuming
I understand that there are many ways of looking at it, but this seems backwards to me. Consumption does not meaningfully grow the economy; it shrinks it. Once something is consumed, it is used up, gone, no longer available. The notion of consumption being almost some sort of civic duty that appears to be pushed recently (for example, as a rationale for returning to city offices) seems to me like absolute nonsense.
> I understand that there are many ways of looking at it, but this seems backwards to me. Consumption does not meaningfully grow the economy; it shrinks it. Once something is consumed, it is used up, gone, no longer available.
This doesn't seem right to me. If I buy food from the grocery store and eat it, does that shrink the economy? If I enjoy a meal at a restaurant and tip the server, does that shrink the economy? I think both of those actions grow the economy. Both of these actions would grow the country's GDP.
Even though I don't agree with the reasoning, it is interesting. Essentially since a material has been used, and is no longer available, now the whole economy is poorer. There are different ways of measuring wealth, and you seem to be endorsing the amount of materials that an economy has determines its total wealth.
This doesn't seem right to me. If I buy food from the grocery store and eat it, does that shrink the economy? If I enjoy a meal at a restaurant and tip the server, does that shrink the economy? I think both of those actions grow the economy. Both of these actions would grow the country's GDP.
Even though I don't agree with the reasoning, it is interesting. Essentially since a material has been used, and is no longer available, now the whole economy is poorer. There are different ways of measuring wealth, and you seem to be endorsing the amount of materials that an economy has determines its total wealth.
If you buy food and eat it, you have personally used it, perhaps enjoyed it, and consumed it. All the effort, materials etc. that went into making the food and getting it to you - have been 'used up', by you.
Now, insofar as the goal of the economy is people's enjoyment of stuff, it served its goal, which is great. But don't think that by consuming the food you did everyone else (the economy) any good - you didn't. You personally benefitted, which is great, and used up the resources - people's time and effort as well as materials that went into it.
A better illustration would be you commissioning a yacht for yourself, using it recklessly and sinking it (with its value going to 0 - i.e. you consuming all of the value). How did the economy benefit? Engineers that could have worked on better problems, e.g. automating labour, instead spent their time on your yacht. So did workers. So did all the people to procure the materials. And now all of that is at the bottom of the ocean, benefitting nobody. But you had your fun while consuming this product.
GDP is a lousy measure of a success of an economy. The point of an economy is to serve the needs of the people - sometimes it seems that some people believe it is the other way round, and people exist to 'grow the GDP'. If people are happy with consuming less, they can work less - what's so bad about that? Or they can invest instead of consuming - that is, work hard now so that they can work less in the future (or have more/better stuff). It is the investment, production and innovation that grows the economy in a meaningful sense (or getting access to cheaper labour, which does not seem very ethical).
Encouraging consumption under the guise of benefitting the economy is absurd.
Now, insofar as the goal of the economy is people's enjoyment of stuff, it served its goal, which is great. But don't think that by consuming the food you did everyone else (the economy) any good - you didn't. You personally benefitted, which is great, and used up the resources - people's time and effort as well as materials that went into it.
A better illustration would be you commissioning a yacht for yourself, using it recklessly and sinking it (with its value going to 0 - i.e. you consuming all of the value). How did the economy benefit? Engineers that could have worked on better problems, e.g. automating labour, instead spent their time on your yacht. So did workers. So did all the people to procure the materials. And now all of that is at the bottom of the ocean, benefitting nobody. But you had your fun while consuming this product.
GDP is a lousy measure of a success of an economy. The point of an economy is to serve the needs of the people - sometimes it seems that some people believe it is the other way round, and people exist to 'grow the GDP'. If people are happy with consuming less, they can work less - what's so bad about that? Or they can invest instead of consuming - that is, work hard now so that they can work less in the future (or have more/better stuff). It is the investment, production and innovation that grows the economy in a meaningful sense (or getting access to cheaper labour, which does not seem very ethical).
Encouraging consumption under the guise of benefitting the economy is absurd.
Put more succinctly, if people suddenly are just as happy with less consumption (e.g. due to covid changing people's habits and setting them back on the hedonic treadmill), that should be celebrated and encouraged. Overall enjoyment per unit of work has gone up, and it's great! We now have to do less work to be just as happy and satisfied. The newly spare capacity can now go into making our lives better in more lasting ways than immediate consumption! E.g. automation, healthcare improvements, making our habitats more pleasant, etc. Or simply go towards leisure, which is fine too!
"if one is familiar with the basics of economics" seems like a fairly large assumption to me here, what economics is a science now? To me it just looks like a working theory to explain the world and some of its fundamental assumptions (that the world is not zero sum) are dubious at best.
Focusing on this problem we are discussing, how much can you control the effects of whar you consume other than by controlling how much? You might buy vegan patties in whole foods, but other than the fraction of your money that went to the direct purchase of energy or land for agriculture, the money just gets split between roughly the remainder of the economically contributing population. Thus as long as you consume, you only have partial power over it's good-faith nature, the remainder is controlled by how much the entire population (and businesses) act in good faith in general. If you really want to keep your hands clean then I see no way other than not to spend a single dime more than you absolutely need to for your survival. Everything else is morally ambiguous at best.
And this statement that we absolutely need the economy to grow for continued prosperity is preposterous. It's clearly A method of continuing prosperity, and clearly a method that at least works. But it's not proven to be the only method and we better start inventing new ones that don't put all their money on what's basically a glorified Ponzi scheme that everyone's forced to play.
Focusing on this problem we are discussing, how much can you control the effects of whar you consume other than by controlling how much? You might buy vegan patties in whole foods, but other than the fraction of your money that went to the direct purchase of energy or land for agriculture, the money just gets split between roughly the remainder of the economically contributing population. Thus as long as you consume, you only have partial power over it's good-faith nature, the remainder is controlled by how much the entire population (and businesses) act in good faith in general. If you really want to keep your hands clean then I see no way other than not to spend a single dime more than you absolutely need to for your survival. Everything else is morally ambiguous at best.
And this statement that we absolutely need the economy to grow for continued prosperity is preposterous. It's clearly A method of continuing prosperity, and clearly a method that at least works. But it's not proven to be the only method and we better start inventing new ones that don't put all their money on what's basically a glorified Ponzi scheme that everyone's forced to play.
> some of its fundamental assumptions (that the world is not zero sum) are dubious at best.
???
The world is very decidedly not zero sum, you don't need economics to tell you that. Zero-sum games are a particular case.
???
The world is very decidedly not zero sum, you don't need economics to tell you that. Zero-sum games are a particular case.
The economy as a whole is definitely not a zero-sum game, there's plenty of evidence of that.
However, that doesn't in any way invalidate the claim that parts of the economy are in fact zero-sum. How large are those zero-sum parts - 10%? 20%? 40%? 80%? Nobody truly knows. Needless to say, the optimal survival strategy in a world of 80% zero-sum is vastly different than in the 10% world.
Also, although there's no data to back it up, I believe the zero-sum games as a proportion of our society are growing, and quite rapidly at that. It's a very worrying trend.
However, that doesn't in any way invalidate the claim that parts of the economy are in fact zero-sum. How large are those zero-sum parts - 10%? 20%? 40%? 80%? Nobody truly knows. Needless to say, the optimal survival strategy in a world of 80% zero-sum is vastly different than in the 10% world.
Also, although there's no data to back it up, I believe the zero-sum games as a proportion of our society are growing, and quite rapidly at that. It's a very worrying trend.
Can you elaborate how the world is not zero sum? We live in a clearly round planet thats completely isolated except for a daily dose of sunlight. Again this sunlight isn't going to change.
You might argue that it's not zero sum for practical purposes, but to me it looks very decidedly zero sum in many different dimensions (usable land, co2 release, fresh water, many minerals, list goes on).
You might argue that it's not zero sum for practical purposes, but to me it looks very decidedly zero sum in many different dimensions (usable land, co2 release, fresh water, many minerals, list goes on).
onus probandi incumbit ei qui dicit, non ei qui negat. But anyway.
Utility isn't measured in amount of land used or CO2 emitted or minerals extracted. This is not a Starcraft game.
Using land better increases the sum. Making energy production more efficient increases the sum. Zero-sum means that any of my gains is someone else's loss, which is not how any human society has ever worked.
A zero-sum world implies that any form of human cooperation that isn't literally war is impossible, which doesn't empirically seem to be the case.
Utility isn't measured in amount of land used or CO2 emitted or minerals extracted. This is not a Starcraft game.
Using land better increases the sum. Making energy production more efficient increases the sum. Zero-sum means that any of my gains is someone else's loss, which is not how any human society has ever worked.
A zero-sum world implies that any form of human cooperation that isn't literally war is impossible, which doesn't empirically seem to be the case.
Did someone need to downvote me to disagree with my opinion? What kind of person are they?
Now, some points.
Now, some points.
what economics is a science now
how much can you control the effects of whar you consume other than by controlling how much
You might buy vegan patties…
you only have partial power over it's good-faith nature
If you really want to keep your hands clean
more than you absolutely need to for your survival
we absolutely need the economy to grow for continued prosperity
preposterous
It's clearly A method
a glorified Ponzi scheme
Nevertheless, we currently want to be ML people, more or less. It's sexy to download __some__ dataset and develop a new model architecture. It's not really sexy to collect good data - not just any data, but good data.
Now, coming from academia, I think this is just a normal thing. It happens in every discipline in some way. In academic fields that are driven by observational data, data collection is usually a difficult and tedious task. Developing theories, methods and, of course, new results is much more interesting. And rewarding!
Look at economics and Piketty's work: His main contribution was to collect data on wealth. Nobody really cares about his models much. However, for some reason, that data simply did not exist before. And it's now valuable for hundreds of different projects.
As a data scientist, one needs to think about data, first and foremost. One needs to spend most time on collection, on measurement errors, on selection effects, on identification, censoring and all that stuff that nobody likes to think about.
Fitting the next best function approximator to the data is the most fun, but probably not the most valuable thing a data scientist can do.
From that standpoint, I think data scientist is a proper job that is probably needed in every company in the future. However, no coursera course on the latest ML package will make you a valuable data scientists.
I think data scientists who are expert statisticians, survey and experiment designers, causal inference gurus and combine that with proper CS techniques will be valuable.