Kudos to FiveThirtyEight on being transparent and analyzing what happened. But also...this was a series of mistakes, some of them pretty scary.
FiveThirtyEight's biggest mistake seems to be trusting an academic dataset when they had no idea how it was collected. This is understandable, especially when the data was published on the Arizona State University's Center for Policy Informatics data portal. (You can go there right now and download the bad data - scroll to CATALIST DATA here https://policyinformatics.asu.edu/broadband-data-portal/data...) A university should be a trusted source. But FiveThirtyEight took an unbelievable outlier from this dataset and wrote an entire post about it (https://fivethirtyeight.com/features/lots-of-people-in-citie...). The dataset claims that only 29% of Washington D.C.'s adults have broadband. (The real number according to the other datasets FiveThirtyEight looked at in the new post is closer to 70%.) They even make a point of how extreme the Washington D.C. datapoint is on the histogram in the article as the only large county with such a low percent. That should be a clue to question your data.
What I find worse is that the academic researchers published this dataset. They bought behavioral marketing data and trusted a salesperson that the variable HTIA (“Denotes interest in ‘high tech’ products and/or services as reported via Share Force. This would include personal computers and internet service providers. Blended with modeled data.”) was a good proxy for broadband access. To be clear, HTIA includes modeled data, which means they took demographics, voting records, and whatever other individual data they could grab (maybe they have records of your purchases, I'm just guessing), and predicted whether each adult in the US was interested in tech. This is the kind of data companies buy for ad campaigns, figuring that if they advertise to these adults, it might be better than random. There's no reason to think the aggregates of these numbers would be accurate or calibrated correctly, especially for an entirely different purpose (broadband vs high tech).
It's disturbing that these sort of datasets are floating out there in academia and really makes you wonder what other bad data is being blindly trusted to write blog posts, research papers, and news articles.
FiveThirtyEight's biggest mistake seems to be trusting an academic dataset when they had no idea how it was collected. This is understandable, especially when the data was published on the Arizona State University's Center for Policy Informatics data portal. (You can go there right now and download the bad data - scroll to CATALIST DATA here https://policyinformatics.asu.edu/broadband-data-portal/data...) A university should be a trusted source. But FiveThirtyEight took an unbelievable outlier from this dataset and wrote an entire post about it (https://fivethirtyeight.com/features/lots-of-people-in-citie...). The dataset claims that only 29% of Washington D.C.'s adults have broadband. (The real number according to the other datasets FiveThirtyEight looked at in the new post is closer to 70%.) They even make a point of how extreme the Washington D.C. datapoint is on the histogram in the article as the only large county with such a low percent. That should be a clue to question your data.
What I find worse is that the academic researchers published this dataset. They bought behavioral marketing data and trusted a salesperson that the variable HTIA (“Denotes interest in ‘high tech’ products and/or services as reported via Share Force. This would include personal computers and internet service providers. Blended with modeled data.”) was a good proxy for broadband access. To be clear, HTIA includes modeled data, which means they took demographics, voting records, and whatever other individual data they could grab (maybe they have records of your purchases, I'm just guessing), and predicted whether each adult in the US was interested in tech. This is the kind of data companies buy for ad campaigns, figuring that if they advertise to these adults, it might be better than random. There's no reason to think the aggregates of these numbers would be accurate or calibrated correctly, especially for an entirely different purpose (broadband vs high tech).
It's disturbing that these sort of datasets are floating out there in academia and really makes you wonder what other bad data is being blindly trusted to write blog posts, research papers, and news articles.