I would argue having significant part of a society whether right or wrong in anyone's view having their thoughts not disclosed and represented to the rest of the society should be worrying for everyone in that society.
The history of our nature has shown self-censorship usually erupts by the most radicals of that group which is an uglier problem to have I would think than allowing their right or wrong thoughts to be out in the light.
>The other factor is the rise of internet social media, which didn't exist back then.
This to me only highlights the extend to which we should be worried rather than exaggerating it.
>I'm not sure that it's scary. After all, casual racism and sexism used to be a lot more common in the past. Now people are called out and criticized on those. Is that bad?
Isn't it scary or least worrying that 40 percent of Americans reported that they did not feel free to speak their minds? Or are you implying they belong to the group who participated in casual racism and sexism that used to be a lot more common in the past and now people (majority of the other 60%) are calling them out and criticize them on those hence concluding not scary and perhaps good?