This was an extremely interesting paper to me, about a topic that I see as economically and sociologically fundamental.
I was actually impressed by the methods they used. I found myself thinking "this is what I'd really like to see," and then they'd report it. Validating their method on the MusicLab data seemed critical to me, as did examining reddit resubmissions versus YouTube views.
Although I thought methodologically it was almost as well done as it could have been outside of an experiment, I disagreed with the author's conclusions. They acknowledge some of the problems, such as the problem of the huge number of forgotten posts they didn't model at all, but other issues they don't.
For example, it seems the question of most interest is, given an observed post score, what's the actual "quality"? If you look at, say, Figure 3, it's apparent that there's huge variability in quality conditional on score, as observed score increases.
I think the correlational-style relationship they focus on obscures things like this that are critical to interpreting the findings. Yes, there's a strong estimated relationship between quality and score, if you ignore all the missing data that constitutes the bulk of submissions, and the fact that the relationship is being driven very strongly by a large quantity of very low-"quality" posts versus everything else, and the variability everywhere else. It's an odd, heteroscedastic, nonlinear relationship that isn't well-captured by a correlation, even a nonparametric one.
I also would have liked to see examination of variability in links across sites. How much variability is there in rank of an initial link, to the same material, across reddit, HN, Twitter, etc.? Maybe tellingly, the authors report the relationship between YouTube views and number of reddit submissions, but not the relationship (if I'm reading correctly) between YouTube views and rank of initial reddit submissions, which is kind of the key relationship.
So, liked the paper but if anything it just reconfirms the conclusions of earlier studies to me, that social network dynamics has a big influence on apparent popularity.
There's kind of two issues at least. One is the continuous-discrete issue and the other is the moment issue.
As for the moment issue, the short story is that as you get into three or four moments, there isn't a general maximum entropy distribution anymore, except for some special idiosyncratic cases in the case of three I think. So the normal is, in some ways, the most conservative distribution you can have in a general, unspecified scenario sense. You can specify more moments, but then there isn't a single maxent distribution you can specify that would apply across all third and fourth-moment scenarios in the same way that would apply for the first two moments.
As for the continuous versus discrete thing, there's some caution that's warranted, but a lot of the maxent principles apply, and there are similar, closely related principles (minimum description length, which has been shown to be equivalent to maximum entropy inferentially in a sense) that generalize in the continuous case. If you think of everything as discretized (as is the case with machine representation), there's some work showing that the discretized and continuous cases are sort of related up to a constant (doi: 10.1109/TIT.2004.836702).
I realize this is a bit hand-wavy but it is a HN post.
This is such an unrecognized part of the problem: overcredentialing. It's rampant: it's explicit in areas like healthcare and law, and implicit in the HR practices of many corporations.
We bitch about people going into debt, but turn around an are fine with companies being picky as hell about having a specific degree, as if that's everything about a person's ability or background. We also bitch about healthcare costs, but then act like the sky will fall if we start discussing the possibility of pharmacists, optometrists, or psychologists prescribing or offering more services. Your observation about law is equally astute.
I'm going to beat a dead horse until it rises from the grave, but this is the situation with liberal arts degrees as well: they're from a time when it was assumed that you could major in, say, philosophy, and take comp sci classes, do work in that area, and build up a career in comp sci without anyone questioning it. Now your local HR department uses that comp sci degree to screen you, as if you are your degree.
Everyone knows that these degrees are helpful but imperfect indicators, but we treat them as perfect indicators because it's easier to maintain the myth, and it benefits those who benefit from rent-seeking and overregulation.
There are some recent studies suggesting that globally, on average, the most environmentally friendly diets have some animal product component because of their ability to make use of landmass that we wouldn't be able to make use of directly. E.g., cattle can eat plants that grow in areas we can't grow human-friendly crops on.
I can't find citations to these studies offhand unfortunately. (this is an example but not what I had in mind: www.ncbi.nlm.nih.gov/pmc/articles/PMC5522483/) But what I remember is that globally, the most environmentally friendly diets had some small animal product component.
This also probably varies a lot by location too, so it probably is the case that for some people, the best diet might be vegan; for others it might involve more animal product.
Just pointing this out, because there are nonpolitical-psychological-sociological reasons for a "soft" approach on plant-based-diets.
I agree with a tiny caveat, in that I'd change Jeffreys prior to reference prior.
On the other hand, these priors can be difficult to create in some (many?) situations and it's often more tractable to do ML.
Bayesian inference seems more principled to me in general if you allow for and use reference priors, but outside of that I think there are still reasons to prefer ML. There's two areas where I still have problems with priors.
The first is that the sequential testing paradigm (that is, prior -> posterior -> prior) doesn't always work in reality because you often have multiple experimenters operating simultaneously and independently with different priors. In one sense this is a trivial problem but in another sense it is not. E.g., if you are a meta-analyst faced with integrating such results, is prior variation akin to publication bias? What implications does that have?
The second is that there are situations in which using a prior actually might lead to unfair inequities. For example, let's say you're trying to make some inference about an individual, and know that ethnicity provides information in a statistical sense about the parameter you are making an inference about. Is it prejudicial or not to use a prior? I think using a reference prior would address this situation, but depending on the scenario you could make an argument that it is unfair (e.g., if the informative prior would suggest a positive outcome, not using it might be seen as prejudicial, but if the informative prior would suggest a negative outcome, using it might be seen as unfair). In this case, not using a prior at all actually might make sense--you might make a similar argument about non-Bayesian inference as Bayesian reference inference, but using non-prior-based inference does sidestep the issue in a sense, in that there is no longer a prior to decide about. This might be especially important in that, e.g., if you have a series of individuals, the act of choosing a prior might be seen as prejudicial in itself.
I generally consider myself as an "objective Bayesian" in the Jaynesian / reference prior sense, but there are practical and theoretical scenarios where I think people are likely to run into problems.
With the birth of our daughter, we had a different, and more complicated experience.
Pre-delivery there was no pressure to do a c-section. None at all. My wife definitely did not want one.
At the time of delivery, though, there was very much pressure to do a c-section. Although the admitting resident didn't seem to pressure my wife, care was quickly transferred (because of shift reasons) to other physicians (resident and attending) who did.
The way this manifested, though, was sort of subtle. For example, my wife had a procedure done to speed up the delivery; however, as we found out later, we were definitely not sufficiently informed of the consequences of the procedure, one of which was increased likelihood of a c-section. They tried to talk my wife into a c-section, and then when she declined, they tried to talk her into other procedures that would speed up the delivery, and would omit mention of the fact that they were associated with increased likelihood of c-section. Overall, even if c-sections weren't being explicitly mentioned, they were kind of relied on or assumed, for time and convenience reasons. The discussion was sort of like "Oh you don't want a c-section? Ok, then how about X to speed things up? Oh--I forgot to mention that now we probably have to do a c-section? Oops!" It came across as manipulative to me.
My wife did not have a c-section, but this was probably only because the nurses there (who were phenomenal) were actively arguing with the physicians to not do one, and to wait. We weren't really in the hospital that long either.
Am I missing something or are parts of this article really distorted?
For example, this seems to set up most of the article:
"Economics involves a lot of math and statistics. The most commonly used tools to crunch numbers are the spreadsheet software Microsoft Excel and programming languages Stata and Mathematica."
Is this really true? Mathematica and Stata seem like established but niche products to me at this point. I wouldn't say either of them are "the most commonly used tools to crunch numbers."
If you asked me to predict what a quantitative economist would be using, it would be Python, followed by R, and maybe followed by Java or C, or something like that.
This was an interesting article in the sense I like learning these sorts of things about people, but the premise seemed off to me.
But I'm not an economist so maybe this is something about economics per se.
Not sure if I'm left-leaning or not, but the problem is that we need increased competition.
The reason why this is a problem is that sometimes it means decreased regulation, and sometimes it means providing more government services and more regulation. But that doesn't really fit well into the two major political parties.
For example, I'd probably seen as even more radical about healthcare than what you're writing--I'd advocate eliminating licensure laws and radically reorient the mission of the FDA in part by removing its regulatory authority over a lot of things it currently.
But I also advocate sharply reducing patents and copyright terms, and rolling out federal and municipal broadband. I also think public education needs a lot more funding.
It seems like political discussions in the US become oriented around protecting entrenched business interests, or protecting citizens through increased regulation.
As someone who does research in this area, broadly defined, I think you're on to something, but I also think there are some misleading things about this article (which I nevertheless think is interesting) and caveats to what you're saying.
Lots of thoughts:
1. Intelligence is a broad construct. It is by definition, and it is not the only cognitive construct. It does have a lot of utility for certain purposes though, such as in identifying pervasive neurological disease.
As others are noting, this is relevant to the article in that we tend to focus on extremes when making these kinds of comparisons, when the full spectrum is really what's important sometimes. We tend to fixate on whether someone went to some prestigious university or less prestigious university, or whether our incomes are in the upper middle class or upper class, but in the sense of outcomes, compared to all outcomes, these can be relatively minor distinctions and hard to predict.
2. There are other variables that are relevant, like conscientiousness, ruthlessness, and so forth. This is certainly true.
3. There are still other variables that have nothing to do with the individuals involved though. The elephant in the room are societal and other random factors that prevent any individual attribute from mattering as much as they could. The article starts out by dismissing prediction among females out of hand because of societal limitations, which is reasonable. But there are lots of other variables involved, random and nonrandom societal and environmental forces at play. The hidden story is that there are limits to predicting outcomes at all from the individual at hand, meaning that other variables in the environment are working.
4. Measurement of intelligence is fuzzy and imperfect as you're alluding to. It's stochastically imprecise, in the sense that giving the same test twice, or two different tests, will give you somewhat different answers. But it's also imperfect in that the thing it's measuring isn't really what we probably want to measure in an ideal case. Even if the tests were giving the same answer all the time, it wouldn't really be intelligence in the way we want to talk about intelligence.
5. I'm not sure that we really want cognitive functioning measures to be perfectly stable, because I don't think cognitive functioning is actually perfectly stable. It probably varies across the day, for example.
6. Physical measurements are certainly more precise. But the objects systemically are much less complex. It's easier to talk about measuring the mass of a cubic meter of oxygen than it is to talk about measuring climatological variables; something analogous is in play with things like intelligence.
Also, even physical measurements at a certain level become fuzzy and highly interdependent. Measuring mass "precisely" depends on your scale and other variables.
My same concern about the ethics of this. I might have felt a little differently if this AI startup had paid for all of the data collection proactively (although I would have still had concerns about the exclusivity of any such agreements to patient access), but as it is this seems unethical.
The biomedical-industrial complex in the US makes my stomach churn. So many conflicts of interest, rent-seeking, monopolies, and nepotism.
It's been said before but needs to be said again: this is happening everywhere in the biomedical and related fields. The neurosciences, oncology... the list goes on and on. Anyone take a look at AI research lately? How much tweaking is going on there? How much is your big data finding due to the idiosyncracies of your particular dataset?
Psychology, as has historically always been the case--meta-analysis itself bloomed largely from the field--is the one turning inward and looking at itself. And it's getting crap from people who love to use it as their favorite punching bag.
The irony of this article is that it's psychologists looking at other psychologists, doing the math etc.
The truth is closer to the second hypothesis by the author: bullshit is incentivized everywhere in academics. Reality is less interesting, harder, more incremental. Everyone wants the next genius savior to point to because it's a simpler story than reality. Sexy means more pubs, more grant money.
I think the issues go deeper than the degree program per se. Often the humanities are part of a liberal arts degree, which differs from other degrees not just in the major subject, but also in that it is broad in focus. Where I went to school, for example, you could get a math major from a college of science and engineering, or from the college of liberal arts and sciences. The difference was largely in terms of how much of the coursework was in major versus out of major.
I bring this up because often liberal arts degrees are predicated on an assumption that a person will be seen as more than their degree, even in the labor market. That is, someone who can complete a liberal arts degree in philosophy with good grades and the right extracurricular experiences, who has taken a lot of the right coursework, can go on to get a master's in computer science, or biochemistry, or a law degree, or MD, or learn the right skills from their employer.
What we have now is a problem where too much focus is put on certification. Employers (or their HR departments?) see a degree as a certification to do a particular skill. They don't want to try to surmise these skills from other experiences, or to train new employees in those skills, they want a box checked that says "this person can do A."
This I think is the source of this trend more than anything. It's an equating of degree major with skillset, or ability, or whatnot.
When you live in a society where having X degree is required, either by employer hiring practices or by law, as a certification of being able to do A, B, or C task--even if you do not actually need that particular degree to do those tasks--you are inevitably going to see everyone want to have X degree. When you create an economic environment driven by rent seeking and regulatory capture of one sort or another, you're going to see people try to position themselves accordingly.
I think the rise in interests in MOOCs, etc., and criticism of traditional educational structures in part is a response to this overcredentialing. The irony is that the liberal arts degree, which is supposed to be a kind of happy medium, kind of has been squeezed out from both sides of that argument.
I'm skeptical of citation rate as well, although I'm not sure I have anything better to offer in terms of metrics. Hype, after all, is related to citation rate. I think what we need is something that is more like "sustained rate" or something like that.
It's pretty clear China is progressing in the sciences substantially, so I don't want to discount that--no one should--but this progression is occurring in what I consider to be a crisis of academic integrity globally. The result is that metrics like publication rate and citation rate are much fuzzier to interpret, and something I distrust a lot because they are somewhat meaningless relative to replicability or something of that sort.
I generally feel like academics and industry at large is suffering from a kind of hype crisis or bubble. I think it's strongly related to income inequality (inflated attributed value of higher-income individuals relative to lower-income individuals) and all sorts of other societal problems at the moment. How this relates to China I'm not sure but overall it makes me skeptical of any attempt to measure or rank countries relative to one another (I'd say the same thing about the US or any other country for that matter).
I have mixed reactions to pieces like this. It's very salient to me because I fit the stereotype being discussed. At 43, I did my first triathlon and am, I suppose, training to do more (the season for them is basically over where I live, so I can't really do more this year).
The problem with the idea of the midlife "crisis" is that it's more a period of change and not all crisis. And it's somewhat, but not all, about mortality. At that age, in your 40s, you've often had enough time to establish a career, and for many, realize it's not what you thought it would be. It has nothing to do with mortality; if you started your career earlier or later, you'd be reaching that point earlier or later. Also, you've reached a point of wisdom to realize, yes, exercise is good, maintaining your health is good, all these articles you've been reading about it for years are right, and so forth and so on. You might have a new family, which changes things at any age. And finally, speaking of kids, I do think there's something about the 40s being the new 20s, the 60s being the new 40s, and so forth. I could go on and on about many things that cause change, but don't have anything to do with death or even old age.
If you knew me, for example it would be clear that I am in fact in crisis in many ways, and have been in recent years, in the ways that the author suggests. But that's not what motivated me to do a triathlon. It has nothing to do with some need to prove myself or anything like that. It was all about being prodded by friends to join them, and it was something I had always been interested in. I've always been a little athletic. It could have happened in my 20s or 30s but didn't. Why my 40s? I'm not sure. I guess I've just reached a point where there are many things like triathlon I've been wanting to do for years, and am now getting around to it. When you reach that age there are things that accumulate like that.
I think midlife change is very real, and often comes as a crisis, but I think that focusing on the mortality issues that do arise is really missing many, if not most, of the other factors involved.
In the case of athleticism, it stigmatizes exercising and wellness. So you're doing a triathlon or a marathon in your 40s? Now this is a bad thing? Can those of us in our 40s get a break? It's another form of ageism in many regards.
I was actually impressed by the methods they used. I found myself thinking "this is what I'd really like to see," and then they'd report it. Validating their method on the MusicLab data seemed critical to me, as did examining reddit resubmissions versus YouTube views.
Although I thought methodologically it was almost as well done as it could have been outside of an experiment, I disagreed with the author's conclusions. They acknowledge some of the problems, such as the problem of the huge number of forgotten posts they didn't model at all, but other issues they don't.
For example, it seems the question of most interest is, given an observed post score, what's the actual "quality"? If you look at, say, Figure 3, it's apparent that there's huge variability in quality conditional on score, as observed score increases.
I think the correlational-style relationship they focus on obscures things like this that are critical to interpreting the findings. Yes, there's a strong estimated relationship between quality and score, if you ignore all the missing data that constitutes the bulk of submissions, and the fact that the relationship is being driven very strongly by a large quantity of very low-"quality" posts versus everything else, and the variability everywhere else. It's an odd, heteroscedastic, nonlinear relationship that isn't well-captured by a correlation, even a nonparametric one.
I also would have liked to see examination of variability in links across sites. How much variability is there in rank of an initial link, to the same material, across reddit, HN, Twitter, etc.? Maybe tellingly, the authors report the relationship between YouTube views and number of reddit submissions, but not the relationship (if I'm reading correctly) between YouTube views and rank of initial reddit submissions, which is kind of the key relationship.
So, liked the paper but if anything it just reconfirms the conclusions of earlier studies to me, that social network dynamics has a big influence on apparent popularity.