The science that’s never been cited(nature.com)
nature.com
The science that’s never been cited
https://www.nature.com/articles/d41586-017-08404-0
47 comments
I think you hit an interesting example.
What if there was a significant difference in insulating properties of squirrel tails? That could lead to new ways of making insulating fabrics.
Quite a few of my research papers look like "we investigated reasonable question X. The answer turned out to be boring".
Does that mean I shouldn't have done the work? We didn't know the answer would be boring in advance. Should I not publish? Then someone else might repeat the work.
I think, if anything, science needs to publish more boring negative results, to stop people repeating the same boring experiments and not publishing.
What if there was a significant difference in insulating properties of squirrel tails? That could lead to new ways of making insulating fabrics.
Quite a few of my research papers look like "we investigated reasonable question X. The answer turned out to be boring".
Does that mean I shouldn't have done the work? We didn't know the answer would be boring in advance. Should I not publish? Then someone else might repeat the work.
I think, if anything, science needs to publish more boring negative results, to stop people repeating the same boring experiments and not publishing.
Agreed. The value of seminal papers is in the research they enable... the obscure, uncited branches are the payoff.
To be honest, now that you mentioned this, I'm really curious how warm a squirrel's tail is.
Me too. I discovered this paper: "Feeling the heat: Ground squirrels heat their tails to discourage rattlesnake attack" [0] which seems interesting and sort of relevant. It also turns out to have been cited only once so far. [1]
0. http://www.pnas.org/content/104/36/14177
1. http://www.pnas.org/cgi/crossref-forward-links/104/36/14177
0. http://www.pnas.org/content/104/36/14177
1. http://www.pnas.org/cgi/crossref-forward-links/104/36/14177
Seriously, you should write a paper on it when you learn
Not my field, but this[1] is kind of related. It looks at a certain type of squirrel native to southern Africa that uses its tail as a parasol to provide shade.
[1]: http://www.journals.uchicago.edu/doi/abs/10.1086/physzool.57...
[1]: http://www.journals.uchicago.edu/doi/abs/10.1086/physzool.57...
While I sympathize with your sentiment - indeed much research is done that is incremental, not groundbreaking - your comment does not accurately describe academic research.
Full-time scientist positions, whether in academia or elsewhere, are competitive in the extreme. Even with tenure protections, if your hypothetical researcher does not generate impactful research, he or she could expect to be out of a job in relatively short order. In other words, just publishing a paper is not enough anymore. Scientists are constantly evaluated by their peers, not just in the form of peer review. PIs, in particular, are judged harshly by the amount and quality ("impact") of their work, and the grant money they bring in. (Fun fact: usually about half of all grants just go to the university; the PI only gets to keep, and use, a fraction of it.)
Of course nowadays most PIs do not write the papers themselves. Most of the writing falls on postdocs and graduate students. The PIs are full-time writing grant proposals to fund such work. (Of course anything else, including teaching classes or advising graduate students, goes on top of this full-time job.) As any practicing scientist can tell you, grant committees fall easily for fashion and fad, which in the scientific community are driven by citations.
The scientific process today is dominated by the grant mechanism, which is cyclic by design. A PI, when awarded a grant, funds grad students/postdocs/junior staff to produce research that is documented in the form of publications. These publications are used in turn as justification for a new grant, which will fund the next round of research. Citation-less papers are primarily useless in this regard, as the first evaluation a grant committee will make - often before even considering the scientific justification - is to examine the impact of the published work justifying it, which is usually measured algorithmically (citation number.) (Incidentally, the next step a grant committee will make is examining the reputation of the PI, which is a longer-term reflection of... citations.) It isn't hard to see why this incentivizes incremental work - groundbreaking work is risky and might not pay off; it takes time and resources, which are hard to obtain; and incremental work establishes a larger and more visible presence in the field, which is more likely to be noticed.
To put it simply, in nearly all realistic scenarios - excluding very prestigious (and rare) open-ended fellowships that ask no questions of the researcher - the reason that paper on the insulating properties of squirrel tails would exist is because a grant was awarded funding that research. In other words, because somebody gave a crap about it.
Full-time scientist positions, whether in academia or elsewhere, are competitive in the extreme. Even with tenure protections, if your hypothetical researcher does not generate impactful research, he or she could expect to be out of a job in relatively short order. In other words, just publishing a paper is not enough anymore. Scientists are constantly evaluated by their peers, not just in the form of peer review. PIs, in particular, are judged harshly by the amount and quality ("impact") of their work, and the grant money they bring in. (Fun fact: usually about half of all grants just go to the university; the PI only gets to keep, and use, a fraction of it.)
Of course nowadays most PIs do not write the papers themselves. Most of the writing falls on postdocs and graduate students. The PIs are full-time writing grant proposals to fund such work. (Of course anything else, including teaching classes or advising graduate students, goes on top of this full-time job.) As any practicing scientist can tell you, grant committees fall easily for fashion and fad, which in the scientific community are driven by citations.
The scientific process today is dominated by the grant mechanism, which is cyclic by design. A PI, when awarded a grant, funds grad students/postdocs/junior staff to produce research that is documented in the form of publications. These publications are used in turn as justification for a new grant, which will fund the next round of research. Citation-less papers are primarily useless in this regard, as the first evaluation a grant committee will make - often before even considering the scientific justification - is to examine the impact of the published work justifying it, which is usually measured algorithmically (citation number.) (Incidentally, the next step a grant committee will make is examining the reputation of the PI, which is a longer-term reflection of... citations.) It isn't hard to see why this incentivizes incremental work - groundbreaking work is risky and might not pay off; it takes time and resources, which are hard to obtain; and incremental work establishes a larger and more visible presence in the field, which is more likely to be noticed.
To put it simply, in nearly all realistic scenarios - excluding very prestigious (and rare) open-ended fellowships that ask no questions of the researcher - the reason that paper on the insulating properties of squirrel tails would exist is because a grant was awarded funding that research. In other words, because somebody gave a crap about it.
For some who may be less familiar, PI = Principal Investigator. This is how a receiver of a grant is referred to.
https://en.m.wikipedia.org/wiki/Principal_investigator
https://en.m.wikipedia.org/wiki/Principal_investigator
Finally, and thank you!
The squirrel cares.
The squirrel is pretty unlikely to care about the relative warmth of its tail vs other tails.
I tried to ask my mother once what women typically considered to be the dimensions of female attractiveness (e.g. breast size is something people commonly care about, while toe length isn't). Instead I got an answer consisting of a list of things a woman trying to be attractive would do before appearing in a public place. I asked whether facial features weren't considered a big part of attractiveness, and the reply was that yes, they are, but women devote almost no time to their facial features because they are basically impossible to change.
SImilarly, a squirrel may be too cold, or too hot, but it can't really change its tail.
I tried to ask my mother once what women typically considered to be the dimensions of female attractiveness (e.g. breast size is something people commonly care about, while toe length isn't). Instead I got an answer consisting of a list of things a woman trying to be attractive would do before appearing in a public place. I asked whether facial features weren't considered a big part of attractiveness, and the reply was that yes, they are, but women devote almost no time to their facial features because they are basically impossible to change.
SImilarly, a squirrel may be too cold, or too hot, but it can't really change its tail.
besides makeup, you mean?
remember, you're not looking at faces through an accurate depth camera, you're inferring facial geometry through color vision.
mascara draws attention to the eyes, away from the chin. eye shadow provides the illusion of depth, which, combined with subtle application of blush or contouring foundation, totally changes the shape you see.
you're really only inferring an oval from a small handful of points. you can control those points by accenting and concealing.
not to mention hairstyles! bangs! glasses! rhinoplasty!
but it's a time and money sink, and we're increasingly okay with our faces as they are.
remember, you're not looking at faces through an accurate depth camera, you're inferring facial geometry through color vision.
mascara draws attention to the eyes, away from the chin. eye shadow provides the illusion of depth, which, combined with subtle application of blush or contouring foundation, totally changes the shape you see.
you're really only inferring an oval from a small handful of points. you can control those points by accenting and concealing.
not to mention hairstyles! bangs! glasses! rhinoplasty!
but it's a time and money sink, and we're increasingly okay with our faces as they are.
You get to look at someone's face from many different angles while it moves around. The "accurate depth camera" model is more accurate than the "it's an optical illusion" model.
Surgery is indeed a way to alter your facial features, but it's more intrusive than most.
Surgery is indeed a way to alter your facial features, but it's more intrusive than most.
They aren't in the target audience. Squirrels can't read, plus even if they did the squirrel university library didn't pay $10k for the subscription to access to the journal the paper is published in.
Even if the squirrel was able to access the article, the follow up research the squirrel does won't be indexed by Google so I can't add to my H-index (my H-index is a whopping 1 btw), so none of it really matters.
Even if the squirrel was able to access the article, the follow up research the squirrel does won't be indexed by Google so I can't add to my H-index (my H-index is a whopping 1 btw), so none of it really matters.
True, especially when you consider self-citations or citations that are basically a political back-scratch between collaborators.
I wonder if part of the reason for uncited papers being so low is that authors cite their own papers in subsequent papers. For example, an author is studying field X, and publishes papers. No one cites the first paper. The author then writes a new paper on X, but cites the first paper. Thus, although no one but the author cited the papers that the author wrote, all but the last one have citations in other papers.
Another way this works is that authors in the same department may cite each other's papers especially if the fields are somewhat related.
I wonder what the data would look like if they excluded citations by the authors of the original paper.
Another way this works is that authors in the same department may cite each other's papers especially if the fields are somewhat related.
I wonder what the data would look like if they excluded citations by the authors of the original paper.
Exactemente. It's fairly bizarre that this article didn't mention self-citation, huh.
I sure hope that the study corrects for self-citations. It’s a pretty simple procedure with Web of Science.
That said, who you cite can be a fairly political decision, it’s like who to follow on Twitter, at least in fields with multiple people working on the same topic.
That said, who you cite can be a fairly political decision, it’s like who to follow on Twitter, at least in fields with multiple people working on the same topic.
In the methods section, they do say that "first author self-citations were excluded." One of the figures in the Nature article also pulls out self-citations.
https://arxiv.org/abs/0809.5250
https://arxiv.org/abs/0809.5250
Do note however that many times the first author is not the principal investigator (ie the person that has the funding for the study and is often tenure track faculty) but is often a graduate student or a post-doc. The principal investigator may be in the middle or even at the end of the author list. Thus just excluding first author self-citations is probably inadequate. The chances are high that a grad student or post-doc is going to cite their adviser's previous papers.
The rule to read the author list, simplified: first author did the work. Last author funded the research. Everyone else contributed somewhat.
That’s only true in some areas (e.g. some biological fields). In other areas (e.g astronomy or much of physics), it’s decreasing order of contribution. In mathematics, the author list is usually alphabetical.
It is also traditional that the first author, in addition to doing the most of the scientific work, takes on most of the writing He/She also edits together the various other authors contributions.
One metric used to evaluate publication history is the combined count of first and last authorships for this very reason.
[deleted]
So if a pair of researchers work on a set of papers, all they have to do is alternate the first author position? Then they never self-cite even if they cite their own work 10 times in a row.
Yes, but for hiring or tenure the researchers are always being evaluated by others in the field, so there would be a human sanity check.
Plus in practice as others have alluded to it's usually Prof. Alice's lab, where grad students Bob, Cindy, Dave and Evelyn each pursue their own corner of the research, and they're all on each paper with the first author designated appropriately.
Plus in practice as others have alluded to it's usually Prof. Alice's lab, where grad students Bob, Cindy, Dave and Evelyn each pursue their own corner of the research, and they're all on each paper with the first author designated appropriately.
[deleted]
> “Lack of citation cannot be interpreted as meaning articles are useless or valueless,” says David Pendlebury, a senior citations analyst at Clarivate.
On the other hand, presence of citation can not be interpreted as meaning articles are useful or valuable. For one, there could be chains of useless and valueless papers citing each other.
On the other hand, presence of citation can not be interpreted as meaning articles are useful or valuable. For one, there could be chains of useless and valueless papers citing each other.
Striking similarity to PageRank and Link Farms.
PageRank was initially invented for scientific publications and later applied to web sites.
Wow, that’s amazing to know - thanks for sharing!!
It seems to me, based to some extent on my own experience, that a paper has a harder time getting cited when it's a bit off the beaten track (and might in fact be more original, trail-blazing and interesting), because by definition it's a bit lonely in its field. Conversely, when the work is fairly derivative it gets lumped in with others and is easier to cite as part of the related work section of another paper, even if its influence is minimal.
Some papers are also just very useful as citations. My least interesting paper is my most cited for this very reason.
I was looking into a CS academic who claimed to be widely cited. Mostly it was self-citation, in his own papers. He seemed to be the editor of a journal which published a lot of his own papers citing himself.
I couldn't help thinking of that looking at the graph of decreasing uncited papers.
I couldn't help thinking of that looking at the graph of decreasing uncited papers.
If he actually has a journal and does that, he's nothing more than a con artist.
Hehe yes, that's the general conclusion. I'm not sure if I should name names. (Ah well) The guy I'm talking about is Azlan Iqbal, a Malaysian CS professor. I know him as the writer of numerous wacky articles on ChessBase (leading chess news site) about his chess+computer studies. People criticize his shoddy work in the comments, and he says, nay, jeers they're not qualified to judge because they didn't read his uni thesis, dont know CS, dont understand his algorithm, etc. Kind of amazing. I guess he can't be an impostor in CS, but most of his published articles are about computers + chess, with the Dunning-Kruger effect in full flight. I guess you have to know a bit about chess to fully realize how inept his stuff is. But just the embarrassing way he attacks all critics in the comments is remarkable. Although the CB readers largely ignore him now. See the stories with most comments to see what I mean. [0]
Are there a lot of people like this? Or worse?
[0] https://en.chessbase.com/author/azlan-iqbal
Are there a lot of people like this? Or worse?
[0] https://en.chessbase.com/author/azlan-iqbal
> Are there a lot of people like this? Or worse?
This guy is something else... I have honestly never seen anything like this before.
Most of his journal papers [1] are not even close to standard journal paper quality. His latest one, titled "A Simple Encryption Method for FTP Passwords", is a two-page document showing how to use a substitution cipher to encrypt FTP passwords... That wouldn't even pass off as a project for an undergrad CS course!
The second journal paper seems acceptable, but I am also not familiar with chess algorithms, so I cannot provide a definite opinion.
Some amazing quotes from [2]:
"Let me begin by saying that the aesthetics model in chess that I developed for my Ph.D. is considered a seminal piece of work in then uncharted waters."
"While I have written many papers, to the extent memory serves, I do not self-publish at all even though admittedly, some of my publications are certainly better or more prestigious than others (like any academic)." (note: you should never self-publish, so there really is no need to "consult" your memory)
"In fact, I am quite confident I could last at least 20 moves even against Magnus Carlsen under tournament conditions."
"[...] I simply do not need to be a chess master as there are many official chess masters only too happy to assist and work with me on projects. I am frankly quite amazed at how open-minded and forward-thinking some of them are."
[1] http://metalab.uniten.edu.my/%7Eazlan/
[2]: https://en.chessbase.com/post/women-and-beautiful-chess-a-re...
This guy is something else... I have honestly never seen anything like this before.
Most of his journal papers [1] are not even close to standard journal paper quality. His latest one, titled "A Simple Encryption Method for FTP Passwords", is a two-page document showing how to use a substitution cipher to encrypt FTP passwords... That wouldn't even pass off as a project for an undergrad CS course!
The second journal paper seems acceptable, but I am also not familiar with chess algorithms, so I cannot provide a definite opinion.
Some amazing quotes from [2]:
"Let me begin by saying that the aesthetics model in chess that I developed for my Ph.D. is considered a seminal piece of work in then uncharted waters."
"While I have written many papers, to the extent memory serves, I do not self-publish at all even though admittedly, some of my publications are certainly better or more prestigious than others (like any academic)." (note: you should never self-publish, so there really is no need to "consult" your memory)
"In fact, I am quite confident I could last at least 20 moves even against Magnus Carlsen under tournament conditions."
"[...] I simply do not need to be a chess master as there are many official chess masters only too happy to assist and work with me on projects. I am frankly quite amazed at how open-minded and forward-thinking some of them are."
[1] http://metalab.uniten.edu.my/%7Eazlan/
[2]: https://en.chessbase.com/post/women-and-beautiful-chess-a-re...
Hehe OK I'm glad you 'enjoyed' that, glad I mentioned it. Yeah, the non-stop bragging and credentials-showing hehe. Oh you didn't get onto the 'good' parts of the women and chess one. Not sure which was 'best', the cringeingly sexist language or the incredibly inept experiment design...
Well, people in the comments (CB comments have never been half so impressive) accurately ripped his work to shreds, not much for me to add. OK, maybe he's somewhat a CS impostor too. :-) It's Dunning-Kruger it its most virulent. Well, it's CB's fault for publishing him there too; people beg them not to, each time they do. I guess no other academics send their stuff there.
Well, people in the comments (CB comments have never been half so impressive) accurately ripped his work to shreds, not much for me to add. OK, maybe he's somewhat a CS impostor too. :-) It's Dunning-Kruger it its most virulent. Well, it's CB's fault for publishing him there too; people beg them not to, each time they do. I guess no other academics send their stuff there.
> people like this
Boris Stilman [0], but Stilman is not worse than Iqbal.
[0] http://www.ucdenver.edu/academics/colleges/Engineering/resea...
Boris Stilman [0], but Stilman is not worse than Iqbal.
[0] http://www.ucdenver.edu/academics/colleges/Engineering/resea...
> Then, checking on Google Scholar, Heneberg saw that many of the remaining papers actually had been referenced by other works indexed in the Web of Science, but had been missed because of data-entry errors or typos in the papers
I wonder if you could look at the number of citations to non-existent papers to find these typos. If you collected such a set, you could do a loose search against the list of published papers, and see how many are a close match. For instance, a typo in an author's name or the paper's title could be ignored while matching the journal, date and page could give the auditors a list of best guesses for this missed citations.
I wonder if you could look at the number of citations to non-existent papers to find these typos. If you collected such a set, you could do a loose search against the list of published papers, and see how many are a close match. For instance, a typo in an author's name or the paper's title could be ignored while matching the journal, date and page could give the auditors a list of best guesses for this missed citations.
I wonder if citation research could become self sustaining, if it generates enough cited papers.
I like how the 3dB point for engineering papers is 4 years (that is where 50% of the papers remain uncited). I am a bit surprised it goes down from there. I have always felt that a big chunk of the engineering or technical papers out there reference technology that is obsolete after 5 years[1]. There are the seminal papers of course, the ones that it seems are always cited, but a huge chunk which were useful when published, are much less so later.
[1] No I don't have a citation :-) but having written a few papers and looking for related work I found the chance of finding something relevant dropped precipitously when you got into papers that were more than a few years old.
[1] No I don't have a citation :-) but having written a few papers and looking for related work I found the chance of finding something relevant dropped precipitously when you got into papers that were more than a few years old.
The reason for this isn't really surprising: Most research isn't that groundbreaking. Most research is done because if you don't produce a paper every now and then or else the university will fire you. So you crank out that study you did on the insulating properties of the grey squirrel's tail vs the red squirrel's tail and you keep your job, although nobody gives a crap about how warm a squirrel's tail is.