Talks That Attendees See in Person More Likely to Be Cited(nature.com)
nature.com
Talks That Attendees See in Person More Likely to Be Cited
https://www.nature.com/articles/d41586-023-01604-x
47 comments
This is why it is dangerous to think that the best scientific theories always prevail and that science is a meritocracy. The people invited to give talks are usually well known or have connections with the program committees. There is also an element of bias since the people on the organizing and program committees will invite speakers whose research is consistent with their own work.
I don't think anyone with even the slightest awareness of scientific philosophy or history actually thinks that though. Everyone knows that there are social factors involved to varying degrees in different fields, and that paradigms and research programs often only shift when the old guard in a field dies. The scientific method is a directional force guiding this messy social enterprise in the general direction of truth but if anyone thinks science as an institution is purely meritocratic they haven't thought about it hard enough or done the required reading.
>I don't think anyone with even the slightest awareness of scientific philosophy or history actually thinks that though.
I don't know, this whole 'marketplace of ideas' thing is still very popular in politics and media (perhaps not academia; but what does academia count for in a neoliberal world?)
I don't know, this whole 'marketplace of ideas' thing is still very popular in politics and media (perhaps not academia; but what does academia count for in a neoliberal world?)
Not true in any field I have been associated with. Many, if not most conference review processes use blind reviews. Conferences are not full of "invited" speakers. Maybe one or two plenary sessions at the beginning, but the rest submitted their own papers and went through the blind review process. Certainly true in the best EE, CS, and ML conferences. I've served on program committees for international conferences. Around 100 papers selected typically, only two invited papers.
Most of the physical sciences have open conferences so anyone can submit without peer review. If a session has an invited speaker they are invited because the organizer knows their work.
This is such a weird denialism. Let me ask you something: if you go look at the top 100 cited researchers do you really think their enormous citation counts come from the brilliance of their work? Or is it just possible it's something else?
I'll tell you first hand, my advisor who is among those top 100, accretes citations like a black hole because he/she is famous, sits on committees (at uni and a national lab), speaks in front of Congress etc etc etc. and thus gets invited to be a coauthor on a billion papers a year (and not because of his/her brilliance).
Also if you think program committees and reviewers don't know who wrote a paper when the same group has been submitting to the same top conference every year for over a decade then I have a bridge to sell you.
I'll tell you first hand, my advisor who is among those top 100, accretes citations like a black hole because he/she is famous, sits on committees (at uni and a national lab), speaks in front of Congress etc etc etc. and thus gets invited to be a coauthor on a billion papers a year (and not because of his/her brilliance).
Also if you think program committees and reviewers don't know who wrote a paper when the same group has been submitting to the same top conference every year for over a decade then I have a bridge to sell you.
Speaking as a failed scientist (I was “dishonorably discharged” with an MS from a top US institution) I will tell you that the brilliant people around me knew that paper citations don’t correlate exactly with the impact of work within. Certainly didn’t feel fair. But I can also say the system is way more meritocratic than medicine or finance
Is it this or is it that people decide to see the talks they have larger odds of citing?
Getting to speak on a conference is easier than publishing in a periodic.
Getting to speak on a conference is easier than publishing in a periodic.
This definitely feels like the case in my field. Nothing good goes to journals.
> This is why it is dangerous to think that the best scientific theories always prevail and that science is a meritocracy.
This is because people form trapped priors and so some scientific advances come "one funeral at a time", but it's equally dangerous to think that science won't or cannot form clear consensus on observable, factual matters or that if there isn't always meritocracy, then there must be none.
Science is the only reason we can even have this conversation right now, after all.
This is because people form trapped priors and so some scientific advances come "one funeral at a time", but it's equally dangerous to think that science won't or cannot form clear consensus on observable, factual matters or that if there isn't always meritocracy, then there must be none.
Science is the only reason we can even have this conversation right now, after all.
Maybe from a far it seems that it is a meritocracy but there are documented examples of people falling through the cracks.
Eg Douglas Prasher did nobel prize winning work but couldn’t stay in academia.
Virginijus Siksnys did nobel prize work but never got the nobel prize.
The list goes on at the nobel prize level, so just consider the people who fall through the cracks for first class work, but not nobel prize level.
Eg Douglas Prasher did nobel prize winning work but couldn’t stay in academia.
Virginijus Siksnys did nobel prize work but never got the nobel prize.
The list goes on at the nobel prize level, so just consider the people who fall through the cracks for first class work, but not nobel prize level.
"Sometimes people do good work and don't win major awards" is hardly a rebuttal to the notion that meritocracy exists, because it's not some binary state. To say there's no such thing as merit, you have to say that recognition of merit is on average directionally wrong.
Given the huge technological advances of even just the past few decades, that's a pretty hard sell.
Given the huge technological advances of even just the past few decades, that's a pretty hard sell.
Thanks for the quote but that’s clearly not what I said.
Try quoting my first paragraph instead
>>“Maybe from a far it seems that it is a meritocracy but there are documented examples of people falling through the cracks.”
Some nuance in the discussion would be appreciated.
Try quoting my first paragraph instead
>>“Maybe from a far it seems that it is a meritocracy but there are documented examples of people falling through the cracks.”
Some nuance in the discussion would be appreciated.
Well, we went from talking about whether the best 'scientific theories' were recognized to whether certain individuals won certain awards, so it's hard to do anything but broad strokes.
I'm not here trying to refute the idea that sometimes people get passed over undeservedly, I'm refuting the all too common idea that this means that meritocracy is binary and anyone ever getting passed over means that merit doesn't exist and cannot be recognized.
If you don't hold that idea then good, we don't have to argue.
I'm not here trying to refute the idea that sometimes people get passed over undeservedly, I'm refuting the all too common idea that this means that meritocracy is binary and anyone ever getting passed over means that merit doesn't exist and cannot be recognized.
If you don't hold that idea then good, we don't have to argue.
At this point, I’d ask what you mean by meritocracy here. Is the (supposed) lack of meritocracy in the awarding of accolades the same as a lack of meritocracy when it comes to the recognition of the actual science done? Clearly from your comments, while these scientists didn’t win the Nobel prize, their work certainly got the recognition it did and made the impact it’s supposed to have.
This whole thread is a cesspool of bait and switch and moving of goalposts surrounding the idea of meritocracy. Has the Hacker News community always been so dogmatic about such topics, or is this a recent trend I’m observing in the last few years?
This whole thread is a cesspool of bait and switch and moving of goalposts surrounding the idea of meritocracy. Has the Hacker News community always been so dogmatic about such topics, or is this a recent trend I’m observing in the last few years?
Douglas Prasher didn’t make it in academia, he was a shuttle bus driver until Roger Tsien hired him back as an associate after prizes were won.
I don’t know what kind of goalposts you’re thinking about, but doing nobel prize work but not being able to have a career in science is evidence that people fall through the cracks of this supposed meritocratic structure.
I don’t know what kind of goalposts you’re thinking about, but doing nobel prize work but not being able to have a career in science is evidence that people fall through the cracks of this supposed meritocratic structure.
The fact that prizes were won and he got hired back because of that are both evidence that merit is recognized. You made points that contradict your own stance.
The comment of mine you were replying to describes an example of the moving of goal posts; conflating the merit of the issuing of accolades with the merit of recognition of work allows one to fallaciously reject the notion of merit altogether if one of these definitions of merit fails. Yet clearly, they are not the same thing.
For what it's worth, meritocracy is not a topic that I have a concrete stance on (yet). However, the top comments for this topic lacks nuance, and resorts to the kind of motte and bailey arguments I remarked about earlier. Imagine trying to have a productive conversation about anti-corruption, only to get remarks like "anti-corruption has failed with respect to my specific conception of anti-corruption in some specific scenario, therefore anti-corruption as a whole is a terrible goal or ideal". Given how broad the notion of "anti-corruption" can be, someone's specific conception of it in some scenario doesn't (necessarily) represent every conception or instantiation of the notion of anti-corruption. It's in this broadness where nuance can be found, and unfortunately, I don't find that in the top comments.
The comment of mine you were replying to describes an example of the moving of goal posts; conflating the merit of the issuing of accolades with the merit of recognition of work allows one to fallaciously reject the notion of merit altogether if one of these definitions of merit fails. Yet clearly, they are not the same thing.
For what it's worth, meritocracy is not a topic that I have a concrete stance on (yet). However, the top comments for this topic lacks nuance, and resorts to the kind of motte and bailey arguments I remarked about earlier. Imagine trying to have a productive conversation about anti-corruption, only to get remarks like "anti-corruption has failed with respect to my specific conception of anti-corruption in some specific scenario, therefore anti-corruption as a whole is a terrible goal or ideal". Given how broad the notion of "anti-corruption" can be, someone's specific conception of it in some scenario doesn't (necessarily) represent every conception or instantiation of the notion of anti-corruption. It's in this broadness where nuance can be found, and unfortunately, I don't find that in the top comments.
It should come as no surprise that if you develop a new theory and then proceed to burn it and all evidence of its existence that no one is going to hear of said theory. If you care about your theory prevailing, you need to go out and make an effort to communicate it. Science is meritocratic in the sense that even the small guy has a chance of fighting the machine as eventually things need to be defended on their technical merits. After all, Galileo was able to defeat the Church which was the largest country in Europe at the time. I think its dangerous to conflate meritocracy with not being required to communicate.
> Galileo was able to defeat the Church which was the largest country in Europe
The Church is not a country and never was? And Galileo was forced to recant then spent the rest of his life under house arrest. The Church defeated him.
> Science is meritocratic in the sense that even the small guy has a chance of fighting the machine as eventually things need to be defended on their technical merits
Nobody who has followed the shenanigans of scientific institutions in the last decade believes this anymore, sorry. Modern science consists of credentialed charlatans plying entirely fraudulent claims and those being accepted, over and over, with no end in sight, and everyone who tries to point out the problems in The Science end up being cancelled or burning out because the employers of said charlatans are the modern Church. They simply do not care if their people tell the truth or not.
The Church is not a country and never was? And Galileo was forced to recant then spent the rest of his life under house arrest. The Church defeated him.
> Science is meritocratic in the sense that even the small guy has a chance of fighting the machine as eventually things need to be defended on their technical merits
Nobody who has followed the shenanigans of scientific institutions in the last decade believes this anymore, sorry. Modern science consists of credentialed charlatans plying entirely fraudulent claims and those being accepted, over and over, with no end in sight, and everyone who tries to point out the problems in The Science end up being cancelled or burning out because the employers of said charlatans are the modern Church. They simply do not care if their people tell the truth or not.
Just addressing the Galileo part, but as was recently linked in hackernews [1][2], Galileo was wrong (in the sense that he was arguing for an already discredited theory, and that the theory he was arguing against was also already discredited).
Using Galileo as the standard of the suppression of free thought is sorta painting the Church as overzealously standing up for good science, not highlighting the Church as an enemy of truth.
1. http://tofspot.blogspot.com/2013/10/9-great-ptolemaic-smackd... 2. https://thonyc.wordpress.com/2010/11/12/galileo%E2%80%99s-gr...
Using Galileo as the standard of the suppression of free thought is sorta painting the Church as overzealously standing up for good science, not highlighting the Church as an enemy of truth.
1. http://tofspot.blogspot.com/2013/10/9-great-ptolemaic-smackd... 2. https://thonyc.wordpress.com/2010/11/12/galileo%E2%80%99s-gr...
Interesting, thanks!
> The Church is not a country and never was?
The Papal states were literally a country, although surely not the largest in Europe. Galileo did not live there but rather in the Duchy of Florence (later Grand Duchy of Tuscany); however, as a Catholic he could still be tried by the Inquisition, which was essentially the Interpol of heresy.
The Papal states were literally a country, although surely not the largest in Europe. Galileo did not live there but rather in the Duchy of Florence (later Grand Duchy of Tuscany); however, as a Catholic he could still be tried by the Inquisition, which was essentially the Interpol of heresy.
It's true in the same sense as the efficient market hypothesis or saying that the truth will prevail. It just tells you nothing about the timing of it.
Which is why it is especially important to have different expectations regarding newer science with little data and testing. But that's still where all the active research is going to happen.
Which is why it is especially important to have different expectations regarding newer science with little data and testing. But that's still where all the active research is going to happen.
The act of physically immersing myself into something just leaves my brain with a stronger memory of the content, than only viewing the content.
If some tech could close this gap, all sorts of learning rates would increase.
Until then, it's a downside of being remote (perhaps balanced by many upsides for many people!)
If some tech could close this gap, all sorts of learning rates would increase.
Until then, it's a downside of being remote (perhaps balanced by many upsides for many people!)
Screen and other distractions notwithstanding, attending a conference in person is inherently more immersive for a lot of people with reduced day-to-day distractions and in-person discussions about topics. Even if videos are good people are just more likely to tune out or go off and do something else.
Maybe I’m odd, but when I watch talks recorded at conferences I take notes, pause to visit resources discussed by the speaker and skip back to things I misheard or didn’t grasp properly. Then I’ll save it for reference along with the notes and slides if made available.
In person I’m usually feeling uncomfortable to be surrounded by so many people and can’t focus on the talk in the same way.
In person I’m usually feeling uncomfortable to be surrounded by so many people and can’t focus on the talk in the same way.
“Scientists more likely to cite the work of friends. Scientists also more likely to attend the talks of friends.”
Wonder if they controlled for this? I’m assuming the article doesn’t mention it (they rarely address obvious first order nitpicks).
Wonder if they controlled for this? I’m assuming the article doesn’t mention it (they rarely address obvious first order nitpicks).
There are so many factors that it's difficult to sort them out. Back when I attended conferences (have only been to one since Covid) I choose to attend talks that I'm interested in and skip those that are less interesting. Back when I was publishing, if there were another author working on the same problem as me for sure I would try to attend their talk and would probably end up citing it in a future paper.
If you tally just the papers and talks that you cited, not those you did not cite, what is the ratio of the amount of time spent on reading papers from conference talks and listening to the talks to the amount of time spent on reading papers without talks?
No idea. It's been a long time since I was in a research-oriented group and published frequently and I didn't track my time in the detail that would be needed to answer this.
But it also seems like scientists are more likely to cite friends whose presentations they see in person and cite friends less whose presentations they see virtually.
What would you do to control for that?
Only study scientists with no friends.
Canceling out author-reciprocal citations would be one approach to consider, though that would itself have some hazards we’d have to avoid.
Wouldn't this correlate with people with a bias for action? (both to physically go to a talk, and to take the action of creating something that would cite it)
Also, physically attending a talk shows a very strong interest in a subject, meaning you would probably already be working on some adjacent knowledge.
Also, physically attending a talk shows a very strong interest in a subject, meaning you would probably already be working on some adjacent knowledge.
Indeed. I would suspect that also the talk goes into more detail about some of the less-familiar contributions, which helps with understanding and retention. People won't cite papers if they don't understand them well, and that could happen if they're so dry that the point doesn't come across as easily.
I'm not sure this is as strong an effect as the research[1] suggests. Is it not possible that, when in choosing which of two liked papers with conflicting schedules to watch, researchers are demonstrating which is the better paper?
[1]: https://arxiv.org/pdf/2209.01175.pdf
[1]: https://arxiv.org/pdf/2209.01175.pdf
> They found that for attendees who had multiple schedule conflicts with a liked talk, the probability of later citing that talk was 2.6%. But for attendees who did not have a conflict, that figure rose to 3.8%.
Doesn’t seem super scientific, schedules and conflicts vary so much between people, personal events, exams, tragedy’s… pretty hard to lump these all together.
This could just be a matter of, the people who went, wanted to go more, therefore they’re more interested and more likely to cite the work in the future
Doesn’t seem super scientific, schedules and conflicts vary so much between people, personal events, exams, tragedy’s… pretty hard to lump these all together.
This could just be a matter of, the people who went, wanted to go more, therefore they’re more interested and more likely to cite the work in the future
Everything can be viewed as a product. A scientific paper is a product. Products need distribution and sales. Talks and 1-1 conversations are "sales" for papers.
unfortunately this mindset has plagued academia for decades now. when your KPI is the number of citations and your tenure depends on number of publications, you cannot expect quality teaching and even high quality research (in some fields, most research is a joke tbh, lots of papers that are not peer reviewed and are irreproducible).
There’s another alternative: conference talks have a higher signal to noise ratio than just pushing a paper out into the void (the background noise). To get a talk, you need to be selected, so that act of selection is the first filter. After that it’s way easier to identify the good talks from the bad.
"The mere-exposure effect is a psychological phenomenon by which people tend to develop a preference for things merely because they are familiar with them."
https://en.wikipedia.org/wiki/Mere-exposure_effect
https://en.wikipedia.org/wiki/Mere-exposure_effect
I've come to truly hate remote scientific conferences. There are a lot of things to commend them in theory, but between day-to-day distractions, the inability to really do the "I'm sorry, I'll be in $CITY" rather than "The conference ends at 3, surely you can still..." thing, fatigue from being in different time zones, etc. they just slide off my brain. I literally can't remember what sessions I even saw.
I can believe it, given the awfulness of remote conference tools. Someone will crack this problem - it might require better hardware - but the current systems are nothing like attending in person.
I often wondered why people got paid to flip signs for stores/tax services. Turns out it’s way more eyeball catching because of the way we evolved. Can’t this be something similar?
[deleted]
i wonder if this concept has implications for careers of remote workers