A follow-up on our analysis of the anomalous restriction map of SARS-CoV-2(alexwasburne.substack.com)
alexwasburne.substack.com
A follow-up on our analysis of the anomalous restriction map of SARS-CoV-2
https://alexwasburne.substack.com/p/open-letter-to-the-world
40 comments
There is zero chance this would have been accepted for publication in any serious journal. Alina Chan's paper got stuck in review for nearly 2 years. I think the authors were smart to solicit feedback in public, even if it's messy. It at least forced some engagement.
How do you feel about the market origins preprint that claimed to have "dispositive evidence" of market origin based off of cherry picked data and a photo from 2014 reported in every major outlet?
Pre-prints should be avoided for any study that has a political bent (even if the authors don't intend it to).
Pre-prints are fine for 99% of papers, because even if they are controversial they don't have a lot of reach. It is just a way for scientists to stake claims while undergoing review, which can take awhile.
For papers like these, traditional review is important because it can temper a lot of the more biased or political statements in the paper or help remove some of the more sketchy results/conclusions. It is far far from perfect, but it is better than nothing.
As I said above. The time pressure for covid is over. There is no reason to not undergo peer-review first for papers like these.
Pre-prints are fine for 99% of papers, because even if they are controversial they don't have a lot of reach. It is just a way for scientists to stake claims while undergoing review, which can take awhile.
For papers like these, traditional review is important because it can temper a lot of the more biased or political statements in the paper or help remove some of the more sketchy results/conclusions. It is far far from perfect, but it is better than nothing.
As I said above. The time pressure for covid is over. There is no reason to not undergo peer-review first for papers like these.
"I left academia because of the toxicity of COVID science communication, both within the scientific community and at the science-policy and science-public interfaces."
This truly sad and bias inducing for academia.
This truly sad and bias inducing for academia.
Can you please clarify your sentence?
From the article:
Personally, I think the dichotomy between hypothesis-testing and likelihood-quantification is a false one. The “P=0.05” cutoff we use to “reject” a hypothesis is an arbitrary one. When I read papers, I never “accept” or “reject” hypotheses but rather consider likelihood quantification as a measure of the weight of evidence or a distance of the data from some null hypothesis, as measured by some statistic. I encourage everyone else to consider this probabilistic worldview when viewing our paper: we aimed to quantify probabilities of this system occurring in nature, and P-values were convenient and commonly understood ways of communicating quantiles.
Though p-values are probabilities.
They are the probability that the data seen (or more extreme) in the experiment were generated given the null-hypothesis is true.
Now, of course to fully understand the p, you also have to understand the null hypothesis. And yeah, sometimes it is misspecified. (by e.g testing out many null-hypotheses and only showing the more interesting ones, or accidentally creating a bad unlikely null-hypothesis which may allow for many uninteresting alternative hypotheses.)
They are the probability that the data seen (or more extreme) in the experiment were generated given the null-hypothesis is true.
Now, of course to fully understand the p, you also have to understand the null hypothesis. And yeah, sometimes it is misspecified. (by e.g testing out many null-hypotheses and only showing the more interesting ones, or accidentally creating a bad unlikely null-hypothesis which may allow for many uninteresting alternative hypotheses.)
Obligatory p-value snippet from the ASA:
P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
To follow that up (so people know what they actually are), what p-values represent is the likelihood we would observe our data, given the null hypothesis.
Setting a cutoff of .05 is saying “if there’s less than a 5% chance we’d see this data, assuming the null hypothesis, then we can assume that the null hypothesis is false”
Setting a cutoff of .05 is saying “if there’s less than a 5% chance we’d see this data, assuming the null hypothesis, then we can assume that the null hypothesis is false”
But this statement only applies to a 'naive' (or first) statistical analysis or test on the dataset. Once the researcher starts changing their assumptions in response to the results they're seeing, they're p-hacking and p-values are no longer meaningful. In addition, once you have multiple researchers looking at the same dataset with different assumptions, and you factor in publication bias, the p-value also loses meaning.
Well, yes and no. The p-value still means the same thing, but when you take a dataset and go looking for any result that is under a certain threshold, you’ll probably find it. “Unlikely” events happen all the time!
What your comment is highlighting is an issue with bad experimental design. (And, obviously, with our publication regime)
What your comment is highlighting is an issue with bad experimental design. (And, obviously, with our publication regime)
I'm becoming more and more convinced we need to multiply anything that is not strictly a probability (CIs, ML model scores, p-values) by 100.
"I have a confidence of 95" has very different ring to it than "I am 95% confident."
It would also prevent people from doing stupid things like using these values to compute expectations.
"I have a confidence of 95" has very different ring to it than "I am 95% confident."
It would also prevent people from doing stupid things like using these values to compute expectations.
I'm impressed with the elegance of this method to get a single nice number to determine how evenly spaced things are in the whole:
> We chose our statistic based on principles. Since all fragments sum to the length of the genome, if the longest fragment is equal to the inverse of the number of fragments, it implies the sites are perfectly evenly spaced. Conversely, if the longest fragment is nearly the entire length of the genome, it implies the restriction sites are concentrated in a tiny region of the genome.
> We chose our statistic based on principles. Since all fragments sum to the length of the genome, if the longest fragment is equal to the inverse of the number of fragments, it implies the sites are perfectly evenly spaced. Conversely, if the longest fragment is nearly the entire length of the genome, it implies the restriction sites are concentrated in a tiny region of the genome.
article's flagged by s/o. attn dang for moderation
The author is part of a small team (himself plus two others) who recently released a preprint of a study suggesting that SARS-CoV-2 is of synthetic origin. He has been accused of not having the requisite academic credentials to be making such an assertion, and in this post he recounts his quite-impressive scholarly achievements.
The attacks upon the preprint are the product of an epistemically polluted trust network. In this case, the network (I believe) is aligned with the political Left. If the author were being attacked for asserting that Covid vaccines are generally safe and effective, one would assume the compromised trust network was aligned with the right. We as a society need to come up with a way to both foster freedom of speech and disseminate among the general populace the skills needed to vet competing accounts of scientific phenomena. And we all need to be willing to say, when appropriate, "There is not yet enough evidence to arrive at a firm conviction -- we will have to wait and see."
The attacks upon the preprint are the product of an epistemically polluted trust network. In this case, the network (I believe) is aligned with the political Left. If the author were being attacked for asserting that Covid vaccines are generally safe and effective, one would assume the compromised trust network was aligned with the right. We as a society need to come up with a way to both foster freedom of speech and disseminate among the general populace the skills needed to vet competing accounts of scientific phenomena. And we all need to be willing to say, when appropriate, "There is not yet enough evidence to arrive at a firm conviction -- we will have to wait and see."
Article author here - lots of interesting perspectives, I can't respond to them all, but I think this comment and the issues of bias induced by asymmetric application of trust (or vitriol) is important to consider.
Some other interesting points on probability etc. - happy to discuss further. Perhaps a call sometime? We're of course open to feedback & civil discourse, and I've always admired the ycombinator crowd. I thought of applying to start a phage therapy company, but investors required a patent, which requires disclosing the tech to the whole world, and the tech can be used to make extremely dangerous viruses capable of killing people, so I folded on that venture.
Anyways, I appreciate all of you entrepreneurs and smart people & your efforts to make the world better with disruptive innovations. I'm here - feel free to email me at [email protected] if you want to discuss anything further, and if possible perhaps a call with many people present and organized questions could be a way to engage in productive discourse? I'm flexible!
Some other interesting points on probability etc. - happy to discuss further. Perhaps a call sometime? We're of course open to feedback & civil discourse, and I've always admired the ycombinator crowd. I thought of applying to start a phage therapy company, but investors required a patent, which requires disclosing the tech to the whole world, and the tech can be used to make extremely dangerous viruses capable of killing people, so I folded on that venture.
Anyways, I appreciate all of you entrepreneurs and smart people & your efforts to make the world better with disruptive innovations. I'm here - feel free to email me at [email protected] if you want to discuss anything further, and if possible perhaps a call with many people present and organized questions could be a way to engage in productive discourse? I'm flexible!
>The attacks upon the preprint are the product of an epistemically polluted trust network. In this case, the network (I believe) is aligned with the political Left.
Because some equate a hypothesis related to the WIV with racism?
Because some equate a hypothesis related to the WIV with racism?
If WIV was the source, it's very plausible that NIH-funded GoF research (and implicitly the existing field of virology) was the cause of the pandemic.
There's been a lot of wagon-circling in the virology community at the implication that they caused instead of preventing a global pandemic.
There's been a lot of wagon-circling in the virology community at the implication that they caused instead of preventing a global pandemic.
If you look at the paper objectively even someone without the proper credentials should see red flags popping up left and right. He may have the credentials but his social media activities and his blatant hate for the scientific community (as he repeated in the post) during the pandemic seems to be driven by anger and hate. If he despises the scientific community that much and sees his studies and pre doctorates as a waste of time but uses them to give himself credibility seems odd. The "we did what we could we did the data we have" is also strange even if 90% of scientists are in on the conspiracy you would find somene who could provide a proper dataset. He could have avoided all of this if he would have just waited for the peer reviews, but we mostly know how they will turn out if he is that angry now. There are certainly still "Heureka" moments out there in which a hobbyist can outshine the status quo of the scientific community but this is not it.
> He may have the credentials but his social media activities and his blatant hate for the scientific community (as he repeated in the post) during the pandemic seems to be driven by anger and hate. If he despises the scientific community that much and sees his studies and pre doctorates as a waste of time but uses them to give himself credibility seems odd.
(Full disclosure: I have a science PhD)
By the end of it, I hated my supervisor so much I could barely manage to speak to him and I went out of my way to avoid him.
At the very end, after I'd done my viva and some minor corrections to my thesis I knew I've ticked all the boxes and the only thing left was to go in to clear out my lab space. At that point I simply didn't want to talk to anyone from the group so I went in very late one evening when I knew no-one would be around and packed up all my stuff and took it away.
I didn't say goodbye and there was no leaving party. I was there for three-and-a-bit years, and then I wasn't. I have had no contact with the group, or my supervisor, since.
I can easily imagine that someone with credentials / qualifications in a space might have issues with the system/community that appears to govern that space.
(Full disclosure: I have a science PhD)
By the end of it, I hated my supervisor so much I could barely manage to speak to him and I went out of my way to avoid him.
At the very end, after I'd done my viva and some minor corrections to my thesis I knew I've ticked all the boxes and the only thing left was to go in to clear out my lab space. At that point I simply didn't want to talk to anyone from the group so I went in very late one evening when I knew no-one would be around and packed up all my stuff and took it away.
I didn't say goodbye and there was no leaving party. I was there for three-and-a-bit years, and then I wasn't. I have had no contact with the group, or my supervisor, since.
I can easily imagine that someone with credentials / qualifications in a space might have issues with the system/community that appears to govern that space.
> If you look at the paper objectively ... He may have the credentials but his social media activities
So did you read the paper and come up with a substantive scientific critique, or did you judge him on his social media activities and social opinions?
So did you read the paper and come up with a substantive scientific critique, or did you judge him on his social media activities and social opinions?
I read articles, got more intrested, read the paper, found some procedural mistakes that threw red flags, looked for scientists in the field to see what they think about it and after all that I looked at his social media and got annoyed real fast that it was just another nothing burger by an angry person.
Curious, can you enumerate the red flags you saw?
The biggest was probably stating the results as facts in the result section and only as "more likely" in the conclusion. Sample size, taking only part of the genomes. The code quality of their scripts on GitHub and especially the comments in them. Bad spelling and convulted sentences... there is a lot more have a look yourself.
So you judged the science based on spelling in source code comments?
If you are looking for scientists in the field that tells us you are not in the field. But then you tell us you judged major procedural mistakes. How could someone who knows nothing about the field be an expert in procedural techniques in this field? Something tells me you are part of a group who feels threatened by these findings.
The process of writing scientific papers is mostly the same no matter the subject. The biggest flag was probably that he stated that he is 99,9% sure in interviews but the number never shows in the paper and in the conclusion part he just states "more likely". You don't need a bioengineering degree that something like that throws a red flag.
So you didn't find a red flag in the scientific paper but in something that was said in one interview?
He also stated it as fact in the shortform result (without the 99,9% number) contracdicting his own conclusion in the conclusion section. Sometimes providing information that has a clear correlation with the source material shows you that most people in this thread are trying to argue without reading the paper itself... helps sieving out if people are commenting in good faith or not.
And those red flags are…?
You seem to be criticizing the author of the paper instead of the paper itself, which, in my book, is itself a red flag.
You seem to be criticizing the author of the paper instead of the paper itself, which, in my book, is itself a red flag.
https://twitter.com/acritschristoph/status/15834864034169692... among other scientists has a long list of criticisms. I have not looked into any of this work and make no judgements about the relative believability, but many of the criticisms are entirely about the claims, not the pedigree of the scientist.
See also https://news.ycombinator.com/item?id=33279281
The biggest was probably stating the results as facts in the result section and only as "more likely" in the conclusion. Sample size, taking only part of the genomes. The code quality of their scripts on GitHub and especially the comments in them. Bad spelling and convulted sentences... there is a lot more have a look yourself.
> his blatant hate for the scientific community
Some areas of academia seem to do that to people. Nature even has a paper on it: https://www.nature.com/articles/d41586-021-00663-2
Some areas of academia seem to do that to people. Nature even has a paper on it: https://www.nature.com/articles/d41586-021-00663-2
There seems to be a fundamental problem with the logic here. It is asserted that because the COVID pandemic had a huge impact that the cause must be investigated, but if the origin was zoonotic then the origin may remain ambiguous and even if understood there may be no effective way to respond. That is, there may be multiple species functioning as reservoirs in the wild and no way to know for certain and no obvious way to eliminate contact with humans. It seems that the desire for a cause that can be handled with a simple response is leading to this conclusion.
> It is asserted that because the COVID pandemic had a huge impact that the cause must be investigated,
This is generally how we focus funds. If things are serious, they get serious funding. I'm not sure if you have seen how many people covid killed, but it was a lot. Plus, humans, despite being really bad at it, love causality.
> There seems to be a fundamental problem with the logic here.
> but if the origin was zoonotic then the origin may remain ambiguous and even if understood there may be no effective way to respond.
Most of science is wandering into the darkness. I'm not sure why you're just telling us to give up before we even start. There's plenty of scientific works that took well over a hundred years to solve. Giving up before you begin is the fundamental problem with logic here. It is better to improve our knowledge than sit by complicity. Your comment is fundamentally anti-scientific.
This is generally how we focus funds. If things are serious, they get serious funding. I'm not sure if you have seen how many people covid killed, but it was a lot. Plus, humans, despite being really bad at it, love causality.
> There seems to be a fundamental problem with the logic here.
> but if the origin was zoonotic then the origin may remain ambiguous and even if understood there may be no effective way to respond.
Most of science is wandering into the darkness. I'm not sure why you're just telling us to give up before we even start. There's plenty of scientific works that took well over a hundred years to solve. Giving up before you begin is the fundamental problem with logic here. It is better to improve our knowledge than sit by complicity. Your comment is fundamentally anti-scientific.
The usual thing is to start with ideas about what might be possible. Then hone understanding to narrow possibilities to what is probable and use that to form a hypothesis that can be tested. Deciding from the start that simple causality with easy safety enhancing actions must be the case because that would be desirable fails to account for a critical possibility. If you start investigation by throwing out basic practice then the potential for compelling and broadly accepted results is limited.
> Deciding from the start that simple causality with easy safety enhancing actions must be the case because that would be desirable fails to account for a critical possibility
This is a real bad faith read of my comment.
This is a real bad faith read of my comment.
The dramatic claims of the paper are not supported by experiments present. They highlight a real pattern, but that is far from proving SC2 being of synthetic origin. A better approach would have been waiting for peer review, probably getting dinged on speculative conclusions, softening the language and then publishing in a peer reviewed journal
Instead they juked the review and posted on a pre-print with speculative language intact. There is no time pressure, they could have waited a few months to have outside eyes on it. I think they knew their conclusion was a reach and wanted to get it out there anyway for the twitter clout.
Having contrarians in science is helpful because they will always tilt at windmills, and sometimes succeed, but usually not. But that's why review is so important. It should have been reviewed by three experts before publishing. Instead they chose to have peer-review be public, by thousands of people, many of whom are not experts. They made their bed.