Feynman’s Maze-Running Story(gwern.net)
gwern.net
Feynman’s Maze-Running Story
https://gwern.net/maze
49 comments
I thought so too at first, but that would only make sense if "Cargo Cult Science" were an unedited third-party transcript of the commencement address, and I don't think that it is, given its inclusion in "Surely You're Joking, Mr. Feynman!." The address refers to "a man named Young", not "a man, let's call him Young".
I see your point, and it's a good one.
I wonder if it's possible that there was a transcription error, or perhaps something introduced by the editor of the journal where the speech was first published [1]. If there was, then it's perhaps also plausible that the error could have been carried forward by copying and pasting the journal article into Surely you're joking.
All wild speculation on my part, of course. I wonder if any further evidence will ever turn up!
[1] https://calteches.library.caltech.edu/51/2/CargoCult.pdf
I wonder if it's possible that there was a transcription error, or perhaps something introduced by the editor of the journal where the speech was first published [1]. If there was, then it's perhaps also plausible that the error could have been carried forward by copying and pasting the journal article into Surely you're joking.
All wild speculation on my part, of course. I wonder if any further evidence will ever turn up!
[1] https://calteches.library.caltech.edu/51/2/CargoCult.pdf
> that would only make sense
You're suggesting that _no one_ ever used a pseudonym for another without explicitly mentioning it in advance/shortly thereafter?
Besides, "Young" as a false-name doesn't have to be a direct clever "reference" to "a young person". It could just be a name he thought of at the time.
In any case, I think Gwern is far too conspiratorial in the reasoning for this article. There are some interesting connections found but the evidence, while reasonable, is not nearly as conclusive as he suggests.
You're suggesting that _no one_ ever used a pseudonym for another without explicitly mentioning it in advance/shortly thereafter?
Besides, "Young" as a false-name doesn't have to be a direct clever "reference" to "a young person". It could just be a name he thought of at the time.
In any case, I think Gwern is far too conspiratorial in the reasoning for this article. There are some interesting connections found but the evidence, while reasonable, is not nearly as conclusive as he suggests.
> You're suggesting that _no one_ ever used a pseudonym for another without explicitly mentioning it in advance/shortly thereafter?
In academic writing like this, where he is being quite precise about the names and explicitly stating 'a man named Young', where the speech is in fact about being precise & scientific, and the anecdote makes up a good chunk of the speech, which was written in advance, and published at least twice by him afterwards over a decade? Yes. Feynman would have highlighted or signaled a pseudonym, he would not have used a plausible surname (I will remind the reader again, quite a common one back then, and still is).
> the evidence, while reasonable, is not nearly as conclusive as he suggests.
You are suggesting that all this is a red herring and we will find another, completely independent, multi-year investigation into rats using floor cues in mazes done by American grad students in the late 1930s (when American academia was far smaller than it is now), published in a way Feynman could find out, at institutions with links to Feynman via Cornell & Michigan, which also chose to use sand for the same task and rationale, which is not cited by Shepard or Curtis or indeed by any researchers in any of the scores of papers I have jailbroken & read on the topic of sensory leakage in mazes, which has simultaneously eluded all discovery by at least a decade and a half of people taking cracks at it (the 2009 Quora, and then several blogs going back to 2004, not counting the '90s Usenet discussions), and which happens to by sheer coincidence, be the actual source for the anecdote?
That seems unlikely.
In academic writing like this, where he is being quite precise about the names and explicitly stating 'a man named Young', where the speech is in fact about being precise & scientific, and the anecdote makes up a good chunk of the speech, which was written in advance, and published at least twice by him afterwards over a decade? Yes. Feynman would have highlighted or signaled a pseudonym, he would not have used a plausible surname (I will remind the reader again, quite a common one back then, and still is).
> the evidence, while reasonable, is not nearly as conclusive as he suggests.
You are suggesting that all this is a red herring and we will find another, completely independent, multi-year investigation into rats using floor cues in mazes done by American grad students in the late 1930s (when American academia was far smaller than it is now), published in a way Feynman could find out, at institutions with links to Feynman via Cornell & Michigan, which also chose to use sand for the same task and rationale, which is not cited by Shepard or Curtis or indeed by any researchers in any of the scores of papers I have jailbroken & read on the topic of sensory leakage in mazes, which has simultaneously eluded all discovery by at least a decade and a half of people taking cracks at it (the 2009 Quora, and then several blogs going back to 2004, not counting the '90s Usenet discussions), and which happens to by sheer coincidence, be the actual source for the anecdote?
That seems unlikely.
The real morality play here is to publish your work.
I suggest that Shepard had a net-negative effect on the field. Anyone who might eventually have done the same research would have heard about Shepard's work and been discouraged from re-doing it. Yet, it remained obscure and little-used. It may be better to not do the research at all than to do it, mark the territory, and not publish.
I suggest that Shepard had a net-negative effect on the field. Anyone who might eventually have done the same research would have heard about Shepard's work and been discouraged from re-doing it. Yet, it remained obscure and little-used. It may be better to not do the research at all than to do it, mark the territory, and not publish.
Talking to people who worked in that era, it was a different time with different constraints. There was no internet, and publishing a paper could be a time-intensive and laborious process. Typing, mailing, typesetting, and so forth. So it was common for a lot of stuff to just be published in house as technical reports and so forth, to not involve the overhead of external publishers. If you wanted something, you'd mail or call a colleague and ask them, and they'd mail you a white paper or monograph from their stack of copies in the hallway.
I started doing research long ago enough that I got a bit of exposure to that world, in the sense that that publication process was still briefly in place, and I saw those white papers passed around and used. Some of them were very influential.
The problem in some ways is that as people migrated to the current system with an emphasis on published articles, that stuff that people were actually using kind of dropped out of consciousness, sort of along the lines the article is describing. I can think of unpublished white papers that were cited immensely; people made fun of them never being turned into published papers. Maybe it's hurt the the paper in the current day along the lines of the target piece, but what's sad or funny is it never occurred to people that there was a work that was being cited a lot that never got published, as in to question the value of the publication process itself.
I don't disagree with you but there's a bit of broader historical context that's missing from the piece I think.
The irony is that today we might be seeing a return to the white paper model, in the form of preprints and open access archival papers, but it's just massively distributed. Maybe there's some lessons to be learned from that lost previous era (Google search dropping links, etc.)? Maybe it's just inevitable that as time goes on some things will be lost regardless.
I started doing research long ago enough that I got a bit of exposure to that world, in the sense that that publication process was still briefly in place, and I saw those white papers passed around and used. Some of them were very influential.
The problem in some ways is that as people migrated to the current system with an emphasis on published articles, that stuff that people were actually using kind of dropped out of consciousness, sort of along the lines the article is describing. I can think of unpublished white papers that were cited immensely; people made fun of them never being turned into published papers. Maybe it's hurt the the paper in the current day along the lines of the target piece, but what's sad or funny is it never occurred to people that there was a work that was being cited a lot that never got published, as in to question the value of the publication process itself.
I don't disagree with you but there's a bit of broader historical context that's missing from the piece I think.
The irony is that today we might be seeing a return to the white paper model, in the form of preprints and open access archival papers, but it's just massively distributed. Maybe there's some lessons to be learned from that lost previous era (Google search dropping links, etc.)? Maybe it's just inevitable that as time goes on some things will be lost regardless.
This. I had a grant proposal rejected because the reviewers knew that something similar was being done by someone else -- who then never bothered to complete the work and publish properly.
What recently happened with me is that we proved an open problem, and while writing it up, we found that the result was claimed in an arxiv paper that never got published anywhere. Their proof was incomplete.
This meant that we had to acknowledge that the result already was claimed by someone else, but that we couldn't understand the proof attempt.
This of course leads to problems because now our result is somehow viewed as less valuable, and more difficult to publish. Which again means that research in that area will suffer from an unpublished claim.
This meant that we had to acknowledge that the result already was claimed by someone else, but that we couldn't understand the proof attempt.
This of course leads to problems because now our result is somehow viewed as less valuable, and more difficult to publish. Which again means that research in that area will suffer from an unpublished claim.
What are you measuring to determine that now your result is viewed as less valuable? Is it the peer reviewers or personal conviction? In the end all that matters for you, to the extent that peer-reviewed publications are important to your career, is that your paper gets cited. And if people are citing the unfinished arxiv paper instead of yours then that speaks poorly of their own research and not yours.
[deleted]
The idea of an academic organization "fixing its hermeticism" is thought provoking.
Were these nerds toiling to the benefit or tune of some charlatan, some _Trismegistus_ -- or were they too much the hermit, one who toils without the clutch or gearing for meshing with the gears of society? With the central limit theorem their sovereign then probably both I guess, instinct says more the latter.
The way you file the teeth of your gears is a real struggle. I wish there were more resources for it, AI has an opportunity to fill a gap here. I know the path of salvation is doing my normal do _just more aggressively and in public_.
I managed to carve out a real small niche making an indie game publicly over a year and a half, posted a few hundred videos. Some knocks and I lost the momentum for constant publishing. The work hasn't stopped, but the ephemeral threads of a narrative feel harder to grasp. Do I just start posting things out of nowhere, acting like I never stopped? Do I montage the unpublished backlog? Do I wait until I've achieved some scientific breakthrough?
I've got videos of flood fill lightning and neural networks and sheep shapeshifting and explosions and probably 100 other things I've forgotten. Maybe the problem is that I never knew who my audience was. Shouting into the canyon is harder than it looks.
Were these nerds toiling to the benefit or tune of some charlatan, some _Trismegistus_ -- or were they too much the hermit, one who toils without the clutch or gearing for meshing with the gears of society? With the central limit theorem their sovereign then probably both I guess, instinct says more the latter.
The way you file the teeth of your gears is a real struggle. I wish there were more resources for it, AI has an opportunity to fill a gap here. I know the path of salvation is doing my normal do _just more aggressively and in public_.
I managed to carve out a real small niche making an indie game publicly over a year and a half, posted a few hundred videos. Some knocks and I lost the momentum for constant publishing. The work hasn't stopped, but the ephemeral threads of a narrative feel harder to grasp. Do I just start posting things out of nowhere, acting like I never stopped? Do I montage the unpublished backlog? Do I wait until I've achieved some scientific breakthrough?
I've got videos of flood fill lightning and neural networks and sheep shapeshifting and explosions and probably 100 other things I've forgotten. Maybe the problem is that I never knew who my audience was. Shouting into the canyon is harder than it looks.
> Maybe the problem is that I never knew who my audience was.
Knowing your audience is indeed step one. Truly.
Build at least one Ideal Customer Avatar [0]. Don't skip the step about visualizing them, even if it feels silly - it helps.
Maybe start a monthly newsletter, mostly so people are reminded you exist once in a while. Your unpublished backlog is probably a great resource for this.
And yes - just start posting again.
Good luck with the game, looks fun.
0 - https://gibbonet.co.uk/5-steps-to-build-your-ideal-customer-...
Knowing your audience is indeed step one. Truly.
Build at least one Ideal Customer Avatar [0]. Don't skip the step about visualizing them, even if it feels silly - it helps.
Maybe start a monthly newsletter, mostly so people are reminded you exist once in a while. Your unpublished backlog is probably a great resource for this.
And yes - just start posting again.
Good luck with the game, looks fun.
0 - https://gibbonet.co.uk/5-steps-to-build-your-ideal-customer-...
Thank you for this, ideal customer design is within my wheelhouse, I'll give it a try. I rationally get the benefit of newsletters but I need to nail some things down first.
I think this is a good first step you've pointed me towards.
I think this is a good first step you've pointed me towards.
got a link to it?
https://twitter.com/leapjosh is where I posted most of my things. I've got a small demo here -- https://store.steampowered.com/app/1537490/Tentacle_Typer/
Basically all of my textual content ends up here on HN :)
Basically all of my textual content ends up here on HN :)
Wow this is one of those gems on Steam I was looking for last night... I hope you release it one day because I've never seen anything like it: a Lovecraft-ian white-collar spell-casting acid trip?
that is one weird looking game... I'll keep an eye out on it.
Absolutely ridiculous that Caltech has had Feynman's papers for over 30 years without digitizing them or allowing people to read them without visiting LA! One of the greatest minds to ever exist, and Caltech is doing a disservice to his name with their white tower holding his documents. My blood's boiling just reading about Gwern's search through the literature. Shoutout to Gwern for digging this deep into one of my hero's citations
The OP says Feynman "made a practice of donating" his papers to Caltech. Did he know or expect they'd be managed in such a manner?
Edit: According to Caltech, they have "93 boxes" of Feynman's work that has never been backed up digitally[0]. Beyond the issue of access, what happens if there's ever a sprinkler burst in the wrong room of the archives? [0]https://collections.archives.caltech.edu/repositories/2/reso...
Edit: According to Caltech, they have "93 boxes" of Feynman's work that has never been backed up digitally[0]. Beyond the issue of access, what happens if there's ever a sprinkler burst in the wrong room of the archives? [0]https://collections.archives.caltech.edu/repositories/2/reso...
to quote myself, "All too often journals, galleries, and museums are becoming not
disseminators of knowledge—as their lofty mission statements
suggest—but censors of knowledge, __because censoring is the one thing
they do better than the Internet does__."
It's a great quote and I will use it.
This may have been submitted before, but the update today is that Gwern appears to have found the research that Feynman was referring to.
i did my a-level (british pre-university exam) project in 1971 on maze-running rats. i soon worked out that the horrible creatures were pissing everywhere and following the scent (and difficult to deal with for a plethora of other reasons) and switched to gerbils - a slight improvement (desert animals - don't pee so much) , and less likely to have your finger taken off compared with an enraged rat.
How times change. To run the same experiment in the UK today you'd need a Home Office licence, a project licence, and on-site veterinary facilities.
possibly. i wasn't doing anything nasty to the rats/gerbils except keeping them a bit hungry, and i despise animal cruelty in all its forms. i actually gave up attending my HND biology course (needed to progress in career as a microbiology tech) because they needlessly made us pith a frog (i like frogs). still won the prize for best student - god knows how! stick with your convictions is probably the answer.
Sounds like an improvement to me.
I'm curious, if it was a possibility that your finger get bitten off, did people not wear leather handling gloves or could rats bite through the gloves or something else?
most breeds of lab rats are pretty docile, but sometimes even they get pissed off when they have been starved (to motivate them) and then stuffed in a maze.
no, i never wore gloves. them was hard times!
no, i never wore gloves. them was hard times!
I have long complained about the orphan works problem and the need to free them up through copyright reform. This case is a quintessential example and copyright reformers should take note.
Copyright needs some kind of usucaption mechanism: if the holder doesn't make any effort in maintaining his works available for an extended period of time they should automatically enter public domain.
[deleted]
Are university results covered by copyright in the US?
Typically the copyright belongs to the journal which published the results.
And that (c) applies to only the published copy. It is very common, even standard in many fields, for researchers to make the same material, just not exactly the published copy, publicly available. See eg arxiv.org.
The anecdote is telling, and of course for the HN crowd we are as susceptible to cargo cults as anyone.
Personally I'm very interested in the survivability of scientific information. I used to think that this was a worry of the Internet age, but papers locked up in working group reports and journals are increasingly hard to find beyond about a 25 year window. There was a thread on HN in 2020 where someone was looking for a NATO working group report from 1960 and you'd think NATO of all people would preserve records...but there's a black hole.
Personally I'm very interested in the survivability of scientific information. I used to think that this was a worry of the Internet age, but papers locked up in working group reports and journals are increasingly hard to find beyond about a 25 year window. There was a thread on HN in 2020 where someone was looking for a NATO working group report from 1960 and you'd think NATO of all people would preserve records...but there's a black hole.
If this was discovered in the 1930s, why haven't others attempted to remove side channel information as they perfected their experiments, even if the original research was not published broadly?
Feynman implies that psychology is a "cargo-cult" science where variations of experiments and assumed hypothesis are valued over scientific rigor or reproducibility. That's my guess too. See: misattribution of arousal studies, Millikan oil drop studies, Covid-19 and Didier Raoult Livre with hydroxychloroquine... etc. etc.
Edit: The article states B.F. skinner did create a box to remove these side-channel effects for rat experiments
Edit: The article states B.F. skinner did create a box to remove these side-channel effects for rat experiments
Math and hard science have political and cultural biases and inertia as well. At a minimum, hard science is still manipulated by justifying funding.
And then there's this: https://esoterx.com/2014/12/03/murder-by-math-the-irrational...
There was institutional pushback on Godel's incompleteness from the axiomatic mathematicians that thought they could prove everything as they reconstructed math from axioms (if I understand that). From what I recall reading the axiomatic reconstruction was basically a huge industry of mathematical work, and suddenly some pissant young guy shows up and says "yeah, that not going to end the way you hope".
The mechanism of tenure alone will lead to political grandstanding. If your research is borderline on tenure but some other research completely undermines it, that is millions of lifetime dollars on the line.
See: MOND vs Dark Matter, string theory, etc.
And then there's this: https://esoterx.com/2014/12/03/murder-by-math-the-irrational...
There was institutional pushback on Godel's incompleteness from the axiomatic mathematicians that thought they could prove everything as they reconstructed math from axioms (if I understand that). From what I recall reading the axiomatic reconstruction was basically a huge industry of mathematical work, and suddenly some pissant young guy shows up and says "yeah, that not going to end the way you hope".
The mechanism of tenure alone will lead to political grandstanding. If your research is borderline on tenure but some other research completely undermines it, that is millions of lifetime dollars on the line.
See: MOND vs Dark Matter, string theory, etc.
> why haven't others attempted to remove side channel information as they perfected their experiments
How do we know they haven't?
How do we know they haven't?
The replication crisis represents decent evidence that this kind of thing is still a major problem today.
I’d consider that a broad label covering all sorts of situations rather than a smoking gun. To replicate the paper I’m currently reading you’d need to source or rebuild some really shitty 1970s tech and then ignore decades of thinking around the domain. It’d be a waste of time.
The piece kind of focuses on psychology, but this amnesia and collective forgetting tends to happen in all fields to some extent. It might be more significant in psychology, but there is research showing that some of the most cited papers tend to cite both new and very very old work (that is the references cited tend to vary in chronological age significantly), that significant research is often forgotten and sometimes rediscovered.
I think it goes along with the Kuhnian (?) idea of paradigms becoming overly dominant and overly entrenched, and coming and going wholesale in waves. It's also consistent with Stigler's law of eponymy. As fields lurch from one idea or paradigm to another they tend to pretend that anything similar didn't exist beforehand. There's a need to represent ones's self and ideas as revolutionary and genius and implicitly, by comparison, to denigrate everything that came before as irrelevant.
This article describes a related but different phenomenon:
https://www.nature.com/nature-index/news-blog/review-article...
One of my curiosities is whether or not this has gotten worse in science and academics.
It often feels like in academics there's such a pressure for self-promotion and receipt of attention, to capture attention and be at the center of what is the focus of attention, that issues of veridicality or what has and should be the focus of attention gets swept away.
I think it goes along with the Kuhnian (?) idea of paradigms becoming overly dominant and overly entrenched, and coming and going wholesale in waves. It's also consistent with Stigler's law of eponymy. As fields lurch from one idea or paradigm to another they tend to pretend that anything similar didn't exist beforehand. There's a need to represent ones's self and ideas as revolutionary and genius and implicitly, by comparison, to denigrate everything that came before as irrelevant.
This article describes a related but different phenomenon:
https://www.nature.com/nature-index/news-blog/review-article...
One of my curiosities is whether or not this has gotten worse in science and academics.
It often feels like in academics there's such a pressure for self-promotion and receipt of attention, to capture attention and be at the center of what is the focus of attention, that issues of veridicality or what has and should be the focus of attention gets swept away.
...this amnesia and collective forgetting tends to happen in all fields to some extent...
Well, I think there can an opposite problem that appears looking at these things. Psychology is such garbage, so likely to produce worthless experiments, that those that don't want to call a spade a spade instead try to take all the garbage of the field and average it over all science so that they can have something different to say.
This relates to the "replication crisis" [1]. You can say "The replication crisis is frequently discussed in relation to psychology and medicine..." but also said to extend throughout science. It may indeed extend throughout sciences but including a data group (experimental psychology) that you is garbage isn't necessarily a good way to see what's happening in the rest of science.
And, of course, a field "having amnesia" by itself doesn't necessarily imply any kind of problem. A given practitioner A) can only remember so much and B) benefits from a compact statement of a result, which may be quite different from how the result was originally stated. The math I learned in my MA came from text books that gave self-contained summaries of the various topics, rather than from the original papers the ideas came from and that made it much easier.
[1] https://en.wikipedia.org/wiki/Replication_crisis
Well, I think there can an opposite problem that appears looking at these things. Psychology is such garbage, so likely to produce worthless experiments, that those that don't want to call a spade a spade instead try to take all the garbage of the field and average it over all science so that they can have something different to say.
This relates to the "replication crisis" [1]. You can say "The replication crisis is frequently discussed in relation to psychology and medicine..." but also said to extend throughout science. It may indeed extend throughout sciences but including a data group (experimental psychology) that you is garbage isn't necessarily a good way to see what's happening in the rest of science.
And, of course, a field "having amnesia" by itself doesn't necessarily imply any kind of problem. A given practitioner A) can only remember so much and B) benefits from a compact statement of a result, which may be quite different from how the result was originally stated. The math I learned in my MA came from text books that gave self-contained summaries of the various topics, rather than from the original papers the ideas came from and that made it much easier.
[1] https://en.wikipedia.org/wiki/Replication_crisis
Alternative hypothesis: Feynman knew who the individual was but, having not asked them if they objected to being used to illustrate his anecdote, chose to hide their identity with a pseudonym. In that case the reference wasn't to "Mr Q. Young", it was to "Mr Young Researcher".