Fermilab is 'doomed' without management overhaul claims whistleblower report(physicsworld.com)
physicsworld.com
Fermilab is 'doomed' without management overhaul claims whistleblower report
https://physicsworld.com/a/fermilab-is-doomed-without-management-overhaul-claims-whistleblower-report/
37 comments
As someone working in scientific computing, I seem to recall rumors from the grapevine that the Department of Energy and/or reviewers essentially said, "CentOS is identical to Scientific Linux. Why are we paying for this?" and pushed hard to sunset it.
It's really sad because Fermi Linux was one of the first (if not _the_ first) free RH rebuilds out there back in the day. Huge props to Connie Sieh and the maintainers that followed for pulling that off. Really wonderful folks.
It does certainly feel convenient for RH that the end of SL coincided with radical changes to CentOS.
It's really sad because Fermi Linux was one of the first (if not _the_ first) free RH rebuilds out there back in the day. Huge props to Connie Sieh and the maintainers that followed for pulling that off. Really wonderful folks.
It does certainly feel convenient for RH that the end of SL coincided with radical changes to CentOS.
They could've just switched to Debian variants with a little effort.
More newly, since 2021, we have Alma Linux and Rocky Linux that are RHEL based.
More newly, since 2021, we have Alma Linux and Rocky Linux that are RHEL based.
Fermi did swap to AlmaLinux.
AlmaLinux is a perfect replacement for scientific agencies. It's got the fastest turnaround time for security patches, an open community, and sustainable funding sources.
I wonder if there were any nice dinners had by the administration and Red Hat sales.
An investigation shows a problem with leadership and leadership responds by saying the report isnt correct. Classic.
Fermilab doesn't pay its employees fairly. As such, the only non-scientist engineer employees that stay beyond a few years are ones that would struggle to get a job elsewhere. This is a good way to breed dysfunction. As for management, if they're not scientists doing science, at least half-time, they shouldn't be there.
I've always described it as people who valued stability over anything else - not just in the sense of "would not get a job elsewhere", but also "values knowing they're not going to get fired in the next decade no matter what", whether that's because they're there for the benefits or the retirement plan or w/e.
You may not get fired, but how long could you survive without a raise? Not long. Any raises they give are about 0.01%. The benefits aren't extraordinary in any sense. In fact, they don't give a pension, whereas even a local government does.
Some people survive for quite some time, depending on where they are.
But notably, if they're linked to academic institutions, you and yours might get significant benefits around continuing education.
But notably, if they're linked to academic institutions, you and yours might get significant benefits around continuing education.
This is a problem across the hard sciences particularly ones with a computational focus. The salaries are, simply put, pathetic. Even at private labs I have friends who make 80-100k USD when industry would pay the same job twice as much. And it's not like 100k is the same 100k it was 5-10 years ago. It's wild that nothing is being done to increase salaries in science. Everyone just seems to bury their heads in the sand and hope someone else will fix it.
> The issues at Fermilab are, however, also hindering the DOE, which earlier this year called for bids on the contract to operate the lab. The University of Chicago and the URA have submitted a contract bid together with other partners. Associated Universities, Inc., which runs the US-based 100 m-diameter Green Bank Telescope and the Atacama Large Millimeter/submillimeter Array in Chile, has also thrown its hat in the ring. The DOE says it will announce the winner of the contract by 30 September.
Just give "us" the contract and we'll address all the problems that have come to the fore since the request for bids was announced. Does it ever end?
Just give "us" the contract and we'll address all the problems that have come to the fore since the request for bids was announced. Does it ever end?
Anyone know the current state of the art on "tabletop" particle accelerators?
https://newatlas.com/berkely-world-record-compact-particle-a...
https://news.stanford.edu/stories/2019/09/precision-physics-...
https://newatlas.com/berkely-world-record-compact-particle-a...
https://news.stanford.edu/stories/2019/09/precision-physics-...
Should it ever come to pass that FermiLab were shut down, the real-estate developers would grab it in a heart beat. It's a massive block of prime west Chicagoland Real Estate.
The chance of that happening is close to nil. The property is owned by the US Government and would likely get transferred to the US Forest Service to be managed as a tallgrass prairie, like Midewin [1] (large amounts of open space within Fermilab are already being managed as a tallgrass prairie [2]).
[1] https://www.fs.usda.gov/midewin
[2] https://ecology.fnal.gov
[1] https://www.fs.usda.gov/midewin
[2] https://ecology.fnal.gov
Legality aside, I used to work in Hinsdale and I’ve visited Fermilab in Batavia.
I could be all wrong about this but the location didn’t seem that desirable. Batavia is far away from major highways and feels like a forgotten sleepy town out in the boonies. I remember feeling, “who would ever want to live here except physicists who wanted to isolate themselves?”
This is in contrast to Argonne National Lab (also operated by UChicago) down the street (well 30 mi, 40 mins away) in Lemont. It’s next to the I-55 and reachable from populated burbs like Naperville and Westmont, and even Chicago. Food options abound in neighboring suburbs. Now that’s prime real estate.
I could be all wrong about this but the location didn’t seem that desirable. Batavia is far away from major highways and feels like a forgotten sleepy town out in the boonies. I remember feeling, “who would ever want to live here except physicists who wanted to isolate themselves?”
This is in contrast to Argonne National Lab (also operated by UChicago) down the street (well 30 mi, 40 mins away) in Lemont. It’s next to the I-55 and reachable from populated burbs like Naperville and Westmont, and even Chicago. Food options abound in neighboring suburbs. Now that’s prime real estate.
Argonne is completely surrounded by (and was carved out of) DuPage County Forest Preserve land. They'll want it back if the government is ever done with it. The forest preserve district will simply remove all the roads on their land and leave it inaccessible.
You haven't been down Kirk road in the past few years, have you? There are so many McMansions across the street in land that used to be farms.
The central problem is the bread and butter of Fermilab, operating an accelerator to push the frontiers of physics, is reaching its limit. At some point one must admit the place has discovered what it's going to discover and decide to shut the place down.
This problem is facing particle physics as a whole, world wide, since accelerators have become increasingly expensive as the needed energy has increased.
This problem is facing particle physics as a whole, world wide, since accelerators have become increasingly expensive as the needed energy has increased.
> In the new whistleblower report, the group claims there are “too many deficiencies in the culture and behavioural areas” at Fermilab. They point, for example, to the lab’s dismissal of an early-career researcher in 2023 who had alleged sexual assault in 2018, and raised several cover-ups by management of dangerous behaviour. The report also highlights a case of guns being brought onto Fermilab’s campus in 2023; a male employee’s attack on a female colleague using an industrial vehicle in 2022; and retaliation against an employee who had predicted and warned management about the failures of beryllium windows.
What do these things have to do with the state of particle physics?
What do these things have to do with the state of particle physics?
They have to do with the state of the management (and the culture).
An organization that is under increasing existential pressure will exhibit all sorts of dysfunction. For example, they will lowball bids for projects that are necessary for their survival, from which one can then expect "surprise" cost overruns.
Exactly. There was the famous quote from a US presidential candidate who couldn't even remember what the DOE was called, even though he wanted to abolish it[1].
[1]: https://time.com/4598910/rick-perry-department-energy-oops-g...
[1]: https://time.com/4598910/rick-perry-department-energy-oops-g...
I don't know all the details, but I think they are starting to do novel work with the accelerator to produce neutrinos in an experiment called Dune. So there actually is important stuff being done that isn't just superceded by Cern.
Yes, they are squeezing as much juice as they can out of the lemon. But there's diminishing returns. It can't continue forever, and the budget will be under inexorable pressure as the scientific returns become increasingly marginal.
This is a problem with science, particularly particle physics: discovering is like mining. Eventually the "ore body" of new phenomena is used up. The period of rapid discovery (up to say the mid 1970s, with a decreasing trickle to 2012 with the discovery of the Higgs Boson) in particle physics will look very atypical and transitory in historical hindsight.
This is a problem with science, particularly particle physics: discovering is like mining. Eventually the "ore body" of new phenomena is used up. The period of rapid discovery (up to say the mid 1970s, with a decreasing trickle to 2012 with the discovery of the Higgs Boson) in particle physics will look very atypical and transitory in historical hindsight.
To be honest, you can't know exactly where you are going to find things. I think the rapid discoveries that particle physics were known for within the last centuries maybe are the limits of our current technology. It is not that we lack the ideas, it maybe that we have squeezed the remaining bits of hanging fruits left.
When considering how vacuum tubes were used for a lot of discoveries in physics (nuclear physics in particular) and compare it with the technologies we had to use for things after that you will find a huge gap. Maybe we are pushing the limits of our current technology but the problem is that we don't know. If we know then it wouldn't be research. The only way to find out if something is going to work is at least do R&D work and design a plan and maybe execute it. In the current scheme this is expensive projects. Although if you divide the cost per personnel you will find that it is comparable to other fields.
Lets not talk about how the R&D benefit whole industries because this is a cliche by now (although it is a valid argument in itself). But the only way to advance is to try new things. When people actually say a crisis in particle physics, they usually mean in terms that we did not discover the whole big new thing. But each day we gain more understanding on details of standard model and how interactions work. These things are not sexy enough to be reported by the mainstream media. I post some of these on HN and rarely they get any traction and I get why. It is usually hard to explain and very specialized. But this is not excuse for people to claim that we are not advancing.
When considering how vacuum tubes were used for a lot of discoveries in physics (nuclear physics in particular) and compare it with the technologies we had to use for things after that you will find a huge gap. Maybe we are pushing the limits of our current technology but the problem is that we don't know. If we know then it wouldn't be research. The only way to find out if something is going to work is at least do R&D work and design a plan and maybe execute it. In the current scheme this is expensive projects. Although if you divide the cost per personnel you will find that it is comparable to other fields.
Lets not talk about how the R&D benefit whole industries because this is a cliche by now (although it is a valid argument in itself). But the only way to advance is to try new things. When people actually say a crisis in particle physics, they usually mean in terms that we did not discover the whole big new thing. But each day we gain more understanding on details of standard model and how interactions work. These things are not sexy enough to be reported by the mainstream media. I post some of these on HN and rarely they get any traction and I get why. It is usually hard to explain and very specialized. But this is not excuse for people to claim that we are not advancing.
That sounds right to me. I don't work in physics, but have many colleagues who have left it for that reason.
~The said, I think Dune still does seem useful. Hard to say whether the cost overruns are a lack of good management or if the experiment is just a quagmire, though. Guess I lean towards the scientists here that it's the former~
EDIT:
Doing some research it seems the Hyper K experiment might just be better, and funding might be better directed there than Fermilab.
~The said, I think Dune still does seem useful. Hard to say whether the cost overruns are a lack of good management or if the experiment is just a quagmire, though. Guess I lean towards the scientists here that it's the former~
EDIT:
Doing some research it seems the Hyper K experiment might just be better, and funding might be better directed there than Fermilab.
Fermilab operated the Tevatron until the LHC was online. They then pivoted to emphasize neutrino research, which is perfectly doable with the linac and the main injector ring. There is a lot to learn about neutrinos. I certainly do not foresee any future where Fermilab is shuttered. There are always frontiers in physics worth exploring.
Vouch. Insightful, not negative.
A place like Fermilab should in theory be minting money making clever physics gadgets, machines, and services with real world value using 50% of its specialist scientific capacity. Once bootstrapped, this would fix so many of its problems to fund the other 50% that's fundamental research.
They're good at outreach. See their YT videos with Don Lincoln. More merch? A Patreon?
In a perfect world we could afford to have lots of basic research institutions that dream wild dreams 100% of the time. In the current context tho maybe it’s not crazy to ask for 10-20% of time/cost to be applied. That’s not the same as suggesting merch or a bake sale, especially if there’s a route to more sustainable self funding, which idea I would think is appealing to any scientific institution that’s beholden to a political one.
Besides, we have lots of urgent problems. Pure basic research is always going to be good/necessary, but whether it seems mostly noble or mostly ridiculous depends on whether the house is burning.
Besides, we have lots of urgent problems. Pure basic research is always going to be good/necessary, but whether it seems mostly noble or mostly ridiculous depends on whether the house is burning.
I'd say that a staggering amount of work at Fermilab is already applied and/or engineering. That's a compliment. Theorists get the PR but building and running those enormous experiments is no joke.
They need more than two or three bucks to run a lab.
Another roundup from Science magazine: https://www.science.org/content/article/trouble-and-strife-d...
And now instead of making a review of their decision and see the actual needs and think about cutting the reliance on RedHat, they went fully dependent on RHEL (one can say that CERN takes part of the blame too).