A 50-year quest: My personal journey with the second law of thermodynamics(writings.stephenwolfram.com)
writings.stephenwolfram.com
A 50-year quest: My personal journey with the second law of thermodynamics
https://writings.stephenwolfram.com/2023/02/a-50-year-quest-my-personal-journey-with-the-second-law-of-thermodynamics/
110 comments
I have found a new algorithm that is remarkably simple but seems capable of generating the entire written works of Stephen Wolfram (and possibly those not yet written).
while (true) {
if (random() % (random() % 19)) {
printf ("me ");
} else {
printf ("i ");
}
}Maybe someone can explain this to me: I (naively) interpret Wolframs theories as trying to generate the observed continuous rules of physics from discrete rules. Thus the value in his approach is that it investigates the possibility that space-time is discrete with the hope that this can reveal something novel about the universe.
Am I misunderstanding something?
I get the feeling that the criticism of his work is because he hasn’t found anything new yet, but that doesn’t mean his computational paradigm isn’t useful and/or simply interesting?
Am I misunderstanding something?
I get the feeling that the criticism of his work is because he hasn’t found anything new yet, but that doesn’t mean his computational paradigm isn’t useful and/or simply interesting?
I don't think that's right. He's trying to find a computational model that reproduces the rules of physics (including those of quantum mechanics, notably the infamous Born rule with its randomness) from a small set of computational laws. He expects this to also lead to a unification of GR with QM, fixing the biggest hole in physics that has eluded us.
One significant aspect of his program is also to restate all of the laws of physics in terms of computational transformations, instead of differential equations.
One significant aspect of his program is also to restate all of the laws of physics in terms of computational transformations, instead of differential equations.
Thanks for replying. I’m still a bit confused. Isn’t the main difference between his computational transformations and differential equations that they are discrete?
They are also an entirely different mathematical framework. I'm not even sure if Wolfram's model is discrete (there are continuous models of computation, for example) . But even if they were fully isomorphic, they still represent a different way of thinking about the problem.
Also, I believe he is looking for some base rules that are different from those of QM, thus leading to a theory that is not consistent with QM, but makes similar experimental predictions.
Also, I believe he is looking for some base rules that are different from those of QM, thus leading to a theory that is not consistent with QM, but makes similar experimental predictions.
Trying to find…
Yet here’s an entire book…
Yet here’s an entire book…
I haven’t read his books yet, just a few of his longer posts. I don’t have a good background in physics thus why I’m trying to test my understanding.
I think you're broadly right. Wolfram did some theoretical and computational physics at young age (numerical quantum chromodynamics), and in last decades he likes to speculate about how all fundamental physics (3D space, matter and basic physics laws governing them) may come out of some computational process based on a simple rule. His team does computational experiments and visualizations using discrete state machines governed by simple rules. It's looking for a way to invent the Simulation.
I would recommend starting with Part 1, rather than Part 2 (which was linked). Here is Part 1: https://writings.stephenwolfram.com/2023/02/computational-fo...
Wolfram's ability to create interesting visualizations is fantastic. But I seem to feel underwhelmed at the significance of his discovery.
In short, he says that the Second Law of Thermodynamics is a consequence of the fact that we as observers are computationally bounded and cannot perceive all the details in the the lower-level, computationally irreducible physical systems around us.
Yet this does not seem enormously different, or perhaps not different at all, than the standard account in statistical mechanics in which the observer is considered to be coarse-graining over the fine details of the system which are too complex to track.
What is new in Wolfram's approach other than using computation-related labels to describe the situation? For example, his 'computationally bounded' observer is just the standard 'coarse-graining', right?
Wolfram's ability to create interesting visualizations is fantastic. But I seem to feel underwhelmed at the significance of his discovery.
In short, he says that the Second Law of Thermodynamics is a consequence of the fact that we as observers are computationally bounded and cannot perceive all the details in the the lower-level, computationally irreducible physical systems around us.
Yet this does not seem enormously different, or perhaps not different at all, than the standard account in statistical mechanics in which the observer is considered to be coarse-graining over the fine details of the system which are too complex to track.
What is new in Wolfram's approach other than using computation-related labels to describe the situation? For example, his 'computationally bounded' observer is just the standard 'coarse-graining', right?
> I would recommend starting with Part 1, rather than Part 2 (which was linked)
I think part 2 is much more interesting. If anyone starts with part 1 and decides to give up on it I would recommend that they give part 2 a chance. The subject and the style are quite different.
I think part 2 is much more interesting. If anyone starts with part 1 and decides to give up on it I would recommend that they give part 2 a chance. The subject and the style are quite different.
You are right. There is nothing new here.
This has to be the smartest person I have ever seen in terms of precocity. I knew his background was exceptional but at 13 he was essential an expert at physics. damn impressive. Imagine if he was born in the 2000s and having access to the internet and cheap computing. Up there with Von Nuemann..another one.
I think that underestimates the massive amount of distractions a person had to deal with being born in 2000.
I can remember being bored out of my mind and so practicing Bach on guitar for hours in 1990. There is just no way I would have done that in 2015. That feeling was gone forever the first time my modem connected to the internet.
It wouldn't be shocking that the intellectual giants of old became so because they had nothing much else to do but read and think.
I do put Wolfram in the same league as Von Nuemann in terms of not being ashamed to admit they are beyond me. I can distinctly remember getting New Kind of Science from the library, getting home and then quickly realizing there is no chance in hell I can read a 1000 pages of this.
I can remember being bored out of my mind and so practicing Bach on guitar for hours in 1990. There is just no way I would have done that in 2015. That feeling was gone forever the first time my modem connected to the internet.
It wouldn't be shocking that the intellectual giants of old became so because they had nothing much else to do but read and think.
I do put Wolfram in the same league as Von Nuemann in terms of not being ashamed to admit they are beyond me. I can distinctly remember getting New Kind of Science from the library, getting home and then quickly realizing there is no chance in hell I can read a 1000 pages of this.
Keep in mind that you are not reading the account of an impartial outside observer, but of the subject himself.
We think von Neumann was smart not because von Neumann himself said so.
We think von Neumann was smart not because von Neumann himself said so.
While Wolfram is objectively crazy smart, I don’t think he’s on the level of von Neumann.
You seem to be agreeing with what I wrote.
No no no.
I recommend reading part 3 at least. It will show how we came to take the second law as a theorem without ever having proved it formally.
Read it the following order:
1. https://writings.stephenwolfram.com/2023/01/how-did-we-get-h...
2. https://writings.stephenwolfram.com/2023/02/computational-fo...
3. Skip https://writings.stephenwolfram.com/2023/02/a-50-year-quest-... if you are tight on time. It helps tying things between 1 & 2.
Given the context, the thesis gives sound foundations to the second law in terms of computation. It ties in remarkably well with Cellular Automata, and feels like "how did I not see this?".
Here is my first attempt at summarising the whole thing (badly): The Second Law holds when systems are computationally irreducible for computationally bounded observers. The applicability changes based on the observers' computational capacities.
Though it might seem like "it is obvious" to many, the restatement and interpretation of the second law is quite novel. The claim is that the second law is in fact a property of computational universe.
Myself not being a physicist, I don't know how much and how well it ties with Physics Project. From a shallow perspective I am thinking "IF the universe is computational...". I have talked with some people who told me that without having established Physics Project as foundation, some interpretations of this thesis might be a bit stretched. But that is to be expected since the Physics Project is "his bet".
In any case, it seems to me that the thesis presented in https://writings.stephenwolfram.com/2023/02/computational-fo... stands on its own for all of its core ideas.
Read it the following order:
1. https://writings.stephenwolfram.com/2023/01/how-did-we-get-h...
2. https://writings.stephenwolfram.com/2023/02/computational-fo...
3. Skip https://writings.stephenwolfram.com/2023/02/a-50-year-quest-... if you are tight on time. It helps tying things between 1 & 2.
Given the context, the thesis gives sound foundations to the second law in terms of computation. It ties in remarkably well with Cellular Automata, and feels like "how did I not see this?".
Here is my first attempt at summarising the whole thing (badly): The Second Law holds when systems are computationally irreducible for computationally bounded observers. The applicability changes based on the observers' computational capacities.
Though it might seem like "it is obvious" to many, the restatement and interpretation of the second law is quite novel. The claim is that the second law is in fact a property of computational universe.
Myself not being a physicist, I don't know how much and how well it ties with Physics Project. From a shallow perspective I am thinking "IF the universe is computational...". I have talked with some people who told me that without having established Physics Project as foundation, some interpretations of this thesis might be a bit stretched. But that is to be expected since the Physics Project is "his bet".
In any case, it seems to me that the thesis presented in https://writings.stephenwolfram.com/2023/02/computational-fo... stands on its own for all of its core ideas.
He tried his best with an intriguing idea (a discrete, information theoretic basis for the physical universe). By all accounts the program did not deliver. At two levels: first at the obvious target, reinterpeting the existing body of knowledge and possibly predicting and explaining new things.
But also important, the program did not produce mental tools (e.g., mathematical structures or other concepts) that maybe somebody else might adopt and try a new, more productive, approach.
This doesnt mean the original idea will forever fail to be relevant. (Though other "deep insights" of Wheeler, like geometrodynamics, also did not pan out.)
The life of ideas evolving in our collective mental spaces is poorly correlated with the biological lifespans of the 'carriers' and even more so with their human needs and feelings.
But also important, the program did not produce mental tools (e.g., mathematical structures or other concepts) that maybe somebody else might adopt and try a new, more productive, approach.
This doesnt mean the original idea will forever fail to be relevant. (Though other "deep insights" of Wheeler, like geometrodynamics, also did not pan out.)
The life of ideas evolving in our collective mental spaces is poorly correlated with the biological lifespans of the 'carriers' and even more so with their human needs and feelings.
> Curiously enough, looking at the numbers now, I realize that the base speed of the LARC was only 20x the Elliott 903C, though with floating point, etc.—a factor that pales in comparison with the 500x speedup in computers in the 40 years since I started working on cellular automata.
500x? Shouldn’t the speedup over 40 years be something between 10,000x (increase in single-threaded performance) and 1,000,000x (increase in number of transistors).
500x? Shouldn’t the speedup over 40 years be something between 10,000x (increase in single-threaded performance) and 1,000,000x (increase in number of transistors).
Wolfram is right to draw attention to a puzzle in physics: why does the universe as a whole seem to be getting more ordered, but the 2nd law of thermodynamics suggests it should become more disordered?
Julian Barbour's solution to this puzzle is to point out that the second law only applies to a system 'in a box'. The universe is not 'in a box' and so the second law doesn't apply to the universe as a whole.
Julian Barbour's solution to this puzzle is to point out that the second law only applies to a system 'in a box'. The universe is not 'in a box' and so the second law doesn't apply to the universe as a whole.
What makes you think that the universe is getting more ordered? It might look that way if you look at visible stuff and ignore the waste heat. Stars are giant entropy sources throwing out light and plasma. On planets, that light can be used for work, making life, but nearly all of it ends up as heat.
Also, the 2nd law of thermodynamics is only about thermodynamics. Other forces, like gravity squishing things or strong force making fusion can provide order and extra energy. But it is temporary, there will eventually be no more gas to collect and fuse. Then all there is cooling off into the void.
Also, the 2nd law of thermodynamics is only about thermodynamics. Other forces, like gravity squishing things or strong force making fusion can provide order and extra energy. But it is temporary, there will eventually be no more gas to collect and fuse. Then all there is cooling off into the void.
> At that point, things went crazy. There was talk of Nobel Prizes (I wasn’t buying it). There were official complaints from the French embassy about French scientists not being adequately recognized. There was upset at Thinking Machines for not even being mentioned. And, yes, as the originator of the idea, I was miffed that nobody seemed to have even suggested contacting me—even if I did view the rather breathless and “geopolitical” tenor of the article as being pretty far from immediate reality.
> At the time, everyone involved denied having been responsible for the appearance of the article. But years later it emerged that the source was a certain John Gage, former political operative and longtime marketing operative at Sun Microsystems, who I’d known since 1982, and had at some point introduced to Brosl Hasslacher. Apparently he’d called around various government contacts to help encourage open (international) sharing of scientific code, quoting this as a test case.
> But as it was, the article had pretty much exactly the opposite effect, with everyone now out for themselves. In Princeton, I’d interacted with Steve Orszag, whose funding for his new (traditional) computational fluid dynamics company, Nektonics, now seemed at risk, and who pulled me into an emergency effort to prove that cellular automaton fluid dynamics couldn’t be competitive. (The paper he wrote about this seemed interesting, but I demurred on being a coauthor.) Meanwhile, Thinking Machines wanted to file a patent as quickly as possible. Any possibility of the French government getting a Connection Machine evaporated and soon Brosl Hasslacher was claiming that “the French are faking their data”.
Yet again, one mustn't go against the "american exceptionalism"[1] dogma, that's sad to hear The result of this ego-centric americanism; Russia able to develop their supersonic weapons before anyone in the west, thanks to the work of the silenced non-US scientists [2]
[1] - https://en.wikipedia.org/wiki/American_exceptionalism
[2] - https://www.youtube.com/watch?v=Jn8b3E9oUHY
> At the time, everyone involved denied having been responsible for the appearance of the article. But years later it emerged that the source was a certain John Gage, former political operative and longtime marketing operative at Sun Microsystems, who I’d known since 1982, and had at some point introduced to Brosl Hasslacher. Apparently he’d called around various government contacts to help encourage open (international) sharing of scientific code, quoting this as a test case.
> But as it was, the article had pretty much exactly the opposite effect, with everyone now out for themselves. In Princeton, I’d interacted with Steve Orszag, whose funding for his new (traditional) computational fluid dynamics company, Nektonics, now seemed at risk, and who pulled me into an emergency effort to prove that cellular automaton fluid dynamics couldn’t be competitive. (The paper he wrote about this seemed interesting, but I demurred on being a coauthor.) Meanwhile, Thinking Machines wanted to file a patent as quickly as possible. Any possibility of the French government getting a Connection Machine evaporated and soon Brosl Hasslacher was claiming that “the French are faking their data”.
Yet again, one mustn't go against the "american exceptionalism"[1] dogma, that's sad to hear The result of this ego-centric americanism; Russia able to develop their supersonic weapons before anyone in the west, thanks to the work of the silenced non-US scientists [2]
[1] - https://en.wikipedia.org/wiki/American_exceptionalism
[2] - https://www.youtube.com/watch?v=Jn8b3E9oUHY
I disagree with the disdainful posts about Wolfram here. Sounds like insider vs. outsider politics, jealousy, or lack of actually reading and understanding Wolfram's work. His recent Physics Project is fascinating and original - and seems to be converging on some interesting discoveries.
It's been 20 years since NKS; please let us know in 2044 how you still feel about the "seems to be converging on" part?
(if you want a connection between mathematical physics and informatics, I'd suggest looking at quantales before cellular automata)
Lagniappe: https://www.youtube.com/watch?v=i6rVHr6OwjI
(if you want a connection between mathematical physics and informatics, I'd suggest looking at quantales before cellular automata)
Lagniappe: https://www.youtube.com/watch?v=i6rVHr6OwjI
Personally, I find it just takes the willingness to actively separate his exposition of the concepts (whether novel or not) from his self-evaluation.
It helps me to keep in mind his own admission of being quite egocentric (something like: "After all I am part of the club of those that named their company after themselves").
It helps me to keep in mind his own admission of being quite egocentric (something like: "After all I am part of the club of those that named their company after themselves").
Does an admission of guilt make a thief more palatable? Even if he keeps thieving?
Yeah, I guess that's the kind of stuff a boy had to back in the day before they invented tiktok...
There is no question that Wolfram is gifted for math and physics but his reputation for exaggerating claims is well known by now, and hearing him casually say that he was mulling over the second law of thermodynamics at age 12 makes me go "Yeah, right".
I have met several people telling weaker versions of that or similar claims. I believe them. A more accessible example is to intuit fairly early what computational irreducibility is. Typically one doesn't develop the language to be precise about these ideas until much later. That Wolfram had the correct language available at that age doesn't strike me as unlikely.
That claim seems like not a stretch at all, he had published quantum physics papers at 15.
this link is
> part 2 in a 3-part series about the Second Law:
> 1. Computational Foundations for the Second Law of Thermodynamics
> 2. A 50-Year Quest: My Personal Journey with the Second Law of Thermodynamics
> 3. How Did We Get Here? The Tangled History of the Second Law of Thermodynamics
conundrum:
1. can i win by starting with part 2?
2. will I have a better chance of breaking even if I start with part 1?
https://writings.stephenwolfram.com/2023/02/computational-fo...
3. do I have to read part 3?
time to find out...
> part 2 in a 3-part series about the Second Law:
> 1. Computational Foundations for the Second Law of Thermodynamics
> 2. A 50-Year Quest: My Personal Journey with the Second Law of Thermodynamics
> 3. How Did We Get Here? The Tangled History of the Second Law of Thermodynamics
conundrum:
1. can i win by starting with part 2?
2. will I have a better chance of breaking even if I start with part 1?
https://writings.stephenwolfram.com/2023/02/computational-fo...
3. do I have to read part 3?
time to find out...
You could help everyone else win by letting us know of your findings.
It sounds like you already know the answer!
You can't win. You can't break even. And you can't get out of the game.
You can't win. You can't break even. And you can't get out of the game.
for n00bs who don't get the joke but will enjoy it, the 3 laws of thermodynamics, statistical mechanics, have famously been paraphrased as a casino where
[maybe think of this like building a perpetual motion machine]
(1) you can't win [if you build a perpetual motion machine, don't expect it to generate extra power to run other things]
(2) you can't break even [you can't even build a perpetual motion machine cuz friction etc.]
(3) you have to play the game [your entire life is like a failed perpetual motion machine, you only survive by killing other things and even so you're going to run down and die, along with the rest of the universe]
but it suffuses everything, more than just perpetual motion machines, like if your coffee water has sugar dissolved in it, don't expect it to climb out and reform crystals, and if you have sugar crystals, it's just going to dissolve itself in humidity in the air, and you can neither stop nor make these things happen. If you think you can, if you figure out a way to get the chaotic dissolved sugar back out of the water as organized crystals (rock candy!), that only happens if you have created even more chaos (evaporating the liquid water, burning carbon to heat the water, etc.) along the way.
[maybe think of this like building a perpetual motion machine]
(1) you can't win [if you build a perpetual motion machine, don't expect it to generate extra power to run other things]
(2) you can't break even [you can't even build a perpetual motion machine cuz friction etc.]
(3) you have to play the game [your entire life is like a failed perpetual motion machine, you only survive by killing other things and even so you're going to run down and die, along with the rest of the universe]
but it suffuses everything, more than just perpetual motion machines, like if your coffee water has sugar dissolved in it, don't expect it to climb out and reform crystals, and if you have sugar crystals, it's just going to dissolve itself in humidity in the air, and you can neither stop nor make these things happen. If you think you can, if you figure out a way to get the chaotic dissolved sugar back out of the water as organized crystals (rock candy!), that only happens if you have created even more chaos (evaporating the liquid water, burning carbon to heat the water, etc.) along the way.
I've heard it as :
1) You can't win
2) In a perfect universe you could break even
3) The universe isn't perfect
TLDR (using Kagi): https://labs.kagi.com/ai/sum?url=https://writings.stephenwol...
I don’t follow everything he figured out / discussed here. But the man is definitely a genius, and through Mathematica alone has had an immense impact on science. Appreciate the link, it was refreshing to read a bit of his life.
If you don't follow, how can you tell that he is definitely a genius [sic] ?
This is a dangerous pattern of thinking that led to scams like FTX or Bernie Madoff.
This is a dangerous pattern of thinking that led to scams like FTX or Bernie Madoff.
I have trouble reconciling the insanely impressive pre-mid-80s Wolfram with the post-80s Wolfram. All his productions before a certain time are touched with brilliance and a preternatural orderliness of mind (including Mathematica, which was a groundbreaking and beautiful piece of software). Then he started making pictures of cellular automata.
I think he mistakenly decided, reinforced by a lot of outside feedback, that he alone was personally responsible for his ability to respond to and record brilliant ideas.
He failed to recognize that he was primarily exceptional in his preparation for, and early participation in, a brilliant collective of individuals. When he left the collective, his limited individual capacity left him chasing instead of leading.
He failed to recognize that he was primarily exceptional in his preparation for, and early participation in, a brilliant collective of individuals. When he left the collective, his limited individual capacity left him chasing instead of leading.
ego + no high quality feedback -> intellectual doom
Guy is trapped inside his own mind and can't get out. Even independent researchers need scientific community to spar with. If you can't explain your thought simply to your peers (followers don't count) maybe you are lost.
Same thing with Erik Weinstein. Left academia early to do something else, became rich. When he finally published the outline of the great thing he had been working for decades, someone easily spotted the errors and that was it.
Guy is trapped inside his own mind and can't get out. Even independent researchers need scientific community to spar with. If you can't explain your thought simply to your peers (followers don't count) maybe you are lost.
Same thing with Erik Weinstein. Left academia early to do something else, became rich. When he finally published the outline of the great thing he had been working for decades, someone easily spotted the errors and that was it.
Yeah he never talks about his theory anymore. Amazing how sudden that went from 100 to 0. Same for him applying gauge theory to the CPI, which also went splat and never to be spoken of again. The reality is, these concepts have engendered a significant and dense literature .The odds of being an outsider and just upending it are close to nil, even if you are brilliant. Sure Einstein did, how often does that happen?
Einstein was not isolated or outsider genius. He was a young physicist who didn't get academic position.
Einstein's best friend Marcel Grossmann was a mathematician specializing in differential geometry and tensor calculus. Those were exactly the tools he needed for his breakthrough. Grossmann was important facilitator for Einstein. There were others. "Olympia Academy" and his wife for example.
Einstein's best friend Marcel Grossmann was a mathematician specializing in differential geometry and tensor calculus. Those were exactly the tools he needed for his breakthrough. Grossmann was important facilitator for Einstein. There were others. "Olympia Academy" and his wife for example.
Newton added "indigo" to the rainbow because of an occult belief in numerology and a divine purpose of the number 7. We all have our faults.
Searching for Indigo : https://www.youtube.com/watch?v=Iyyg8pkDL74
Oliver Sacks experienced Indigo on ahem some substances.
Oliver Sacks experienced Indigo on ahem some substances.
the cellular automata per se are not the problem; there is definitely a field of physics studying the properties of CA systems and whether they can be used to derive physical laws. the problem is that wolfram is trying to develop some grand theory of everything by himself in the form of a stone tablet brought down from a mountain, rather than running his ideas by other scientists for peer review and pushback, which is how you get cranks.
Wolfram was a child prodigy, became successful businessman and created Mathematica. He just desperately wants to be more.
What is left of him as a scientist is just clever crank and eternal wannabe. All his books and projects in the last 20 years, from "A new kind of science" onward are just obfuscating, hand-waving and playing with "digital mud" getting nowhere.
Excellent visualizations of overly complex things hide the fact that he finds no deep new science or explanations.
What is left of him as a scientist is just clever crank and eternal wannabe. All his books and projects in the last 20 years, from "A new kind of science" onward are just obfuscating, hand-waving and playing with "digital mud" getting nowhere.
Excellent visualizations of overly complex things hide the fact that he finds no deep new science or explanations.
What is left of him as a scientist is just clever crank and eternal wannabe. All his books and projects in the last 20 years, from "A new kind of science" onward are just obfuscating, hand-waving and playing with "digital mud" getting nowhere.
Yeah I am as unimpressed by his automata theory now as I was 20 years ago.
Yeah I am as unimpressed by his automata theory now as I was 20 years ago.
This is the feeling I have every time I read Wolfram's writing. It seems deep when first approached, but is ultimately a superficial rewording of established knowledge.
After listening to a podcast where he explains his approach in more depth, I think the value is in that he [tries to] construct the ideas from scratch. That they sound like “superficial rewording” is, I think, great evidence that the knowledge is either fundamentally true or the best humans can do.
It’s like when avionics use independent systems to make decisions. If all the systems agree, they’re probably right. The value is in independent parallel construction of the same result.
It’s like when avionics use independent systems to make decisions. If all the systems agree, they’re probably right. The value is in independent parallel construction of the same result.
Thank you for this. I don’t see many generous readings of Wolframs writings. HN is very dismissive of almost everything of his that ever makes the front page.
As someone who doesn’t know enough to really know if he’s right or wrong, it’s hard to fully believe the HN view that he’s a transparent Charleston (and easy to spot at that) considering his history.
It’s helpful to see a “positive” read of him and gives me more to think about so thank you!
Can you link/mention the podcast?
As someone who doesn’t know enough to really know if he’s right or wrong, it’s hard to fully believe the HN view that he’s a transparent Charleston (and easy to spot at that) considering his history.
It’s helpful to see a “positive” read of him and gives me more to think about so thank you!
Can you link/mention the podcast?
I don’t see many generous readings of Wolframs writings. HN is very dismissive of almost everything of his that ever makes the front page.
This sounds like the scepticism here is unfair.
You need to consider that if somebody establishes a reputation for using their reasonably well-known (and justifiably respected) position as an academic software developer to publicise dubious and grandiose scientific theories to gullible techies and pop-sci readers, rather than submitting them to be evaluated by experts through the normal scientific process, it seems pretty miniscule beer in comparison to have some pushback in threads like this.
This sounds like the scepticism here is unfair.
You need to consider that if somebody establishes a reputation for using their reasonably well-known (and justifiably respected) position as an academic software developer to publicise dubious and grandiose scientific theories to gullible techies and pop-sci readers, rather than submitting them to be evaluated by experts through the normal scientific process, it seems pretty miniscule beer in comparison to have some pushback in threads like this.
It’s not that hard to understand why the lisp guy who became a billionaire by making m-expressions work, but kept the tech proprietary, is resented by many here.
Who else has ever achieved that level of success on the back of a programming language, let alone a Lisp? Our host here is the only one in the same ballpark and his success was more on the application side. Arc is no Wolfram language.
Who else has ever achieved that level of success on the back of a programming language, let alone a Lisp? Our host here is the only one in the same ballpark and his success was more on the application side. Arc is no Wolfram language.
Honestly, this is the first time I've ever heard of m-expressions. I know of Wolfram from Mathematica and he has HUGE credit in my mind for building that.
He was recently on Tim Ferris and discussed his ideas.
Please let's try to avoid the predictable thing that people have been reflexively saying about Wolfram for as long as this site has existed (and at least a decade before that too). Maybe he deserves it, maybe not, but anything so predictable has extremely low signal and therefore is off topic for this site.
Think of this as a stretch exercise for HN.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
(We detached this subthread from https://news.ycombinator.com/item?id=34658960.)
Think of this as a stretch exercise for HN.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
(We detached this subthread from https://news.ycombinator.com/item?id=34658960.)
Wolfram's attitude isn't just a personality issue, it actually violates the norms of science, which exist for good reason.
When Wolfram writes a long screed about his own genius as an introduction to his work that's a claim to intellectual authority. Scientific work isn't supposed to be evaluated based on the authority of its author, because that's not a good guide to whether it is valid. As I understand it (I haven't personally checked) Wolfram doesn't submit any of his stuff to peer review either.
When Wolfram writes a long screed about his own genius as an introduction to his work that's a claim to intellectual authority. Scientific work isn't supposed to be evaluated based on the authority of its author, because that's not a good guide to whether it is valid. As I understand it (I haven't personally checked) Wolfram doesn't submit any of his stuff to peer review either.
As another poster put it perhaps Dr Wolfram has chosen to work outside the community recently. The early cellular automata papers were published in peer reviewed journals, and the OA contains an anecdote about the external refereeing of one short paper. I read the early autobiographical material as just an older man recording his experiences. A memoir perhaps rather than a review paper.
Your post has me thinking about 'independent scientists' more generally.
Some defy characterisation as their work is just outside normal categories. An example could be George Spencer-Brown with his Laws of Form, a book that appeared as a complete and dense argument. Spencer-Brown was a colourful character to say the least.
Others are sort of 'semi-detached' and do publish papers in mainstream academic journals and do collaborate. Julian Barbour would be my example here.
I would imagine that a fair amount of self-confidence is needed (along with resources) to work in this way.
Your post has me thinking about 'independent scientists' more generally.
Some defy characterisation as their work is just outside normal categories. An example could be George Spencer-Brown with his Laws of Form, a book that appeared as a complete and dense argument. Spencer-Brown was a colourful character to say the least.
Others are sort of 'semi-detached' and do publish papers in mainstream academic journals and do collaborate. Julian Barbour would be my example here.
I would imagine that a fair amount of self-confidence is needed (along with resources) to work in this way.
Maybe so, but from a HN point of view, avoiding predictable repetition (especially when it's bilious) is the high order bit.
There's a great deal of similar practice in other places too but the inverted cult of personality in Wolfram threads is sui generis, I think. I tried once to make the case that it's a mirror image of the the thing it's criticizing, but IIRC that didn't go over too well.
I suppose the cynically exaggerated version of this would be to say bad science is fine as long as it doesn't lead to crap threads. Note that I didn't say that.
There's a great deal of similar practice in other places too but the inverted cult of personality in Wolfram threads is sui generis, I think. I tried once to make the case that it's a mirror image of the the thing it's criticizing, but IIRC that didn't go over too well.
I suppose the cynically exaggerated version of this would be to say bad science is fine as long as it doesn't lead to crap threads. Note that I didn't say that.
@dang Would you mind telling us what opinions about which famous people we are not allowed to express becacause they are too tropy? It would greatly help in the moderation of this site.
However, it could be that common opinions about authors are not tropes but warranted. In the first decade of my exposure to Stephen Wolfram, I was in awe of his intelligence. Mathematica is a wonderful tool and its step-by-step explanations of proofs are fantastic. I thought his approach to computational systems was mind-blowing.
Then I leaned more. I saw that much of his work was built on the backs of giants. His cellular automota are simplified versions of what Conroy worked on. Don't get me wrong, Wolfram formalized 1D automata in his own works, but it wasn't revolutionary. He didn't "discover" these features; he built upon the works of others.
My biggest critique of Wolfram is that he will come up with SOME new interpretation of a physical or computation phenomenon and claim that it is THE ONE TRUE INTEPERATION of how the world works. He feels like hubris personified.
But to simplify it, I think he is a smart person who often posts self-congratulatory things and I don't thing it's wrong to call him out when his writings are full of fluff.
However, it could be that common opinions about authors are not tropes but warranted. In the first decade of my exposure to Stephen Wolfram, I was in awe of his intelligence. Mathematica is a wonderful tool and its step-by-step explanations of proofs are fantastic. I thought his approach to computational systems was mind-blowing.
Then I leaned more. I saw that much of his work was built on the backs of giants. His cellular automota are simplified versions of what Conroy worked on. Don't get me wrong, Wolfram formalized 1D automata in his own works, but it wasn't revolutionary. He didn't "discover" these features; he built upon the works of others.
My biggest critique of Wolfram is that he will come up with SOME new interpretation of a physical or computation phenomenon and claim that it is THE ONE TRUE INTEPERATION of how the world works. He feels like hubris personified.
But to simplify it, I think he is a smart person who often posts self-congratulatory things and I don't thing it's wrong to call him out when his writings are full of fluff.
No, I can't give you a list of repetitive tropes. Suppose I missed one?
I'll tell you one thing though: anytime someone uses the phrase "call him out", that's a sign the needle is in the red, or at least twitching to get there.
The thing to realize is that even if you're 100% right, it doesn't make repetitive threads any less tedious. Avoiding tedium is more important than being right—at least on HN. No doubt other websites, with other mandates, have other priorities.
I'll tell you one thing though: anytime someone uses the phrase "call him out", that's a sign the needle is in the red, or at least twitching to get there.
The thing to realize is that even if you're 100% right, it doesn't make repetitive threads any less tedious. Avoiding tedium is more important than being right—at least on HN. No doubt other websites, with other mandates, have other priorities.
I am ever more impressed with Wolfram's ability to obfuscate and self-aggrandize.
The second law is very easy to understand when presented as Boltzmann formulated it, and Shannon later generalized it: the entropy of a system is the (log of the) number of states that system could possibly be in given a set of values we can actually measure. In a gas, for example, we can measure its volume, temperature, and pressure, but that still leaves a lot of possible actual underlying configurations of the gas particles. The more possible configurations there are, the greater the entropy. And this explains the second law: given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.
That's really all there is to it on a conceptual level.
The second law is very easy to understand when presented as Boltzmann formulated it, and Shannon later generalized it: the entropy of a system is the (log of the) number of states that system could possibly be in given a set of values we can actually measure. In a gas, for example, we can measure its volume, temperature, and pressure, but that still leaves a lot of possible actual underlying configurations of the gas particles. The more possible configurations there are, the greater the entropy. And this explains the second law: given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.
That's really all there is to it on a conceptual level.
Well you're giving the statistical mechanics explanation (or information theory), which some thermodynamicists don't appreciate so much, but... I totally agree. Even in person we would joke that his ego filled an entire room, and that's before he got famous. Thank you for reading it so I didn't have to.
> some thermodynamicists don't appreciate so much
Why?
Why?
You do not need statistical physics to define entropy. In fact, entropy as a concept was introduced long before statistical physics was a thing. Being able to do thermodynamics without invoking anything related to statistical physics is very valuable tool in mathematical physics, as it leads to definitions and proofs that rely on way fewer assumptions. People should not claim they understand entropy if they understand only the statistical physics way of introducing it. The deep value of the concept of entropy comes from understanding how the two completely unrelated definitions (in thermodynamics and in statistical physics) are equivalent.
Edit to add: In statistical physics you rely on the idea of equivalent microstates. In thermodynamics you do not introduce anything about the microscopic structure of the system under study (an extremely valuable way to generalize your results to cases not covered by typical stat physics assumptions), rather you introduce the zeroth law of thermodynamics as an axiom.
Edit to add: In statistical physics you rely on the idea of equivalent microstates. In thermodynamics you do not introduce anything about the microscopic structure of the system under study (an extremely valuable way to generalize your results to cases not covered by typical stat physics assumptions), rather you introduce the zeroth law of thermodynamics as an axiom.
> entropy as a concept was introduced long before statistical physics was a thing
That's true, but the historical development doesn't always make for the best pedagogy. Quantum mechanics, to take my favorite example, is vastly simpler than it is commonly made out to be [1]. It just took about 80 years for this to be understood.
The development of thermodynamics followed a similar path because thermodynamics predated the widespread acceptance of the atomic theory. Indeed, statistical mechanics played a major role in getting the atomic theory accepted. But it does not follow that the thermodynamic treatment is superior in any way. I challenge you to explain to me what entropy actually is in fewer words than I used without reference to atoms or microstates. I'll bet you can't even explain what "temperature" is under those constraints, and that is much simpler.
---
[1] See https://en.wikipedia.org/wiki/Quantum_Computing_Since_Democr... chapter 9
That's true, but the historical development doesn't always make for the best pedagogy. Quantum mechanics, to take my favorite example, is vastly simpler than it is commonly made out to be [1]. It just took about 80 years for this to be understood.
The development of thermodynamics followed a similar path because thermodynamics predated the widespread acceptance of the atomic theory. Indeed, statistical mechanics played a major role in getting the atomic theory accepted. But it does not follow that the thermodynamic treatment is superior in any way. I challenge you to explain to me what entropy actually is in fewer words than I used without reference to atoms or microstates. I'll bet you can't even explain what "temperature" is under those constraints, and that is much simpler.
---
[1] See https://en.wikipedia.org/wiki/Quantum_Computing_Since_Democr... chapter 9
You are right about pedagogy. But "understanding" requires much more than giving one simple definition without discussing the consequences of the definition.
Your (statistical) definition of entropy is perfectly reasonable and fairly intuitive and it should be taught first (as you alluded twice and as it is relatively common these days). But "understanding" entropy involves understanding multiple facets of it and why it is useful, not just knowing the easier definitions. Hence, it seem unreasonable to claim "understanding" entropy without having familiarity with the independent thermodynamic definition of entropy and *why are they equivalent*. That would be like saying one understands programming language theory while being familiar only with object oriented programming.
Using atoms and microstates to explain entropy makes the explanation shorter but it also makes it *more* complex because it now depends on the definition of *atoms and microstates*. The thermodynamics explanation of entropy does not actually require more words, but crucially, it requires fewer concepts (heat, useful work, and heat transfer).
Concerning the development of thermodynamics: defining entropy *without* statistical physics requires *fewer* assumptions about the physical system. Statistical physics did not make thermodynamics simpler, nor is it necessary to explain thermodynamics. To your example about quantum mechanics: this is different because statistical physics is not some better more-fundamental explanation of thermodynamics. The two are separate sets of axioms with equivalent explanatory power.
Concerning the definition of temperature: are you aware that there is an implicit assumption about temperature scales when defined in the statistical physics way? That is much more explicit in the thermodynamic treatment. And it is why I brought up the zeroth law of thermodynamics: it is where temperature is defined. If you think temperature is easier to define in the statistical physics setting, you are simply hiding a ton of assumptions under the rug, which is not a very good way to build a mathematical theory.
I am curious what do you claim is a simple statistical explanation of temperature? If it is something about average energy, that is a *wrong* explanation of temperature, valid in only a few special circumstances.
As to actually giving you the thermodynamic definition of temperature: The joke is that it is whatever the thermometer says it is. The rigorous version of that joke is: According to the zeroth law of thermodynamics, if bodies A and B are in thermal equilibrium (no heat transfer) and bodies B and C are in thermal equilibrium, then A and C are also in thermal equilibrium. Use this equivalence property to tag all bodies that belong to the same equivalence class. This tag is your measure of temperature. It is an arbitrary (ordered) scale. If you want to, you can then use the second law of thermodynamics to pick a neater canonical scale.
Finally, the definition of entropy: enumerate the (macroscopic) variables describing your system and draw them as a coordinate system; compute the value of ΔQ/T as you travel through some path in that coordinate system; the second law of thermodynamics says that value is zero on a closed path, thus we have a potential function; name that potential function entropy.
Mathematically, in many aspects that is both a simpler and a more powerful definition. One could hardly claim to understand entropy if they do not understand the enormity of the previous paragraph and its relationship to the independent statistical definition.
TLDR: "Understanding" entropy is about understanding why these vastly different definitions (thermodynamic and statistical) are equivalent, not about being able to recite one or the other. Especially given that statistical physics is not more fundamental than thermodynamics (unlike special relativity which is indeed more fundamental than Newtonian gravity)
Your (statistical) definition of entropy is perfectly reasonable and fairly intuitive and it should be taught first (as you alluded twice and as it is relatively common these days). But "understanding" entropy involves understanding multiple facets of it and why it is useful, not just knowing the easier definitions. Hence, it seem unreasonable to claim "understanding" entropy without having familiarity with the independent thermodynamic definition of entropy and *why are they equivalent*. That would be like saying one understands programming language theory while being familiar only with object oriented programming.
Using atoms and microstates to explain entropy makes the explanation shorter but it also makes it *more* complex because it now depends on the definition of *atoms and microstates*. The thermodynamics explanation of entropy does not actually require more words, but crucially, it requires fewer concepts (heat, useful work, and heat transfer).
Concerning the development of thermodynamics: defining entropy *without* statistical physics requires *fewer* assumptions about the physical system. Statistical physics did not make thermodynamics simpler, nor is it necessary to explain thermodynamics. To your example about quantum mechanics: this is different because statistical physics is not some better more-fundamental explanation of thermodynamics. The two are separate sets of axioms with equivalent explanatory power.
Concerning the definition of temperature: are you aware that there is an implicit assumption about temperature scales when defined in the statistical physics way? That is much more explicit in the thermodynamic treatment. And it is why I brought up the zeroth law of thermodynamics: it is where temperature is defined. If you think temperature is easier to define in the statistical physics setting, you are simply hiding a ton of assumptions under the rug, which is not a very good way to build a mathematical theory.
I am curious what do you claim is a simple statistical explanation of temperature? If it is something about average energy, that is a *wrong* explanation of temperature, valid in only a few special circumstances.
As to actually giving you the thermodynamic definition of temperature: The joke is that it is whatever the thermometer says it is. The rigorous version of that joke is: According to the zeroth law of thermodynamics, if bodies A and B are in thermal equilibrium (no heat transfer) and bodies B and C are in thermal equilibrium, then A and C are also in thermal equilibrium. Use this equivalence property to tag all bodies that belong to the same equivalence class. This tag is your measure of temperature. It is an arbitrary (ordered) scale. If you want to, you can then use the second law of thermodynamics to pick a neater canonical scale.
Finally, the definition of entropy: enumerate the (macroscopic) variables describing your system and draw them as a coordinate system; compute the value of ΔQ/T as you travel through some path in that coordinate system; the second law of thermodynamics says that value is zero on a closed path, thus we have a potential function; name that potential function entropy.
Mathematically, in many aspects that is both a simpler and a more powerful definition. One could hardly claim to understand entropy if they do not understand the enormity of the previous paragraph and its relationship to the independent statistical definition.
TLDR: "Understanding" entropy is about understanding why these vastly different definitions (thermodynamic and statistical) are equivalent, not about being able to recite one or the other. Especially given that statistical physics is not more fundamental than thermodynamics (unlike special relativity which is indeed more fundamental than Newtonian gravity)
> Concerning the definition of temperature: are you aware that there is an implicit assumption about temperature scales when defined in the statistical physics way?
What’s wrong with the usual arguments for the identification of beta as being (proportional to) the inverse of the thermodynamic temperature? You mean that it’s based on an arbitrary choice of constant to identify the entropies?
What’s wrong with the usual arguments for the identification of beta as being (proportional to) the inverse of the thermodynamic temperature? You mean that it’s based on an arbitrary choice of constant to identify the entropies?
Mostly yes (but we can probably debate semantics). It is not at all obvious that the inverse beta should be proportional to what we conventionally call temperature. One needs to prove that.
Once we define the entropy its integrating factor is what we conventionally call temperature. I agree that correspondence between things is not obvious - but once defined it seems to work and I don’t know if we could have more proof than that. I also agree that thermodynamics is worth studying in itself - and otherwise the attempt to reproduce it using statistical mechanics doesn’t make a lot of sense.
> But "understanding" requires much more than giving one simple definition without discussing the consequences of the definition.
But I did discuss the consequences: my definition (well, Boltzmann and Shannon's) leads directly to the second law, and in a way that is much more general and intuitive than the thermodynamic approach. What, for example, is the entropy of a cryptographic key? On the thermodynamic approach that question doesn't even make sense to ask. It's a category error because the thermodynamic definition relies on the concept of temperature and cryptographic keys don't have temperatures.
> Using atoms and microstates to explain entropy makes the explanation shorter but it also makes it more complex because it now depends on the definition of atoms and microstates.
Complexity is in the eye of the beholder. The classical billiard-ball model of atoms is perfectly adequate to explain the second law -- both the thermodynamic and information theoretic version -- and most people have no trouble wrapping their brains around the idea of "lots of tiny billiard balls wiggling around".
> I am curious what do you claim is a simple statistical explanation of temperature?
Higher/lower temperatures mean that the billiard balls are, on average, moving faster/slower relative to each other, all else being equal. And yes, I know that is not entirely accurate, but neither is the idea that atoms are billiard balls in the first place. This model is perfectly adequate as a first-order approximation. It allows you to describe the qualitative features of a heat engine purely in terms of the Newtonian mechanics of colliding billiard balls, which most people have a pretty good intuition for. No, it's not 100% accurate in the thermodynamic case, but the benefit is that it is obvious how to apply the concept of entropy defined in statistical terms to computation and information. I think that's a worthwhile tradeoff. Knowing how to calculate the efficiency of a heat engine is not very useful to most people in today's world. On the other hand, knowing how to estimate the entropy of a password or cryptographic key is very handy.
> The joke is that it is whatever the thermometer says it is.
Yes. But that joke conceals a deep truth: actually describing (let alone defining) thermodynamic temperature without reference to atoms, and without resorting to the thermometer joke, is very, very hard.
(It is analogous to trying to explain quantum mechanics without reference to entanglement, which is how it was done for decades. The problem with that approach is that it leaves you waving your hands about what a "measurement" is, because measurements are entanglements. Once you understand that simple fact, all of the mysteries of QM simply evaporate.)
> if bodies A and B are in thermal equilibrium (no heat transfer)
And how can I tell if A and B are in "thermal equilibrium" without resorting to the joke?
(BTW, your definition as you've given it is actually wrong. I can have two systems that are different temperatures but with no heat transfer between them simply by separating them with a perfect insulator. So now you have to incorporate that into your definition somehow, which means you have to define "insulator", and I don't see how you're going to do that without going around in circles.)
> It is an arbitrary (ordered) scale.
No, it's not. If it were arbitrary, the concept of specific heat would be meaningless.
Ultimately you need to make a connection between heat and motion. That is, after all, the thermodynamic project. You can convert heat into motion, and you can convert motion into heat, but the latter is much, much easier than the former. Why?
The statistical answer is: because heat is motion. It's a particular kind of motion. But if you deny yourself a referent to the things that are moving in the case of heat (atoms) then you deny yourself access to this simple straightforward explanation of the fundamental question of thermodynamics.
> "Understanding" entropy is about understanding why these vastly different definitions (thermodynamic and statistical) are equivalent
I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems. I've never seen that done successfully.
There's a reason that the atomic theory is a thing.
But I did discuss the consequences: my definition (well, Boltzmann and Shannon's) leads directly to the second law, and in a way that is much more general and intuitive than the thermodynamic approach. What, for example, is the entropy of a cryptographic key? On the thermodynamic approach that question doesn't even make sense to ask. It's a category error because the thermodynamic definition relies on the concept of temperature and cryptographic keys don't have temperatures.
> Using atoms and microstates to explain entropy makes the explanation shorter but it also makes it more complex because it now depends on the definition of atoms and microstates.
Complexity is in the eye of the beholder. The classical billiard-ball model of atoms is perfectly adequate to explain the second law -- both the thermodynamic and information theoretic version -- and most people have no trouble wrapping their brains around the idea of "lots of tiny billiard balls wiggling around".
> I am curious what do you claim is a simple statistical explanation of temperature?
Higher/lower temperatures mean that the billiard balls are, on average, moving faster/slower relative to each other, all else being equal. And yes, I know that is not entirely accurate, but neither is the idea that atoms are billiard balls in the first place. This model is perfectly adequate as a first-order approximation. It allows you to describe the qualitative features of a heat engine purely in terms of the Newtonian mechanics of colliding billiard balls, which most people have a pretty good intuition for. No, it's not 100% accurate in the thermodynamic case, but the benefit is that it is obvious how to apply the concept of entropy defined in statistical terms to computation and information. I think that's a worthwhile tradeoff. Knowing how to calculate the efficiency of a heat engine is not very useful to most people in today's world. On the other hand, knowing how to estimate the entropy of a password or cryptographic key is very handy.
> The joke is that it is whatever the thermometer says it is.
Yes. But that joke conceals a deep truth: actually describing (let alone defining) thermodynamic temperature without reference to atoms, and without resorting to the thermometer joke, is very, very hard.
(It is analogous to trying to explain quantum mechanics without reference to entanglement, which is how it was done for decades. The problem with that approach is that it leaves you waving your hands about what a "measurement" is, because measurements are entanglements. Once you understand that simple fact, all of the mysteries of QM simply evaporate.)
> if bodies A and B are in thermal equilibrium (no heat transfer)
And how can I tell if A and B are in "thermal equilibrium" without resorting to the joke?
(BTW, your definition as you've given it is actually wrong. I can have two systems that are different temperatures but with no heat transfer between them simply by separating them with a perfect insulator. So now you have to incorporate that into your definition somehow, which means you have to define "insulator", and I don't see how you're going to do that without going around in circles.)
> It is an arbitrary (ordered) scale.
No, it's not. If it were arbitrary, the concept of specific heat would be meaningless.
Ultimately you need to make a connection between heat and motion. That is, after all, the thermodynamic project. You can convert heat into motion, and you can convert motion into heat, but the latter is much, much easier than the former. Why?
The statistical answer is: because heat is motion. It's a particular kind of motion. But if you deny yourself a referent to the things that are moving in the case of heat (atoms) then you deny yourself access to this simple straightforward explanation of the fundamental question of thermodynamics.
> "Understanding" entropy is about understanding why these vastly different definitions (thermodynamic and statistical) are equivalent
I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems. I've never seen that done successfully.
There's a reason that the atomic theory is a thing.
I mostly agree with the first half of what you wrote, especially in terms of pedagogy, but when you delved in the nitty-gritty in the second half, there are a lot of mistakes that would impede understanding "advanced" concepts in physics. I believe that is at the root of our disagreement - you are perfectly right about early pedagogy, but understanding (which for me means "capability of advanced applications") requires more. Disdain for the thermodynamic definition limits ones toolbox. (Edit: "disdain" is a bad choice of words)
My main claim is: building the notion of entropy without relying on atomic theory is crucial if you want to apply it to "interesting" things like quantum mechanics or black hole physics. Admittedly that does not contradict your earlier posts, but it does significantly contradict the philosophy of your last post.
An assortment of issues I have with the second half of what you wrote as related to our discussion of what "understanding entropy" means:
- Passwords and cryptographic keys do not have entropy. What we call password entropy is a useful heuristic, but it differs from the rigorous information theory notion of entropy (or the thermodynamic one). I claim that it is important to understand that difference as it leads to understanding the limitations of the heuristic.
- The definition I have given is not wrong, it is one of the most standard ways scientist have been defining temperature for the last century. "Perfect insulators do not exist" is an extremely important theorem that you "derive" in theoretical physics and use for a vast array of incredibly important thought experiments on which much of our understanding of nature is build. Not taking seriously the thermodynamic definition of entropy thus limits your power to explain Nature.
Edit: "perfect insulators do not exist" is not a statement about how difficult it is to solve some engineering problem. It is an absolute statement with certainty equal to that of "√2 is irrational" or "you can not solve the halting problem".
- Temperature is indeed an arbitrary scale, again something that has been established in pretty much any textbook on statistical physics or thermodynamics. At this point I really think it is worthwhile to play the authority card: you are claiming multiple things that contradict standard physics textbooks on which the last hundred years of science are based.
- The heat engine efficiency is indeed not something most people care about day to day, but most people do not care about entanglement either. Moreover, I do not care about the heat engine as a question of engineering. However, thought experiments that involve heat engines are crucially important for the derivation of many no-go theorems (including in quantum mechanics).
- This is incredibly important: heat is NOT motion. This is incredibly limiting worldview that is insufficient for anything but the simplest of toy problems. Very useful early pedagogical tool, but it has to be discarded early on if you want your student to grow.
Sure, it is reasonable to say that it is a bad pedagogy to start with the thermodynamic definition. I teach graduate classes both in physics and in quantum information science (theoretical CS) and I actually follow your way of presenting entropy in the first few lectures. But the atomic theory interpretation of entropy is actually limiting when it comes to advanced physics. Really, the only thing I disagree about with you is your insistence that one does not need the thermodynamic definition in order to "understand" entropy. Please excuse my use of the word "insistence" if that is not the case - the limitation of text-based conversation are creeping in.
> I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems.
100% yes
> I've never seen that done successfully.
Most of the graduate classes I have taken have done that relatively well, but I would agree with you that undergrad classes need more work in the US. For what is worth, the French Class Prepa curriculum on this topic is absolutely outstanding. I particularly like classes/textbooks that take the differential geometry approach to thermodynamics. As long as the textbook treats stat physics and thermodynamics separately, it is probably a textbook that does it well.
My main claim is: building the notion of entropy without relying on atomic theory is crucial if you want to apply it to "interesting" things like quantum mechanics or black hole physics. Admittedly that does not contradict your earlier posts, but it does significantly contradict the philosophy of your last post.
An assortment of issues I have with the second half of what you wrote as related to our discussion of what "understanding entropy" means:
- Passwords and cryptographic keys do not have entropy. What we call password entropy is a useful heuristic, but it differs from the rigorous information theory notion of entropy (or the thermodynamic one). I claim that it is important to understand that difference as it leads to understanding the limitations of the heuristic.
- The definition I have given is not wrong, it is one of the most standard ways scientist have been defining temperature for the last century. "Perfect insulators do not exist" is an extremely important theorem that you "derive" in theoretical physics and use for a vast array of incredibly important thought experiments on which much of our understanding of nature is build. Not taking seriously the thermodynamic definition of entropy thus limits your power to explain Nature.
Edit: "perfect insulators do not exist" is not a statement about how difficult it is to solve some engineering problem. It is an absolute statement with certainty equal to that of "√2 is irrational" or "you can not solve the halting problem".
- Temperature is indeed an arbitrary scale, again something that has been established in pretty much any textbook on statistical physics or thermodynamics. At this point I really think it is worthwhile to play the authority card: you are claiming multiple things that contradict standard physics textbooks on which the last hundred years of science are based.
- The heat engine efficiency is indeed not something most people care about day to day, but most people do not care about entanglement either. Moreover, I do not care about the heat engine as a question of engineering. However, thought experiments that involve heat engines are crucially important for the derivation of many no-go theorems (including in quantum mechanics).
- This is incredibly important: heat is NOT motion. This is incredibly limiting worldview that is insufficient for anything but the simplest of toy problems. Very useful early pedagogical tool, but it has to be discarded early on if you want your student to grow.
Sure, it is reasonable to say that it is a bad pedagogy to start with the thermodynamic definition. I teach graduate classes both in physics and in quantum information science (theoretical CS) and I actually follow your way of presenting entropy in the first few lectures. But the atomic theory interpretation of entropy is actually limiting when it comes to advanced physics. Really, the only thing I disagree about with you is your insistence that one does not need the thermodynamic definition in order to "understand" entropy. Please excuse my use of the word "insistence" if that is not the case - the limitation of text-based conversation are creeping in.
> I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems.
100% yes
> I've never seen that done successfully.
Most of the graduate classes I have taken have done that relatively well, but I would agree with you that undergrad classes need more work in the US. For what is worth, the French Class Prepa curriculum on this topic is absolutely outstanding. I particularly like classes/textbooks that take the differential geometry approach to thermodynamics. As long as the textbook treats stat physics and thermodynamics separately, it is probably a textbook that does it well.
I'm not really relying on atomic theory per se, I'm simply relying on the notion of macroscopic observables that encompass an ensemble of microstates. It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition.
Now, it's possible that my insistence on this is a reflection of my ignorance. I'm not a physicist, I'm a computer scientist. But there are a few things that I know (or think I know) that are at odds with what you are telling me. For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. You get to pick two points on the scale, but the remaining points are then determined for you by actual physics. There is a whole field of study called calorimetry which relies on this. Your profile says you are a physicist so you must be aware of these things.
I'm not pointing this out to challenge your bona fides, only your pedagogy. You say, for example, "Heat is not motion" and that this is "incredibly important", and that "perfect insulators do not exist is a theorem", but then you don't provide any explanation or references. You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive.
BTW...
> Passwords and cryptographic keys do not have entropy.
You are mistaken. Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic".
(Imagine how annoying it would be if I stopped there and said nothing further.)
Entropy is only a coherent concept relative to a state of (generally incomplete) knowledge. It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas in cryptography it is determined by withholding information (i.e. keeping secrets). If I flip a coin, and I peek at it but I don't show you, that coin has zero entropy relative to my knowledge, but one bit of entropy relative to yours. The only difference between that and thermodynamics is that in the latter there are O(10^23) coins and it is impossible for anyone to peek at them.
My password has zero entropy relative to my knowledge, but (one hopes) a lot of entropy relative to an adversary's knowledge. Indeed, the amount of entropy in my password (measured in bits) is precisely equal to the base-2 logarithm of the number of possible passwords that an adversary would have to try in order to guess it.
Now, it's possible that my insistence on this is a reflection of my ignorance. I'm not a physicist, I'm a computer scientist. But there are a few things that I know (or think I know) that are at odds with what you are telling me. For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. You get to pick two points on the scale, but the remaining points are then determined for you by actual physics. There is a whole field of study called calorimetry which relies on this. Your profile says you are a physicist so you must be aware of these things.
I'm not pointing this out to challenge your bona fides, only your pedagogy. You say, for example, "Heat is not motion" and that this is "incredibly important", and that "perfect insulators do not exist is a theorem", but then you don't provide any explanation or references. You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive.
BTW...
> Passwords and cryptographic keys do not have entropy.
You are mistaken. Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic".
(Imagine how annoying it would be if I stopped there and said nothing further.)
Entropy is only a coherent concept relative to a state of (generally incomplete) knowledge. It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas in cryptography it is determined by withholding information (i.e. keeping secrets). If I flip a coin, and I peek at it but I don't show you, that coin has zero entropy relative to my knowledge, but one bit of entropy relative to yours. The only difference between that and thermodynamics is that in the latter there are O(10^23) coins and it is impossible for anyone to peek at them.
My password has zero entropy relative to my knowledge, but (one hopes) a lot of entropy relative to an adversary's knowledge. Indeed, the amount of entropy in my password (measured in bits) is precisely equal to the base-2 logarithm of the number of possible passwords that an adversary would have to try in order to guess it.
> It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition.
Yes, but the rather complicated notion of microstate (or probability or knowledge or information) is not necessary in the thermodynamic definition, which is why it is valuable.
> For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary.
If you start by introducing the existence of specific heat (which is a pretty good way to introduce concepts related to temperature both in the statistical and in the thermodynamic case) then you indeed need to choose a canonical temperature scale. Kinda how introducing exponentiation through its derivative provides a canonical choice for the basis of the logarithm. But it is important to appreciate the fact that this canonical choice is somewhat arbitrary: it just makes the equations look more "natural", nothing more.
> I'm not pointing this out to challenge your bona fides, only your pedagogy.
But I never challenged your assertions on pedagogy. I repeatedly and enthusiastically agreed with them in all my posts. I just strongly disagree with your definition of the word "understand".
> You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive.
Part of it is that I never claimed that understanding entropy is easy. I multiple times referred to the axioms in question by name, but did not claim there is a simple explanation that would let one understand what entropy is. The wiki pages on the topic are quite instructive though:
- https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#F...
- https://en.wikipedia.org/wiki/Thermodynamic_temperature
- Most importantly, the various statements of the second law. I claim fully understanding entropy requires understanding why these various statements are equivalent: https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#V...
> Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic".
No, they are different concepts, neither one is generalization of the other. That is the point I am making all along.
"Password entropy", the way it is commonly used and how I assumed you used it, as in "here is my password, calculate its entropy" is not (information theoretic) entropy, it is just a useful heuristic figure of merit. You can not have entropy without having some probability distribution as you already said in your clarification. And a password is just a single microstate: there is no such thing as entropy of a single microstate, but again, it seems we are in agreement about that.
> It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas.
This should say "in statistical physics". Thermodynamic entropy has (superficially) nothing to do with knowledge or information.
Few more fun references if you are into extreme mathematical rigor:
- https://web.ist.utl.pt/berberan/data/68.pdf
- https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#T...
Edit to add: There is a neat "contradiction" between the notions of thermodynamic entropy and statistical entropy. Namely, the notion of ergodicity. Thermodynamic entropy does not let you understand ergodicity or the fact that entropy is permitted to spontaneously drop. Information theoretical approaches do permit you to talk about entropy dropping on its own. While on the topic of pedagogy, discussing this clash is probably quite fruitful. But it is a bit outside my area of expertise. The following is probably a good place from which to start picking terms to google https://math.stackexchange.com/questions/433854/relationship...
Yes, but the rather complicated notion of microstate (or probability or knowledge or information) is not necessary in the thermodynamic definition, which is why it is valuable.
> For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary.
If you start by introducing the existence of specific heat (which is a pretty good way to introduce concepts related to temperature both in the statistical and in the thermodynamic case) then you indeed need to choose a canonical temperature scale. Kinda how introducing exponentiation through its derivative provides a canonical choice for the basis of the logarithm. But it is important to appreciate the fact that this canonical choice is somewhat arbitrary: it just makes the equations look more "natural", nothing more.
> I'm not pointing this out to challenge your bona fides, only your pedagogy.
But I never challenged your assertions on pedagogy. I repeatedly and enthusiastically agreed with them in all my posts. I just strongly disagree with your definition of the word "understand".
> You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive.
Part of it is that I never claimed that understanding entropy is easy. I multiple times referred to the axioms in question by name, but did not claim there is a simple explanation that would let one understand what entropy is. The wiki pages on the topic are quite instructive though:
- https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#F...
- https://en.wikipedia.org/wiki/Thermodynamic_temperature
- Most importantly, the various statements of the second law. I claim fully understanding entropy requires understanding why these various statements are equivalent: https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#V...
> Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic".
No, they are different concepts, neither one is generalization of the other. That is the point I am making all along.
"Password entropy", the way it is commonly used and how I assumed you used it, as in "here is my password, calculate its entropy" is not (information theoretic) entropy, it is just a useful heuristic figure of merit. You can not have entropy without having some probability distribution as you already said in your clarification. And a password is just a single microstate: there is no such thing as entropy of a single microstate, but again, it seems we are in agreement about that.
> It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas.
This should say "in statistical physics". Thermodynamic entropy has (superficially) nothing to do with knowledge or information.
Few more fun references if you are into extreme mathematical rigor:
- https://web.ist.utl.pt/berberan/data/68.pdf
- https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#T...
Edit to add: There is a neat "contradiction" between the notions of thermodynamic entropy and statistical entropy. Namely, the notion of ergodicity. Thermodynamic entropy does not let you understand ergodicity or the fact that entropy is permitted to spontaneously drop. Information theoretical approaches do permit you to talk about entropy dropping on its own. While on the topic of pedagogy, discussing this clash is probably quite fruitful. But it is a bit outside my area of expertise. The following is probably a good place from which to start picking terms to google https://math.stackexchange.com/questions/433854/relationship...
This just occurred to me:
> Perfect insulators do not exist
Then it is not possible for any systems ever to be in thermal equilibrium before the heat death of the universe. So your definition of temperature is based on something that you yourself have said is physically impossible to achieve.
> Perfect insulators do not exist
Then it is not possible for any systems ever to be in thermal equilibrium before the heat death of the universe. So your definition of temperature is based on something that you yourself have said is physically impossible to achieve.
You are kinda right :D There is always a tension between rigor and practicality in theoretical physics (and frankly, in math -- just read how giants like Poincare felt about rigor or how Euler worked with infinite series in horrifically handwavy fashion).
This level of hierarchical abstraction is pretty much the only tool we have though, and it seems it works better than whatever else we have tried. Axiomatic thermodynamics (the last couple of links in my previous post) does answer this particular problem though, it is just that no-one uses it.
This level of hierarchical abstraction is pretty much the only tool we have though, and it seems it works better than whatever else we have tried. Axiomatic thermodynamics (the last couple of links in my previous post) does answer this particular problem though, it is just that no-one uses it.
> Today I would have more strongly made the rather Feynmanesque point that if you have a theory that says everything we observe today is an exception to your theory, then the theory you have isn’t terribly useful.
I've found this quote amusing and I believe it explains to some extent problems with the Second Law.
> The second law is very easy to understand when presented as Boltzmann formulated it
When I read prominent scientists I often become impressed by their creative ability to not understand "simple" ideas and to spend years in attempts to understand them. It seems for me that this ability and willingness to think is the main driver of a scientific progress.
Einstein for example rejected a very simple idea of time, and reinvented it to create the theory of relativity. How on Earth someone with a brain can not understand time? But scientists can and they find this useful. I think they have special secret training on incomprehension.
To apply your mind to a problem you need first to find a problem to attack, and this is the most tricky part of science. If you do not see problems with existing theories you are out of luck, you have nothing to attack. So your first reaction to any idea presented to you better be "I do not understand" then "it is very easy and even obvious".
I've found this quote amusing and I believe it explains to some extent problems with the Second Law.
> The second law is very easy to understand when presented as Boltzmann formulated it
When I read prominent scientists I often become impressed by their creative ability to not understand "simple" ideas and to spend years in attempts to understand them. It seems for me that this ability and willingness to think is the main driver of a scientific progress.
Einstein for example rejected a very simple idea of time, and reinvented it to create the theory of relativity. How on Earth someone with a brain can not understand time? But scientists can and they find this useful. I think they have special secret training on incomprehension.
To apply your mind to a problem you need first to find a problem to attack, and this is the most tricky part of science. If you do not see problems with existing theories you are out of luck, you have nothing to attack. So your first reaction to any idea presented to you better be "I do not understand" then "it is very easy and even obvious".
> Einstein for example rejected a very simple idea of time, and reinvented it to create the theory of relativity.
But that _really_ isn't what he did! He started from the seemingly obvious notion of linear time. Added the _observation_ of the finite speed of light. And _noticed_ that the "obvious" notion of linear time leads to contradictions (basically saying "at the same time" has different meanings for different observers). And from that he _had_ to go out and construct relativity in order to reconcile these contradictions. Does it matter to anyone? In practice? Probably not. As long as you do not get close to the speed of light (either in velocity or in mass. Yes.. that statement makes sense as is) relativity really is indistinguishable from our notion of linear time. This is _necessary_ for relativity to be a useful theory.
But it is _also_ a massive shift in how we think about the world. That's the danger with "intuitive" understandings. They can only ever be based on experiences and those do not lend themselves to extrapolations. The 2nd law of thermodynamics is one of those things. It seems completely obvious if learn about it the first time. Then you go a bit deeper, start picking it a part, thinking about the second and third order consequences of it. Then you argue a bit about whether the probabilistic formulations are worth anything in the first place, then you try to reconcile it with information theory. At this point you get into the whole quantum mess and ask yourself how this "law" can hold in all these circumstance, even if it doesn't seem to have any reason to mean anything anymore.
But that _really_ isn't what he did! He started from the seemingly obvious notion of linear time. Added the _observation_ of the finite speed of light. And _noticed_ that the "obvious" notion of linear time leads to contradictions (basically saying "at the same time" has different meanings for different observers). And from that he _had_ to go out and construct relativity in order to reconcile these contradictions. Does it matter to anyone? In practice? Probably not. As long as you do not get close to the speed of light (either in velocity or in mass. Yes.. that statement makes sense as is) relativity really is indistinguishable from our notion of linear time. This is _necessary_ for relativity to be a useful theory.
But it is _also_ a massive shift in how we think about the world. That's the danger with "intuitive" understandings. They can only ever be based on experiences and those do not lend themselves to extrapolations. The 2nd law of thermodynamics is one of those things. It seems completely obvious if learn about it the first time. Then you go a bit deeper, start picking it a part, thinking about the second and third order consequences of it. Then you argue a bit about whether the probabilistic formulations are worth anything in the first place, then you try to reconcile it with information theory. At this point you get into the whole quantum mess and ask yourself how this "law" can hold in all these circumstance, even if it doesn't seem to have any reason to mean anything anymore.
> problems with the Second Law
Like what?
Like what?
To repeat even more tersely, the fraction of information you have about a system can only go spontaneously down, never up. Knowing less is free, learning is at cost.
That is brilliant. Is that your idea, or did you get it from somewhere?
It is the notion behind "Completely Positive Trace Preserving" maps. It is used a lot in quantum information and quantum computing. It is present by other names in other situations.
That’s not the fundamental formulation which does not require statistical mechanics. Anybody saying entropy is very easy to understand doesn’t understand it.
> That’s not the fundamental formulation
Why not? By whose standard is a different formulation more "fundamental"?
Why not? By whose standard is a different formulation more "fundamental"?
It is more about the fact that there are two "equally fundamental" explanations, and saying there is only one is a great disservice to the curious mind: the most fascinating and deep insights about entropy come from understanding how the two definitions are equivalent.
> The more possible configurations there are, the greater the entropy. And this explains the second law: given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.
This is an argument for observing high entropy macrostates at any time, assuming all microstates are equally probable. But not for evolution in time that makes entropy higher in the future, or at least non-smaller in the future. You assume the closed system evolves its microstate in time "by pure chance", i.e. any microstate is equally probable as the next microstate, which ignores many constraints there are due to physics laws like conservation of phase volume or energy. This probabilistic argument could be applied to past just as well as the future. But observations show entropy of an isolated system is lower in the past than in the present. So your argument does not really derive all aspects of 2nd law.
This is an argument for observing high entropy macrostates at any time, assuming all microstates are equally probable. But not for evolution in time that makes entropy higher in the future, or at least non-smaller in the future. You assume the closed system evolves its microstate in time "by pure chance", i.e. any microstate is equally probable as the next microstate, which ignores many constraints there are due to physics laws like conservation of phase volume or energy. This probabilistic argument could be applied to past just as well as the future. But observations show entropy of an isolated system is lower in the past than in the present. So your argument does not really derive all aspects of 2nd law.
> But not for evolution in time that makes entropy higher in the future, or at least non-smaller in the future.
Well, obviously the system is subject to the constraints of its dynamics. It's not just picking a new microstate at random at each instant in time. It can only evolve to states that are accessible from its current state according to its dynamics. But given that set of accessible microstates, you are more likely to get a macrostate corresponding to a larger number of microstates than a smaller number.
The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery.
Well, obviously the system is subject to the constraints of its dynamics. It's not just picking a new microstate at random at each instant in time. It can only evolve to states that are accessible from its current state according to its dynamics. But given that set of accessible microstates, you are more likely to get a macrostate corresponding to a larger number of microstates than a smaller number.
The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery.
> But given that set of accessible microstates, you are more likely to get a macrostate corresponding to a larger number of microstates than a smaller number.
This is due to mixing in phase space, which is plausible for many particle systems. However, it works in both directions of the parameter t.
The only way to derive 2nd law including its time asymmetry (which is the major point of 2nd law) is to restrict considerations to those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state. This means we consider only subset of all microscopically possible processes. E.g. Jaynes selects those macroscopic processes that bring one macrostate A, via some irreversible evolution, to another macrostate B, repeatedly. Then, assuming this reliability of the resulting state, he derives non-decrease of entropy from maximum entropy principle and Hamiltonian statistical physics. If we didn't assume this restriction and allowed all mechanically possible processes, entropy could both increase and decrease.
> The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery.
This often repeated idea is in vein with Clausius' ostentatious statements on energy and entropy of the Universe, which have been out of place and plagued discussions of thermodynamics with needless mystical assumptions ever since he wrote them. There is no evidence for them, as there are no reliable measurements of entropy of the Universe. There is no need for them, as there are no good reasons to believe Universe has defined entropy at all. Thermodynamics, including 2nd law, is about finite sized systems free of strong gravity effects. It does not extrapolate easily beyond compact systems like planets/stars to ever bigger systems. There is no Universe in it anywhere.
2nd law that we see in daily life or a lab is not a statement about entropy of Universe being lower in the past, nor does it hang on such an assumption. It is (in one of its variants) the statement that a finite isolated system can change its macrostate to some other final macrostate, but this final macrostate is of higher or same entropy, so in real process entropy can't decrease.
This is due to mixing in phase space, which is plausible for many particle systems. However, it works in both directions of the parameter t.
The only way to derive 2nd law including its time asymmetry (which is the major point of 2nd law) is to restrict considerations to those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state. This means we consider only subset of all microscopically possible processes. E.g. Jaynes selects those macroscopic processes that bring one macrostate A, via some irreversible evolution, to another macrostate B, repeatedly. Then, assuming this reliability of the resulting state, he derives non-decrease of entropy from maximum entropy principle and Hamiltonian statistical physics. If we didn't assume this restriction and allowed all mechanically possible processes, entropy could both increase and decrease.
> The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery.
This often repeated idea is in vein with Clausius' ostentatious statements on energy and entropy of the Universe, which have been out of place and plagued discussions of thermodynamics with needless mystical assumptions ever since he wrote them. There is no evidence for them, as there are no reliable measurements of entropy of the Universe. There is no need for them, as there are no good reasons to believe Universe has defined entropy at all. Thermodynamics, including 2nd law, is about finite sized systems free of strong gravity effects. It does not extrapolate easily beyond compact systems like planets/stars to ever bigger systems. There is no Universe in it anywhere.
2nd law that we see in daily life or a lab is not a statement about entropy of Universe being lower in the past, nor does it hang on such an assumption. It is (in one of its variants) the statement that a finite isolated system can change its macrostate to some other final macrostate, but this final macrostate is of higher or same entropy, so in real process entropy can't decrease.
> The only way to derive 2nd law including its time asymmetry...
I don't think that's true, though I'm not sure I understand what you mean by "those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state". (The same as what? The same in what sense?)
The reason you can't unscramble an egg is not because there is anything fundamentally irreversible in the laws of physics, it's because the time-evolution of the scrambling egg is governed not only by the motion of the whisk but also by the random thermalized motion of the constituent particles. To unscramble the egg you would need not only to precisely reverse the motion of the whisk, but also all of the thermal velocities of the constituent particles. The latter might be possible in principle (though not in practice) in a classical world, but throw in quantum mechanics and it's not possible even in principle. So the second law happens simply because the dynamics of non-trivial systems (like eggs) are chaotic, and quantum mechanics always throws in some randomness at the lowest levels.
And yes, quantum mechanics is also time-reversible. If you want to quibble about that, read this first:
https://blog.rongarret.info/2014/10/parallel-universes-and-a...
> There is no evidence for them
If you believe that the 2nd law is universal (and I see no reason to doubt it) then the inescapable conclusion is that the further back you go, the lower the entropy of the universe was, reaching a global minimum at inception.
I don't think that's true, though I'm not sure I understand what you mean by "those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state". (The same as what? The same in what sense?)
The reason you can't unscramble an egg is not because there is anything fundamentally irreversible in the laws of physics, it's because the time-evolution of the scrambling egg is governed not only by the motion of the whisk but also by the random thermalized motion of the constituent particles. To unscramble the egg you would need not only to precisely reverse the motion of the whisk, but also all of the thermal velocities of the constituent particles. The latter might be possible in principle (though not in practice) in a classical world, but throw in quantum mechanics and it's not possible even in principle. So the second law happens simply because the dynamics of non-trivial systems (like eggs) are chaotic, and quantum mechanics always throws in some randomness at the lowest levels.
And yes, quantum mechanics is also time-reversible. If you want to quibble about that, read this first:
https://blog.rongarret.info/2014/10/parallel-universes-and-a...
> There is no evidence for them
If you believe that the 2nd law is universal (and I see no reason to doubt it) then the inescapable conclusion is that the further back you go, the lower the entropy of the universe was, reaching a global minimum at inception.
> I'm not sure I understand what you mean by "those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state". (The same as what? The same in what sense?)
In the formulation of 2nd law for adiabatically isolated systems, there is the initial equilibrium macrostate A, and the different equilibrium macrostate B that the system gets into eventually after the intervention, where either some constraint is released, or some macroscopic work is done. 2nd law states that if the same intervention in A results in B reliably, then S_B >= S_A.
This statement can be derived from mechanics and maximum entropy principle only under the assumption that B is a reliable result of intervention in A (always or with close to 1 probability). This is more than mechanics or logic can provide, and it means we have a restriction on the allowed set of microscopic processes considered.
It is not possible to derive 2nd law from mechanics or logic for all microscopic processes. Some microscopic processes bring the state B to state A, and those break 2nd law.
In the formulation of 2nd law for adiabatically isolated systems, there is the initial equilibrium macrostate A, and the different equilibrium macrostate B that the system gets into eventually after the intervention, where either some constraint is released, or some macroscopic work is done. 2nd law states that if the same intervention in A results in B reliably, then S_B >= S_A.
This statement can be derived from mechanics and maximum entropy principle only under the assumption that B is a reliable result of intervention in A (always or with close to 1 probability). This is more than mechanics or logic can provide, and it means we have a restriction on the allowed set of microscopic processes considered.
It is not possible to derive 2nd law from mechanics or logic for all microscopic processes. Some microscopic processes bring the state B to state A, and those break 2nd law.
> the initial equilibrium macrostate A, and the different equilibrium macrostate B that the system gets into eventually
[emphasis added]
Your use of "the" here, with the implication that there are two unique microstates A and B, is problematic. There is no such thing as "the" equilibrium microstate. A system at thermodynamic equilibrium is not static.
(In fact, I think it is fair to say that the whole point of thermodynamics is to sweep vast numbers of the physical degrees of freedom of a system under the rug so that you can treat a system at equilibrium as if it were static even though it really isn't. It is frankly astonishing that this is even possible at all without losing any fidelity in terms of measurable outcomes.)
> Some microscopic processes bring the state B to state A, and those break 2nd law.
Yes, that's true. The second law is probabilistic. It is "possible" to break the second law in the same sense that it is "possible" for a baseball to quantum-mechanically tunnel through a catcher's mitt. You can do the math to figure the odds in both cases. In fact, actually going through this process is an informative exercise. It isn't difficult.
There are some events whose probabilities are so low that the chance of them occurring anywhere in the universe before heat death is barely distinguishable from zero. The odds of actually observing such an event here on earth are vastly smaller still. Both macroscopic tunneling and 2nd law violations are events of this sort. Neither is categorically impossible, but (to put it mildly) it's pretty safe to bet against them nonetheless.
[emphasis added]
Your use of "the" here, with the implication that there are two unique microstates A and B, is problematic. There is no such thing as "the" equilibrium microstate. A system at thermodynamic equilibrium is not static.
(In fact, I think it is fair to say that the whole point of thermodynamics is to sweep vast numbers of the physical degrees of freedom of a system under the rug so that you can treat a system at equilibrium as if it were static even though it really isn't. It is frankly astonishing that this is even possible at all without losing any fidelity in terms of measurable outcomes.)
> Some microscopic processes bring the state B to state A, and those break 2nd law.
Yes, that's true. The second law is probabilistic. It is "possible" to break the second law in the same sense that it is "possible" for a baseball to quantum-mechanically tunnel through a catcher's mitt. You can do the math to figure the odds in both cases. In fact, actually going through this process is an informative exercise. It isn't difficult.
There are some events whose probabilities are so low that the chance of them occurring anywhere in the universe before heat death is barely distinguishable from zero. The odds of actually observing such an event here on earth are vastly smaller still. Both macroscopic tunneling and 2nd law violations are events of this sort. Neither is categorically impossible, but (to put it mildly) it's pretty safe to bet against them nonetheless.
You have missed my point. The formulation of 2nd law I'm talking about is stating something about all pairs of two MACROstates A,B, not MICROstates, such that B reliably succeeds A after intervention. This pair of MACROstates is not assumed unique, there are hugely many, but formulating the argument uses a single pair. (I'm not a native speaker so don't get distracted by possibly incorrect use of the definite article).
The problem with your handwavy argument about macrostates with higher multiplicity is not due to 2nd law being a probabilistic claim. The problem is that it is not clear from that argument alone why it applies to future states, but it does not apply to the past states. To address that, another assumption is needed, such as the asymmetric relation between my macrostates A,B.
The problem with your handwavy argument about macrostates with higher multiplicity is not due to 2nd law being a probabilistic claim. The problem is that it is not clear from that argument alone why it applies to future states, but it does not apply to the past states. To address that, another assumption is needed, such as the asymmetric relation between my macrostates A,B.
Sorry, my mistake, I somehow misread "macro" as "micro". More than once. But it was obviously an "a" all along because it's an "a" in my quote. I need to start paying closer attention.
So, having now re-read what you wrote upstream more carefully, can you elaborate on what you mean by "intervention"?
> The problem is that it is not clear from that argument alone why it applies to future states, but it does not apply to the past states.
I don't understand this. What do you mean that it "does not apply to past states"?
> another assumption is needed, such as the asymmetric relation between my macrostates A,B
Well, yeah, there is an asymmetric relationship: one precedes the other, and not the other way around. But that's not an "assumption", that is just how you defined A and B.
I really don't see the problem you are trying to describe.
So, having now re-read what you wrote upstream more carefully, can you elaborate on what you mean by "intervention"?
> The problem is that it is not clear from that argument alone why it applies to future states, but it does not apply to the past states.
I don't understand this. What do you mean that it "does not apply to past states"?
> another assumption is needed, such as the asymmetric relation between my macrostates A,B
Well, yeah, there is an asymmetric relationship: one precedes the other, and not the other way around. But that's not an "assumption", that is just how you defined A and B.
I really don't see the problem you are trying to describe.
By intervention I mean some external agent causing the system to leave the initial equilibrium macrostate A, which would not happen spontaneously. E.g. a wall is removed, or a piston is quickly pushed, to make the system evolve towards new equilibrium macrostate.
> given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.
This argument is that future macrostates of an isolated system passing through non-equilibrium states (due to some intervention to the system that was in an equilibrium state before) should be almost always of higher entropy, because those have immensely higher multiplicity than macrostates with lower entropy. There is hugely more microstates that are compatible with higher entropy macrostate, than microstates that are compatible with lower entropy macrostate.
However notice the word "future". We can formulate the same argument using "past": past macrostates of an isolated system passing through non-equilibrium states should be almost always of higher entropy, because those have immensely higher multiplicity than macrostates with lower entropy. There is hugely more microstates that are compatible with higher entropy macrostate, than microstates that are compatible with lower entropy macrostate.
Now, why does the argument give correct answer when applied to future macrostates but not when applied to past macrostates? We need additional assumption about the non-equilibrium process that makes the argument invalid for the past.
> given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.
This argument is that future macrostates of an isolated system passing through non-equilibrium states (due to some intervention to the system that was in an equilibrium state before) should be almost always of higher entropy, because those have immensely higher multiplicity than macrostates with lower entropy. There is hugely more microstates that are compatible with higher entropy macrostate, than microstates that are compatible with lower entropy macrostate.
However notice the word "future". We can formulate the same argument using "past": past macrostates of an isolated system passing through non-equilibrium states should be almost always of higher entropy, because those have immensely higher multiplicity than macrostates with lower entropy. There is hugely more microstates that are compatible with higher entropy macrostate, than microstates that are compatible with lower entropy macrostate.
Now, why does the argument give correct answer when applied to future macrostates but not when applied to past macrostates? We need additional assumption about the non-equilibrium process that makes the argument invalid for the past.
> By intervention I mean some external agent causing the system to leave the initial equilibrium macrostate
OK, that's what I thought.
> We need additional assumption about the non-equilibrium process that makes the argument invalid for the past.
Ah.
No, you don't need any additional assumptions. All you need is to observe that the present constrains the past differently from the future. There are fewer pasts compatible with a given present than there are futures compatible with that same present.
You can't just say that there are many high-entropy microstates, and so you are as likely to encounter one in the past as you are in the future, because any candidate past state has to be able to evolve into the present state. And if the present state is a low-entropy state, the precursor for that is much more likely to be an even lower-entropy state. Yes, there are high-entropy microstates that will evolve into low-entropy ones, but the fraction of those states chosen from among all high-entropy microstates is indistinguishable from zero, and so the odds of a high-entropy microstate in general being the precursor of a present low-entropy state is likewise indistinguishable from zero.
By way of very stark contrast, the successor states of a low-entropy state (or any state for that matter) is much more likely to be a high entropy state than a low-entropy one for the exact same reason: the present constrains the past more rigidly than it does the future.
OK, that's what I thought.
> We need additional assumption about the non-equilibrium process that makes the argument invalid for the past.
Ah.
No, you don't need any additional assumptions. All you need is to observe that the present constrains the past differently from the future. There are fewer pasts compatible with a given present than there are futures compatible with that same present.
You can't just say that there are many high-entropy microstates, and so you are as likely to encounter one in the past as you are in the future, because any candidate past state has to be able to evolve into the present state. And if the present state is a low-entropy state, the precursor for that is much more likely to be an even lower-entropy state. Yes, there are high-entropy microstates that will evolve into low-entropy ones, but the fraction of those states chosen from among all high-entropy microstates is indistinguishable from zero, and so the odds of a high-entropy microstate in general being the precursor of a present low-entropy state is likewise indistinguishable from zero.
By way of very stark contrast, the successor states of a low-entropy state (or any state for that matter) is much more likely to be a high entropy state than a low-entropy one for the exact same reason: the present constrains the past more rigidly than it does the future.
> you don't need any additional assumptions
What you wrote after that is such an assumption. You can't derive asymmetry from nothing, logic or mechanics. The only way to have it, is to assume it in addition to those.
What you wrote after that is such an assumption. You can't derive asymmetry from nothing, logic or mechanics. The only way to have it, is to assume it in addition to those.
The question of what are the states to count and how is not obvious.
“It cannot be emphasized strongly enough that W is not the measure of the set C of all states […] compatible with the external macroscopic constraints; rather, it is the dimension of the subset of those states that can be realized in the greatest number of ways.” [Entropy and the Time Evolution of Macroscopic Systems, Walter T. Grandy Jr.]
“It cannot be emphasized strongly enough that W is not the measure of the set C of all states […] compatible with the external macroscopic constraints; rather, it is the dimension of the subset of those states that can be realized in the greatest number of ways.” [Entropy and the Time Evolution of Macroscopic Systems, Walter T. Grandy Jr.]
That's interesting, I didn't know that. I always thought that W in k ln(W) was the number of possible states. Thanks for bringing this to my attention!
I wonder, though, does this actually matter to anyone but a physicist? Consider the common just-so story about electromagnetic waves: a changing electric field produces a magnetic field, and a changing magnetic field produces an electric field. Therefore, once you start to wiggle an electron around, the changing field become self-sustaining and you end up with a wave.
This story is wrong because it would lead you to conclude that the E and B fields are 90 degrees out of phase when in fact they are in phase, and understanding why that is requires going into quite a bit more detail. But does that actually matter to anyone other than a physicist or an antenna engineer?
I wonder, though, does this actually matter to anyone but a physicist? Consider the common just-so story about electromagnetic waves: a changing electric field produces a magnetic field, and a changing magnetic field produces an electric field. Therefore, once you start to wiggle an electron around, the changing field become self-sustaining and you end up with a wave.
This story is wrong because it would lead you to conclude that the E and B fields are 90 degrees out of phase when in fact they are in phase, and understanding why that is requires going into quite a bit more detail. But does that actually matter to anyone other than a physicist or an antenna engineer?
> I always thought that W in k ln(W) was the number of possible states.
That formula - Boltzmann's entropy - is only applicable when all the microstates are equiprobable. The question of what does it mean in general is subtle.
If we have a gas in a container in thermal equilibrium with a heatbath, for example, the energy of the microstates compatible with those constraints is not fixed and their probability depends on the energy. The energy and the pressure fluctuate.
> I wonder, though, does this actually matter to anyone but a physicist?
The people most concerned with the question of entropy on a conceptual level tend to be physicists - and philosophers. https://deepblue.lib.umich.edu/bitstream/handle/2027.42/4341...
That formula - Boltzmann's entropy - is only applicable when all the microstates are equiprobable. The question of what does it mean in general is subtle.
If we have a gas in a container in thermal equilibrium with a heatbath, for example, the energy of the microstates compatible with those constraints is not fixed and their probability depends on the energy. The energy and the pressure fluctuate.
> I wonder, though, does this actually matter to anyone but a physicist?
The people most concerned with the question of entropy on a conceptual level tend to be physicists - and philosophers. https://deepblue.lib.umich.edu/bitstream/handle/2027.42/4341...
[deleted]
I'm not sure anyone could differentiate Wolfram's own writing from what ChatGPT fine tuned on Wolfram might generate. He has a very distinctive way of writing.
So, it makes you wonder does that make ChatGPT smart since Wolfram says he's smart, or does it make Wolfram dumb since he's just an LLM ?
So, it makes you wonder does that make ChatGPT smart since Wolfram says he's smart, or does it make Wolfram dumb since he's just an LLM ?
This is something I’ve observed as well, and is noticeable not just in his written work, but also in his interviews.
Yet again, one mustn't go against the "american exceptionalism"[1] dogma, that's sad to hear
The result of this ego-centric americanism; Russia able to develop their supersonic weapons before anyone in the west, thanks to the work of the silenced non-US scientists [2]
[1] - https://en.wikipedia.org/wiki/American_exceptionalism
[2] - https://www.youtube.com/watch?v=Jn8b3E9oUHY
The result of this ego-centric americanism; Russia able to develop their supersonic weapons before anyone in the west, thanks to the work of the silenced non-US scientists [2]
[1] - https://en.wikipedia.org/wiki/American_exceptionalism
[2] - https://www.youtube.com/watch?v=Jn8b3E9oUHY
The book "Algorithmic Randomness and Complexity" by Downey and Hirschfeldt goes into all of this. It is way beyond this rule 30 stuff.