Five Most Productive Years: What Happened and What's Next(writings.stephenwolfram.com)
writings.stephenwolfram.com
Five Most Productive Years: What Happened and What's Next
https://writings.stephenwolfram.com/2024/08/five-most-productive-years-what-happened-and-whats-next/
84 comments
A lot to admire in this post about passion and long-term thinking but this is too egregious.
> Back in 1979, for example, I’d invented the idea of transformations for symbolic expressions as a foundation for computational language.
I hope at 65 to have the energy to work this hard, but I also hope at 65 I'm surrounded by people who will kindly correct me when I take credit for ideas that aren't mine, and that I will listen to them.
> Back in 1979, for example, I’d invented the idea of transformations for symbolic expressions as a foundation for computational language.
I hope at 65 to have the energy to work this hard, but I also hope at 65 I'm surrounded by people who will kindly correct me when I take credit for ideas that aren't mine, and that I will listen to them.
> Back in 1979, for example, I’d invented the idea of transformations for symbolic expressions as a foundation for computational language.
Right, so math then. You invented math.
Right, so math then. You invented math.
Indeed I agree it is an egregious claim.
Say what you want of Stephen Wolfram, but he's an interesting person doing interesting things - and managing to finish a lot of them.
A bite-sized idea I liked from the long article: "the very act of exposition was a critical part of organizing and developing my ideas".
I've arrived to the same conclusion for myself (and this article, hopefully, will be the last straw for me to start writing in an organized manner).
My only moderately-successful writing so far has been my ADHD wiki[0], which, in the spirit of Stephen Wolfram, I will shamelessly promote here and now (I've gotten some marvelous feedback from HN in the past, and I believe it to be a useful resource to many).
FWIW Stephen not merely boasting about productivity; a fair bit of the article is dedicated to talking about techniques and tools that he believes help him with that.
While his article on Computational Essays[1] is mostly a Mathematica ad, Mathematica is a great system (which have influenced things like Jupyter notebooks a lot), and the idea of literal programming and interactive data/code/writing is a solid one.
We have yet to have a solid collection of resources like that even for teaching mathematics (where it's natural to play with code to experiment with ideas).
Another bit I liked: and yes, one seems to be able to see the essence of machine learning in systems vastly simpler than neural nets. Sure, we all know about that cellular automata are the Woflram's thing, so it's not surprising to see them pop up in his article about minimal learning computational models[2], but I feel an article like that has been way overdue.
We've been playing with neural nets long enough without having a solid idea what's really going on, and it's limiting to use them as legos of sorts. Why these blocks in particular? What else is out there?
The ChatGPT explainer that he mentioned[1] is still my go-to article for learning about it; I think I'll add another one of his to the list.
Finally, the bit one why history is important - and finding out his archive of writing on history of math and science[4] is great. I believe that history is the most underappreciate science itself, and learning science without its history leaves you without either context or deep understanding of it.
(Personally, I add etymology to history: I find resources like "Earliest Known Uses of Some Mathmatical Terms"[5] invaluable).
And, of course, it's great to see that he's diving into linking mathematics, computation, physics, biology. Great discoveries often lie on the interfaces of various fields.
As Vladimir Arnold wrote in his famous essay[6], divorcing mathematics from physics has been a phenomenal crime. I'm guilty of it too. I recently made a post on reddit[7] with a GIF showing the osculating circles and the evolute of an ellipse. It's pretty, but what is really hiding behind it is the shape of the gear tooth which nearly all gears have.
I learned (and taught!) the mathematics behind it without having any understanding how gears are designed and why they work. And people who make gears make them without understanding the math behind the equations. This came up at work (I'm working on implicit CAD modeling), and from a discussion a better understanding (...and a better product) emerged. There is no reason for narrow specialization that creates barriers in fields that aren't just related, but are necessary for each other - so I believe that the mere fact of Wolfram doing this work is important.
Maybe he'll find out something groundbreaking in those directions. Maybe not. But I can guarantee that he and people around him will stumble into fascinating things along the way that they wouldn't encounter otherwise.
Lord Kelvin thought that what makes atoms different is how they're tied into different knots. That turned out to not describe the reality of atoms at all - but gave rise to knot theory[8], which has since gained a fundamental ground in mathematics, particularly - topology (you can construct any 4-manifold by removing a tubular neighborhood of a knot in 3-sphere and gluing it back with some twists - see Dehn Surgery[9]). So, while Kelvin did not find what he was looking for there, the direction his effort has jump-started may, in fact, fundamentally describe our reality - as we have yet to learn what kind of manifold structure our universe has. And knot
Anyway. All in all, good, thought-provoking article (this comment, which contains some thought at least, is a testament to that); I'd recommend looking beyond the title and looking into the things Wolfram mentions. There's a ton of interesting tangents there.
[0] https://romankogan.net/adhd
[1] https://writings.stephenwolfram.com/2017/11/what-is-a-comput...
[2] https://writings.stephenwolfram.com/2024/08/whats-really-goi...
[3] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
[4] https://writings.stephenwolfram.com/category/historical-pers...
[5] https://mathshistory.st-andrews.ac.uk/Miller/mathword/m/
[6] https://www.math.fsu.edu/~wxm/Arnold.htm
[7] https://old.reddit.com/r/math/comments/1f1yblk/evolute_of_an...
[8] https://en.wikipedia.org/wiki/History_of_knot_theory
[9] https://ncatlab.org/nlab/show/Dehn+surgery
A bite-sized idea I liked from the long article: "the very act of exposition was a critical part of organizing and developing my ideas".
I've arrived to the same conclusion for myself (and this article, hopefully, will be the last straw for me to start writing in an organized manner).
My only moderately-successful writing so far has been my ADHD wiki[0], which, in the spirit of Stephen Wolfram, I will shamelessly promote here and now (I've gotten some marvelous feedback from HN in the past, and I believe it to be a useful resource to many).
FWIW Stephen not merely boasting about productivity; a fair bit of the article is dedicated to talking about techniques and tools that he believes help him with that.
While his article on Computational Essays[1] is mostly a Mathematica ad, Mathematica is a great system (which have influenced things like Jupyter notebooks a lot), and the idea of literal programming and interactive data/code/writing is a solid one.
We have yet to have a solid collection of resources like that even for teaching mathematics (where it's natural to play with code to experiment with ideas).
Another bit I liked: and yes, one seems to be able to see the essence of machine learning in systems vastly simpler than neural nets. Sure, we all know about that cellular automata are the Woflram's thing, so it's not surprising to see them pop up in his article about minimal learning computational models[2], but I feel an article like that has been way overdue.
We've been playing with neural nets long enough without having a solid idea what's really going on, and it's limiting to use them as legos of sorts. Why these blocks in particular? What else is out there?
The ChatGPT explainer that he mentioned[1] is still my go-to article for learning about it; I think I'll add another one of his to the list.
Finally, the bit one why history is important - and finding out his archive of writing on history of math and science[4] is great. I believe that history is the most underappreciate science itself, and learning science without its history leaves you without either context or deep understanding of it.
(Personally, I add etymology to history: I find resources like "Earliest Known Uses of Some Mathmatical Terms"[5] invaluable).
And, of course, it's great to see that he's diving into linking mathematics, computation, physics, biology. Great discoveries often lie on the interfaces of various fields.
As Vladimir Arnold wrote in his famous essay[6], divorcing mathematics from physics has been a phenomenal crime. I'm guilty of it too. I recently made a post on reddit[7] with a GIF showing the osculating circles and the evolute of an ellipse. It's pretty, but what is really hiding behind it is the shape of the gear tooth which nearly all gears have.
I learned (and taught!) the mathematics behind it without having any understanding how gears are designed and why they work. And people who make gears make them without understanding the math behind the equations. This came up at work (I'm working on implicit CAD modeling), and from a discussion a better understanding (...and a better product) emerged. There is no reason for narrow specialization that creates barriers in fields that aren't just related, but are necessary for each other - so I believe that the mere fact of Wolfram doing this work is important.
Maybe he'll find out something groundbreaking in those directions. Maybe not. But I can guarantee that he and people around him will stumble into fascinating things along the way that they wouldn't encounter otherwise.
Lord Kelvin thought that what makes atoms different is how they're tied into different knots. That turned out to not describe the reality of atoms at all - but gave rise to knot theory[8], which has since gained a fundamental ground in mathematics, particularly - topology (you can construct any 4-manifold by removing a tubular neighborhood of a knot in 3-sphere and gluing it back with some twists - see Dehn Surgery[9]). So, while Kelvin did not find what he was looking for there, the direction his effort has jump-started may, in fact, fundamentally describe our reality - as we have yet to learn what kind of manifold structure our universe has. And knot
Anyway. All in all, good, thought-provoking article (this comment, which contains some thought at least, is a testament to that); I'd recommend looking beyond the title and looking into the things Wolfram mentions. There's a ton of interesting tangents there.
[0] https://romankogan.net/adhd
[1] https://writings.stephenwolfram.com/2017/11/what-is-a-comput...
[2] https://writings.stephenwolfram.com/2024/08/whats-really-goi...
[3] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
[4] https://writings.stephenwolfram.com/category/historical-pers...
[5] https://mathshistory.st-andrews.ac.uk/Miller/mathword/m/
[6] https://www.math.fsu.edu/~wxm/Arnold.htm
[7] https://old.reddit.com/r/math/comments/1f1yblk/evolute_of_an...
[8] https://en.wikipedia.org/wiki/History_of_knot_theory
[9] https://ncatlab.org/nlab/show/Dehn+surgery
Just wanted to say, your ADHD wiki is part of what made me get a diagnosis in year 7 of a 4 year PhD, the meds of which then allowed me to finish it successfully.
I've since used it to show 6 fellow high functioning AuDHD people, 2 of which were getting into the depression and anxiety spirals that can come with the Systems failing.
Just in my corner of the world, you saved the dreams of at 3 people with your wiki, I would call it slightly more than moderately successful. Thank you so much.
I've since used it to show 6 fellow high functioning AuDHD people, 2 of which were getting into the depression and anxiety spirals that can come with the Systems failing.
Just in my corner of the world, you saved the dreams of at 3 people with your wiki, I would call it slightly more than moderately successful. Thank you so much.
[deleted]
> The ChatGPT explainer that he mentioned[1]
This should be [3]. Thanks for your detailed post, I stopped paying attention to Wolfram years ago but there are still gems to be found in his work under the piles of boasting.
The second sentence seems to have been cut short.Often I realized I really couldn’t have done [the languishing projects] without the tools and ideas (and infrastructure) I now have.
I find this idea of building your own infrastructure to accomplish goals super interesting, especially because already-made software never quite does exactly what you want. Wolfram actually wrote more about that here: https://writings.stephenwolfram.com/2019/02/seeking-the-prod...
I find this idea of building your own infrastructure to accomplish goals super interesting, especially because already-made software never quite does exactly what you want. Wolfram actually wrote more about that here: https://writings.stephenwolfram.com/2019/02/seeking-the-prod...
I imagine that anyone who is serious about anything will end up building/designing some of their own tools. Maybe a custom pedal (or pedal board) for a guitarist, a custom printer for a 3D-printing diehard, or simply designing the tools needed to perform experiments as a scientist. With software, it's a lot easier, especially over long periods of time. I still use a backup script I wrote 20 years ago, for example.
Edit: not to say that the backup script is somehow special, but rather that it being software means that it can last a lifetime with minimal upkeep, unlike the tools made by toolmakers of old.
Edit: not to say that the backup script is somehow special, but rather that it being software means that it can last a lifetime with minimal upkeep, unlike the tools made by toolmakers of old.
Building your own tools to do things is excellent and a potential trap. My brief tangent into language implementation is now a decade deep and I don't clearly remember what I was trying to build in the first place. YMMV.
From the URL I thought it was one of those posts about graphs of activity (“quantified self”) that I’ve seen before from him, but I’m glad I took a look.
Seeing him taking a walk along the coast with shorts, goofy hat, tablet on hand, made me feel sympathy for him. Most of the things he describes doing at home and traveling are things I’ve either done or would have done if I had the means.
Seeing him taking a walk along the coast with shorts, goofy hat, tablet on hand, made me feel sympathy for him. Most of the things he describes doing at home and traveling are things I’ve either done or would have done if I had the means.
Stephen Wolfram has some Q&A streams on youtube where he answers questions off-the-cuff about the history of science. It's remarkable how much he can immediately recall and how easily he makes connections between seemingly unrelated scientific developments. He is a true polymath and his productivity is next level.
I think -- if PBS had the budget or interest -- he'd be a terrific host for a revamped version of Connections[1].
[1] https://en.wikipedia.org/wiki/Connections_(British_TV_series...
[1] https://en.wikipedia.org/wiki/Connections_(British_TV_series...
Except he'd feel the need to inject himself into way too many stories.
Stephen Wolfram is one of my heroes. But not for the usual thing of being really good at science and technology.
He is my hero because he has won capitalism and entrepreneurship. He is incredibly wealthy for all the stuff that a normal person needs, does what he really enjoys and has no shareholders to worry about.
The only other company I know of that is similar is Valve. Both at the cutting edge, doing very interesting things and just leading a meaningful, stressless life.
I am modeling my companies heavily on Wolfram and Valve. May other companies take some notes from them too.
He is my hero because he has won capitalism and entrepreneurship. He is incredibly wealthy for all the stuff that a normal person needs, does what he really enjoys and has no shareholders to worry about.
The only other company I know of that is similar is Valve. Both at the cutting edge, doing very interesting things and just leading a meaningful, stressless life.
I am modeling my companies heavily on Wolfram and Valve. May other companies take some notes from them too.
I like this take. I'd be interested to hear more what you gained from studying them. What ways do you model your companies after Valve and Wolfram?
So the first thing is the meta learning. Looking at companies like Valve and Wolfram, they provide a template of another way of running companies which seem to consistently produce the best kind of software and incredible wealth for all those involved. The two things you look for when running a software company.
Next, Stephen livestreams his day to day as a CEO. This is so significant. I know the HN trope which dang warned about earlier, but I actually love it. Imagine if you could get detailed logs about how Steve Jobs lived his life. Not from books others write about him and make up fake stuff to make it sell more, but straight from the horse's mouth as they say. That is what his meticulous logs and streams of his life provides.
Gabe Newell of course does much less of this, but he still has some incredible videos which go so in depth in how he runs the business and what he thinks about.
Look, we are nerds. To learn business, we go online and try to piece together information. For example, I know for a fact a bunch of YC companies (both in this batch and earlier) have fallen for scammers like Alex Hormozi because he has a massive Youtube presence and just spews nonsense which sounds like it should make sense.
So in that world, to learn as close to first hand from people who actually run some of the biggest and most interesting business on the planet is just incredible.
---
Live CEOing https://livestreams.stephenwolfram.com/category/live-ceoing/
Gabe Newell: On Productivity, Economics, Political Institutions, and the Future of Corporations https://www.youtube.com/watch?v=Td_PGkfIdIQ
Next, Stephen livestreams his day to day as a CEO. This is so significant. I know the HN trope which dang warned about earlier, but I actually love it. Imagine if you could get detailed logs about how Steve Jobs lived his life. Not from books others write about him and make up fake stuff to make it sell more, but straight from the horse's mouth as they say. That is what his meticulous logs and streams of his life provides.
Gabe Newell of course does much less of this, but he still has some incredible videos which go so in depth in how he runs the business and what he thinks about.
Look, we are nerds. To learn business, we go online and try to piece together information. For example, I know for a fact a bunch of YC companies (both in this batch and earlier) have fallen for scammers like Alex Hormozi because he has a massive Youtube presence and just spews nonsense which sounds like it should make sense.
So in that world, to learn as close to first hand from people who actually run some of the biggest and most interesting business on the planet is just incredible.
---
Live CEOing https://livestreams.stephenwolfram.com/category/live-ceoing/
Gabe Newell: On Productivity, Economics, Political Institutions, and the Future of Corporations https://www.youtube.com/watch?v=Td_PGkfIdIQ
Aren’t they both in Champaign IL?
Stephen is in Massachusetts I think.
Valve is in Bellevue, Washington, and Wolfram is in Champaign.
I’ve always envied Wolfram for his work ethic and accomplishments.
I wonder if his complexity ideas from A New kind of science could be applied to software, do you know some applications on that?
I wonder if his complexity ideas from A New kind of science could be applied to software, do you know some applications on that?
I find it absolutely fascinating that from the ages of 60 to 65, this guy produced nine books, 499 hours of podcasts, and 14 software product releases.
For one thing it's quite interesting just to measure your productivity that way. I'm going through life generally just concerning myself with earning my daily bread. In my entire lifetime I've produced zero books, zero hours of podcasts, and a variety of software releases for other people. I think I'd be more proud of myself if I had a fraction of his track record.
It's also amazing that he's doing this at an age when most people are just getting ready for retirement, and seems to have increased his output over the previous 5 year period.
Goals I suppose. And it takes a bit of pressure off of me to think that you can still be this prolific later in life. The five year time horizon is pretty interesting, you can accomplish some pretty dramatic things in five years, and even in middle age you have several more of these 5 year windows remaining.
For one thing it's quite interesting just to measure your productivity that way. I'm going through life generally just concerning myself with earning my daily bread. In my entire lifetime I've produced zero books, zero hours of podcasts, and a variety of software releases for other people. I think I'd be more proud of myself if I had a fraction of his track record.
It's also amazing that he's doing this at an age when most people are just getting ready for retirement, and seems to have increased his output over the previous 5 year period.
Goals I suppose. And it takes a bit of pressure off of me to think that you can still be this prolific later in life. The five year time horizon is pretty interesting, you can accomplish some pretty dramatic things in five years, and even in middle age you have several more of these 5 year windows remaining.
He also has entire teams working for him
> produced nine books
Mmm. Nine books in five years plus doing tons of other stuff? I don’t know anything about the specifics of this situation but have seen enough of the publishing industry and of the influencer/executive/thought-leader side of it that I can assure you the normal way someone in those positions writes nine books in five years is by writing zero books. “Producing” is the right term indeed… maybe. Usually they outsource that to one company or another as well (or start a company for it and have someone else run the company, too—I’ve seen this, lol). They provide an outline and final say on each chapter.
Mmm. Nine books in five years plus doing tons of other stuff? I don’t know anything about the specifics of this situation but have seen enough of the publishing industry and of the influencer/executive/thought-leader side of it that I can assure you the normal way someone in those positions writes nine books in five years is by writing zero books. “Producing” is the right term indeed… maybe. Usually they outsource that to one company or another as well (or start a company for it and have someone else run the company, too—I’ve seen this, lol). They provide an outline and final say on each chapter.
Sure, probably all true. Who cares? "Producing" a book with the help of a team is also a greater career achievement than not producing one. If I look at which I would be more proud of - producing a book or doing nothing - it's pretty obvious.
You can do it how they do it pretty easily with chatGPT. It’s brought the process within reach of the DIYer who doesn’t want to or isn’t able to (time constraints; skill problems) actually write a book.
That’s how the companies that write these for them do it now, so it’s not even a worse process—ghost writers are gone, ChatGPT does the first draft of each chapter, with some of the money saved on writers going to (way) more editor hours than they used to need.
If you’re decent at prompting chatgpt you can do that part yourself, then you can hire the editing out ($5k-$15k, depends on the length of book and how good the editor is) or also DIY that if you’re good at it. Pro tip: keep using ChatGPT during the editing, it’s great for normalizing tone after you’ve changed stuff.
Or if you don’t need to save cash, just do what others in that position do and hire the whole thing out. $25k-$50k and 20-40 hours of personal time spent and you can “write a book”. Depending on the topic and how much you care about the final product, the time investment part can go under 10 hours, if that’s your preference.
That’s how the companies that write these for them do it now, so it’s not even a worse process—ghost writers are gone, ChatGPT does the first draft of each chapter, with some of the money saved on writers going to (way) more editor hours than they used to need.
If you’re decent at prompting chatgpt you can do that part yourself, then you can hire the editing out ($5k-$15k, depends on the length of book and how good the editor is) or also DIY that if you’re good at it. Pro tip: keep using ChatGPT during the editing, it’s great for normalizing tone after you’ve changed stuff.
Or if you don’t need to save cash, just do what others in that position do and hire the whole thing out. $25k-$50k and 20-40 hours of personal time spent and you can “write a book”. Depending on the topic and how much you care about the final product, the time investment part can go under 10 hours, if that’s your preference.
Amazing comment
It's always so sad to see updates from Stephen. The folks who were at the AI lab in the early days describe him as someone so motivated and arrogant that he was certain that one day he'd earn a Nobel. Now all of these years later he's basically wasted his career with delusions of grandure. Sometimes over a beer we wonder what happened. Did he crack because he realized he maybe wasn't the God he thought he was and then created some fantasy land for himself?
My hypothesis and the reason I write this is that he fell prey to what often happens to truly exceptional folks when they get to exceptional places (I've seen it first hand many times).
You show up and discover that there are people who are nominally better than you in every possible way. It takes time to understand that raw talent can eventually be beaten by hard work. And that there are so many problems in the world that ok, so someone is astronomically better than me in every way, fine, I can still change the world by what metrics matter to me if I work hard. And the people who are astronomically better, they often can't handle serious long term failure and overcome it with grit, that's a hard lesson to learn.
There's a very different notion of grit that you have when you're at the top of the pile than at the bottom. And if you've never experienced the latter it can hit you hard.
I'm not writing this to be mean to Stephen. He's a tragic case. But we rarely talk about the effect such places can have on people even though it happens a lot.
My hypothesis and the reason I write this is that he fell prey to what often happens to truly exceptional folks when they get to exceptional places (I've seen it first hand many times).
You show up and discover that there are people who are nominally better than you in every possible way. It takes time to understand that raw talent can eventually be beaten by hard work. And that there are so many problems in the world that ok, so someone is astronomically better than me in every way, fine, I can still change the world by what metrics matter to me if I work hard. And the people who are astronomically better, they often can't handle serious long term failure and overcome it with grit, that's a hard lesson to learn.
There's a very different notion of grit that you have when you're at the top of the pile than at the bottom. And if you've never experienced the latter it can hit you hard.
I'm not writing this to be mean to Stephen. He's a tragic case. But we rarely talk about the effect such places can have on people even though it happens a lot.
*grandeur
CAs are interesting computational systems but it is really difficult to follow the multiway hypergraph rewriting idea and its connection to physics. It sounds reasonable but I am missing a few inductive steps along the way. That may be a me thing... but I also don't know the physics to say one way or another that the outputs he finds match the physical result.
The mutational stuff for parameter search is known already via genetic algorithms which have been employed across the board in optimization, neural networks, machine learning, and even to CAs.
Exploring the space of all 'CAs' (or even the ruliad) by enumerative exhaustion is somewhat interesting but is it similar to defining a FSA by the exposition of all strings accepted by some FSA? If so that seems to be a waste of time. It seems to be a CS version of reading the tea leaves (tasseography). But I may be missing something here as well.
I think this is why he's moved on to "evolution" i.e. mutation with selective constraints (evolution). That seems to be the only way to find something somewhat automatically but it also seems that there may be an infinite number of paths... however, the domain of these functions would be limited by the size of the 'genome' (in classical genetic algorithms) or in the functional inputs of the CA. CA are generally restricted to neighbors and all information is propagated via local neighbors. This limits your rule search space and defines your state space but you could easily have CAs that depend on all neighbors (global state). NNs carry global state and early single layer perceptron's didn't scale until they were made deep (and given additional explicit structure like convolution or transformation)... yes they are all computationally equivalent but one set aren't very useful (even for us trying to understand/reason about) and others result in ChatGPT...
The mutational stuff for parameter search is known already via genetic algorithms which have been employed across the board in optimization, neural networks, machine learning, and even to CAs.
Exploring the space of all 'CAs' (or even the ruliad) by enumerative exhaustion is somewhat interesting but is it similar to defining a FSA by the exposition of all strings accepted by some FSA? If so that seems to be a waste of time. It seems to be a CS version of reading the tea leaves (tasseography). But I may be missing something here as well.
I think this is why he's moved on to "evolution" i.e. mutation with selective constraints (evolution). That seems to be the only way to find something somewhat automatically but it also seems that there may be an infinite number of paths... however, the domain of these functions would be limited by the size of the 'genome' (in classical genetic algorithms) or in the functional inputs of the CA. CA are generally restricted to neighbors and all information is propagated via local neighbors. This limits your rule search space and defines your state space but you could easily have CAs that depend on all neighbors (global state). NNs carry global state and early single layer perceptron's didn't scale until they were made deep (and given additional explicit structure like convolution or transformation)... yes they are all computationally equivalent but one set aren't very useful (even for us trying to understand/reason about) and others result in ChatGPT...
I have explored trying to make global state CA systems, with no success. If anyone has examples of trainable global state CA systems please share, I'd love to learn more.
My idea was that there would be global information but local rules. And the local rules depended on the global information. However I struggled to train such a system and gave up. I would like to go back to it though and some expert perspective might be enough to push me back to it.
I find CA's interesting because they are both continuous and could conceptually vary the amount of compute attributed to a particular problem. NN's can't really do either of these things.
(I appreciate LLM's vary the amount of compute based on the question. But just running the same model over and over seems like an immature method, there must be a better way.)
My idea was that there would be global information but local rules. And the local rules depended on the global information. However I struggled to train such a system and gave up. I would like to go back to it though and some expert perspective might be enough to push me back to it.
I find CA's interesting because they are both continuous and could conceptually vary the amount of compute attributed to a particular problem. NN's can't really do either of these things.
(I appreciate LLM's vary the amount of compute based on the question. But just running the same model over and over seems like an immature method, there must be a better way.)
Thanks for the reply.
How would CAs vary the amount of compute? Don't you have to compute everything state-wise every iteration?
Right now my understanding is that in neural cellular automata people replace the update rule with a DNN. And this DNN is trained on small inputs. Basically a cell's neighborhood input vs a "pixel" vs token level input... a cellular neighborhood here is basically patches which aligns with DNNs anyway.
A good example is: https://distill.pub/2020/growing-ca/
The examples remind of inpainting though in some sense.
You can apply transformers to this to get a shared memory (people have done this I believe).
Too be honest I feel like neural CAs are a trick but I am probably wrong.
How would CAs vary the amount of compute? Don't you have to compute everything state-wise every iteration?
Right now my understanding is that in neural cellular automata people replace the update rule with a DNN. And this DNN is trained on small inputs. Basically a cell's neighborhood input vs a "pixel" vs token level input... a cellular neighborhood here is basically patches which aligns with DNNs anyway.
A good example is: https://distill.pub/2020/growing-ca/
The examples remind of inpainting though in some sense.
You can apply transformers to this to get a shared memory (people have done this I believe).
Too be honest I feel like neural CAs are a trick but I am probably wrong.
My hypothesis is that CA's could have input and output cells, then just run the CA, pass a single or continuous input and then parse the outputs when available.
But - I haven't been able to get such a concept to work. I maybe missing some fundamental theory / understand which prevents such a structure (or at least limits is value).
One major challenge is how do you train an unconstrained process?
Ill take a look at Neural CA's. thanks for sharing.
Thanks for sharing Neural CA's. I'll spend some time on them, much easier to train a DNN.
But - I haven't been able to get such a concept to work. I maybe missing some fundamental theory / understand which prevents such a structure (or at least limits is value).
One major challenge is how do you train an unconstrained process?
Ill take a look at Neural CA's. thanks for sharing.
Thanks for sharing Neural CA's. I'll spend some time on them, much easier to train a DNN.
Wolfram talks about mobile automata.. This is where you calculate one cell at a time.
https://en.wikipedia.org/wiki/Mobile_automaton and https://www.wolframscience.com/nks/p71--mobile-automata/
I'm confused as to how you actually train a cellular automaton? How does one do it? Is it just rule search to match a pattern at time t given input state from time t-1 (and repeated?).. it seems like you'd have a bunch of boolean equations and then look for and extract its solutions.
https://en.wikipedia.org/wiki/Mobile_automaton and https://www.wolframscience.com/nks/p71--mobile-automata/
I'm confused as to how you actually train a cellular automaton? How does one do it? Is it just rule search to match a pattern at time t given input state from time t-1 (and repeated?).. it seems like you'd have a bunch of boolean equations and then look for and extract its solutions.
What, ultimately, did Wolfram do in these five most productive years? Can it be said in a sentence, or a paragraph?
I always find it interesting when someone claims to have something to say, but can't seem to say it at a short enough length to be held entirely in the human mind.
It's like Bob says he has a beautiful sculpture to show me, but when I ask to see it, he sends me truckload after truckload of modeling clay. The sculpture is constituted from this material, Bob tells me.
Okay, I say, but it seems like you're asking me to do an awful lot of work here, and I don't know you enough to be confident that the effort would be well-spent. So could you maybe show me a complete, smaller scale model, with a bit less detail, that gives me the general idea of what's so special and new about what you've made?
I always find it interesting when someone claims to have something to say, but can't seem to say it at a short enough length to be held entirely in the human mind.
It's like Bob says he has a beautiful sculpture to show me, but when I ask to see it, he sends me truckload after truckload of modeling clay. The sculpture is constituted from this material, Bob tells me.
Okay, I say, but it seems like you're asking me to do an awful lot of work here, and I don't know you enough to be confident that the effort would be well-spent. So could you maybe show me a complete, smaller scale model, with a bit less detail, that gives me the general idea of what's so special and new about what you've made?
Doesn't the picture at the top give a good summary?
No. "I published XX books and YYY articles" doesn't tell me anything about the actual value. Some authors I heard publish a 3-digit-number articles per year...
> What, ultimately, did Wolfram do in these five most productive years? Can it be said in a sentence, or a paragraph?
He shipped a lot of software, wrote a lot of books and papers, did a lot of media, and substantially advanced his passion projects.
It's unclear how much of the last stuff will pan out and advance human knowledge, but it was still done.
He shipped a lot of software, wrote a lot of books and papers, did a lot of media, and substantially advanced his passion projects.
It's unclear how much of the last stuff will pan out and advance human knowledge, but it was still done.
He did a laws of physics thing https://en.wikipedia.org/wiki/Stephen_Wolfram#Wolfram_Physic... though "Physicists are generally unimpressed." (wikipedia)
[deleted]
[deleted]
> Nine books. 3939 pages of writings (1,283,267 words). 499 hours of podcasts and 1369 hours of livestreams. 14 software product releases (with our great team). Oh, and a bunch of big—and beautiful—ideas and results.
That’s from the first paragraph - key nouns like “books”, “podcasts” and the like link to backup for the numbers.
This is amazing productivity - a book is a serious amount of work. Nine in five years though? Thats something else entirely.
That’s from the first paragraph - key nouns like “books”, “podcasts” and the like link to backup for the numbers.
This is amazing productivity - a book is a serious amount of work. Nine in five years though? Thats something else entirely.
I suspect he’s optimizing everything he does to be immediately suitable for publication. He’s not so much publishing the results of his research as he is publishing the process of it. I also suspect that he is driven by a deep-seated need to prove his worth to the world, a need that he has to continuously satisfy (but is never really satisfied), that motivates this level and method of activity.
And that's without using LLMs.
A noteworthy part of this post is the very tall image on the right. It really is very tall indeed (50 × 32050).
Direct link: https://content.wolfram.com/sites/43/2024/08/blog-image-stri...
It’s either too small or a blur on a high-DPI display, however.
It’s either too small or a blur on a high-DPI display, however.
Had to go back to view it as it wasn't visible on my screen width.
> My goal is to create a general book—and course—that’s an introduction to computational thinking at a level suitable for typical first-year college students. Lots of college students these days say they want to study “computer science”. But really it’s computational X for some field X that they’re ultimately interested in. And neither the theoretical nor the engineering aspects of typical “computer science” are what’s most relevant to them. What they need to know is computational thinking as it might be applied to computational X—not “CS” but what one might call “CX”.
Sounds fun!
Sounds fun!
[deleted]
Meta: dang here is incorrect.
Each post could be the first time someone is encountering a Wolfram post. If they haven't built up defenses to this kind of self aggrandizement they might believe his words. This, and every piece from him deserves a community note.
Just because you've seen a lot of misinformation from a source in the past is no excuse to stop labeling it as such.
Each post could be the first time someone is encountering a Wolfram post. If they haven't built up defenses to this kind of self aggrandizement they might believe his words. This, and every piece from him deserves a community note.
Just because you've seen a lot of misinformation from a source in the past is no excuse to stop labeling it as such.
Coming from Physics the first time I heard detailed information about Mathematica (in contrast to Maple) and the Physics related work from a Professor it was also accompanied with a huge disclaimer. Reflecting further on this, a lot of this is very niche and seems to be not explored fully in the spirit of peer review
dang posted a link at the top to such information for those who are unfamiliar, which serves as the note you want without detracting from other discussion
[stub for offtopicness]
Can someone tell me what ideas of Wolfram have had an impact on mathematics & physics as a discipline? Every time I read his posts, without understanding them deeply, his accomplishments seem insane, to the point that I would expect everyone to talk about them - yet I have never heard anyone other than himself talk about them.
I don't recall ever reading anyone else who considered themselves a part of the giant nexus we call science and managed to place themselves always at the center of just about every story they had to tell.
The level of "I-ness" in Wolfram's writing is really an incredible outlier, at least in my experience of science writing (academic and popular).
The level of "I-ness" in Wolfram's writing is really an incredible outlier, at least in my experience of science writing (academic and popular).
So that's why I had to read Anthem in High School.
I'm very glad that I did, and while I agree in a generally professional context it is more socially acceptable to be humble and focus on what "we" did, in what amounts to a public autobiographical memoir, I expect to read a lot of "I" telling me what they themselves did.
Besides, there are many instances where the author invokes "we" in the article starting with these:
> 14 software product releases (with our great team).
> we embarked on our Physics Project
> We announced what we’d figured out in April 2020
I'm very glad that I did, and while I agree in a generally professional context it is more socially acceptable to be humble and focus on what "we" did, in what amounts to a public autobiographical memoir, I expect to read a lot of "I" telling me what they themselves did.
Besides, there are many instances where the author invokes "we" in the article starting with these:
> 14 software product releases (with our great team).
> we embarked on our Physics Project
> We announced what we’d figured out in April 2020
I wasn't suggesting that Wolfram never acknowledges a team or never uses "we".
It is just a question of ratios. Wolfram's writing/blogging seems to assume we want the inside scoop on what it is like to be Steven Wolfram. There's nothing inherently wrong with that - plenty of people want the inside scoop on what it is like to be some entertainment celebrity, so why not a scientist-business-y person - but it is unusual for the disciplines Wolfram considers himself a part of.
It is just a question of ratios. Wolfram's writing/blogging seems to assume we want the inside scoop on what it is like to be Steven Wolfram. There's nothing inherently wrong with that - plenty of people want the inside scoop on what it is like to be some entertainment celebrity, so why not a scientist-business-y person - but it is unusual for the disciplines Wolfram considers himself a part of.
Within the sea glass collecting subculture there is split between those who believe in seeding and those against.
Most sea glass is quite old, 60+ years, from a time when rubbish was thrown in the sea, so it's running out.
Seeding new glass would be needed to keep the hobby alive.
It's quite the 'what it sea glass' dilemma. (It also risks contaminating history for the sub-sub culture, historic collectors)
What to me is very wrong is some people seeding with fake sea glass, glass already rounded, or manufactured rounded.
To me there needs there to be a story behind sea glass seeding, like Banksy doing a (insert topical region) bottle dump, or something like that. Or an old cranky dude who lived locally but is now dead who'd throw them off this canoe.
It reminds me a little of Ann Clayborne in the Mars Trilogy, of course she'd be against all sea glass and probably beaches as we know them in general.
Also Wolfram, if you want to do something else useful please make you great website https://www.wolframalpha.com/ do napkin math, currently it's next to useless but it could be really great.
Most sea glass is quite old, 60+ years, from a time when rubbish was thrown in the sea, so it's running out.
Seeding new glass would be needed to keep the hobby alive.
It's quite the 'what it sea glass' dilemma. (It also risks contaminating history for the sub-sub culture, historic collectors)
What to me is very wrong is some people seeding with fake sea glass, glass already rounded, or manufactured rounded.
To me there needs there to be a story behind sea glass seeding, like Banksy doing a (insert topical region) bottle dump, or something like that. Or an old cranky dude who lived locally but is now dead who'd throw them off this canoe.
It reminds me a little of Ann Clayborne in the Mars Trilogy, of course she'd be against all sea glass and probably beaches as we know them in general.
Also Wolfram, if you want to do something else useful please make you great website https://www.wolframalpha.com/ do napkin math, currently it's next to useless but it could be really great.
I never knew this or had thought about it. My girlfriend is a great lover of sea glass, I had no idea it was a finite resource.
Perfectly off-topic, well done.
johnwheeler(4)
Needs a TLDR - the text is long but there's not much structure.
The issue is not that it's wrong, it's that it's extremely repetitive and we want fresh discussion on HN, preferably about the specific content of an article.