Reminder of Complexity(johndcook.com)
johndcook.com
Reminder of Complexity
https://www.johndcook.com/blog/2020/01/16/memento-complexitatis/
38 comments
> Once you see that any organism that involved organic chemistry is in fact an insanely complicated, biological computer you cannot unsee it. It's amazing.
This blew my mind. In addition to the computer analogy, I think Bio-mechanics is incredibly fascinating. The fact that I can close my eyes and still be able to touch the nose with my index finger repetedly from any starting position of the hand is mind boggling. I cannot fathom the type of inverse-kinematics that is involved in the cerebellum.
This blew my mind. In addition to the computer analogy, I think Bio-mechanics is incredibly fascinating. The fact that I can close my eyes and still be able to touch the nose with my index finger repetedly from any starting position of the hand is mind boggling. I cannot fathom the type of inverse-kinematics that is involved in the cerebellum.
hah. technically your brain is a prediction machine and, when you are trying to touch your nose, it’s really good at minimizing the prediction error.
Also you're not just relying on sight, the sense of proprioception is providing constant feedback of your finger's position even with your eyes closed. So it's not an open loop control problem, it's a closed loop, which is much easier.
Ok as someone who has been programming computers since age five (and currently in the industry) and has a PhD in biochemistry... It's not a computer. If you think of it that way you can wind up in deep trouble with critical nuances. That said, it can be fruitful to treat a small percentage of the systems (my gut says 10%), and exploit their computer-like features for fun and profit.
I'd like to hear more about this.
My understanding is that these things are not Turing complete, and don't resemble Von Neumann machines (or any other architecture we use for computers). But the processes themselves can be understood in terms of computation and/or communication.
Is that wrong? If so, why/how?
Also, could you point out an example or two of the nuances that tend to get people in trouble?
My understanding is that these things are not Turing complete, and don't resemble Von Neumann machines (or any other architecture we use for computers). But the processes themselves can be understood in terms of computation and/or communication.
Is that wrong? If so, why/how?
Also, could you point out an example or two of the nuances that tend to get people in trouble?
The big one is it's nondeterministic. An obvious example of nondeterminism is the genotype which determines mirror-body, it's caused by a defective cilium. Normally that cilium enforces torquoselectivity -it only rotates in one direction which establishes chiral directionality, but a certain genotype disables its rotation, so the orientation of your body is a factor of random chance based on how your cell was rotating at the single cell level. The mendelian inheritance is basically 50/50 stochastic based on the genotype.
That's a pretty macro example (both at the genotypic ratio and phenotype levels), but it can happen at the micro level, too. You can have grossly nonlinear effects (well actually many effects are O(n^k) in concentration n, where k is the oligomer size, 2->12->20++) and sometimes even the phenotypically important species is like 10%? 1%? 0.1%? 0.01%? of the molecule population. And the proportion of the species could depend on the temperature. Are you running at 35C? 38C? 40C? 42C?
In a reductionistic way you could say that any system that follows logical rules is "like a computer" but I think at that point it ceases to be a meaningful statement.
Computers are great because if you give them an input they can give a deterministic answer (you can also choose to have them not give a deterministic answer, as in ML, and yes sometimes at Google scale you have to deal with cosmic radiation) but the choice is inverted for biological systems. You can try really hard to make biological systems give deterministic answers, and in some systems you don't have to try that hard, especially nowadays, but the default still is "nondeterministic".
That's a pretty macro example (both at the genotypic ratio and phenotype levels), but it can happen at the micro level, too. You can have grossly nonlinear effects (well actually many effects are O(n^k) in concentration n, where k is the oligomer size, 2->12->20++) and sometimes even the phenotypically important species is like 10%? 1%? 0.1%? 0.01%? of the molecule population. And the proportion of the species could depend on the temperature. Are you running at 35C? 38C? 40C? 42C?
In a reductionistic way you could say that any system that follows logical rules is "like a computer" but I think at that point it ceases to be a meaningful statement.
Computers are great because if you give them an input they can give a deterministic answer (you can also choose to have them not give a deterministic answer, as in ML, and yes sometimes at Google scale you have to deal with cosmic radiation) but the choice is inverted for biological systems. You can try really hard to make biological systems give deterministic answers, and in some systems you don't have to try that hard, especially nowadays, but the default still is "nondeterministic".
technically what we call computers are nondeterministic. it just so happens that the way we model the hardware and feed/extract data gives us predictability. There are a lot of things going on, even in silicon.
as far as biology goes, i can take any control structure you have in code and show you examples of how it’s being done. just because it’s happening at a large scale and it’s hard to predict at a micro level does not mean it’s not a computation and it does not mean it’s non-deterministiv
as far as biology goes, i can take any control structure you have in code and show you examples of how it’s being done. just because it’s happening at a large scale and it’s hard to predict at a micro level does not mean it’s not a computation and it does not mean it’s non-deterministiv
I see, should I have covered contrapositive of my statement?
technically everything in the universe is nondeterministic but to say "but also computers are nondeterministic" is reductionistic to the point where the categories are meaningless.
It's also not sufficient to say "computation" exists. Well, you could reductionistically describe the orbits of the planets as a very machine that is very good at one computation. So I guess that makes it a computer. The solar system is a computer.
Going beyond the question of determinism, similarly saying you can encode arbitrary computation to an arbitrary degree of determinism (which you can, in biology), is not helpful. This is like pointing to the Game Of Life, and showing that people can build computers in the game of life, and saying that the emergent phenomena are "just like a computer". Well yes, there is a computer underneath, but I think you're probably missing something important if you think the parameter and output spaces of game of life as "just like a computer".
technically everything in the universe is nondeterministic but to say "but also computers are nondeterministic" is reductionistic to the point where the categories are meaningless.
It's also not sufficient to say "computation" exists. Well, you could reductionistically describe the orbits of the planets as a very machine that is very good at one computation. So I guess that makes it a computer. The solar system is a computer.
Going beyond the question of determinism, similarly saying you can encode arbitrary computation to an arbitrary degree of determinism (which you can, in biology), is not helpful. This is like pointing to the Game Of Life, and showing that people can build computers in the game of life, and saying that the emergent phenomena are "just like a computer". Well yes, there is a computer underneath, but I think you're probably missing something important if you think the parameter and output spaces of game of life as "just like a computer".
i believe there is a disconnect between “is like a computer” and “is like a computer that i can understand and program”. if you were going for the latter i agree, we are not there yet.
but it you define a computer as something that runs on some sort of code and translates input to output literally everything can be seen as a computer. the more complex something is the more impressive it is.
you should also probably update your profile description since it’s literally confirming what i am saying :)) (ie I've been: A molecular biologist that discovered that an enzyme was secretly an NPN transistor)
but it you define a computer as something that runs on some sort of code and translates input to output literally everything can be seen as a computer. the more complex something is the more impressive it is.
you should also probably update your profile description since it’s literally confirming what i am saying :)) (ie I've been: A molecular biologist that discovered that an enzyme was secretly an NPN transistor)
An npn transistor is not a computer, and in this case it's literal. The molecule had an negative, a positive, and a negative iron sulfur clusters. However, it most certainly wasn't acting in a logic gate; if anything it's activity functionally was closer to a diode than a transistor, by blocking bulk current flow in one direction but not the other (but, for quantum reasons, it has to be a transistor to do that). There were no data transformations in it's activity.
To wit: I didn't get my job done on that project because I thought of it as a computer, I got it done because having been a curious child who also did computers I happened to know (and remember into adulthood) how computers work at a very low level and was able to make an insight based on that, nothing particularly CSey.
To wit: I didn't get my job done on that project because I thought of it as a computer, I got it done because having been a curious child who also did computers I happened to know (and remember into adulthood) how computers work at a very low level and was able to make an insight based on that, nothing particularly CSey.
an npn transistor can be regarded as a computer. it's like a digital switch. was there something special that made you think npn or could it have also been a pnp transistor? (ie it was just acting as a transistor)
anyways, we may disagree but you do sound like a fascinating person. molecular biology, CS, elixir. that's a rare but exciting combo. what materials would you recommend for someone who is a rookie but wants to learn more about molecular biology?
anyways, we may disagree but you do sound like a fascinating person. molecular biology, CS, elixir. that's a rare but exciting combo. what materials would you recommend for someone who is a rookie but wants to learn more about molecular biology?
It's npn, as I said, because there was a positively charged iron sulfur cluster between two negatively charged iron sulfur clusters. In a row. Like literally it could not be a pnp element. And not all switches are computers. You wouldn't call your wall lightswitch a computer, because in the general case, the current running through it is not meaningfully carrying data.
I don't really recommend 'learning' molecular biology. There really is no substitute for designing a molecular biology experiment, starting with dna synthesis, through cloning, and seeing it through to its biochemical outcome. It's not terribly time consuming, I had an intern that I took through the steps in 3 months, starting from no lab experience he completed 50 mutations on the enzyme, tested half of them, and got a paper out of it.
I don't really recommend 'learning' molecular biology. There really is no substitute for designing a molecular biology experiment, starting with dna synthesis, through cloning, and seeing it through to its biochemical outcome. It's not terribly time consuming, I had an intern that I took through the steps in 3 months, starting from no lab experience he completed 50 mutations on the enzyme, tested half of them, and got a paper out of it.
all switches are computers :) you have an input, a state and an output. if i'm hearing what you're saying correctly, you seem to have a complexity barrier that something must reach to be called a computer. i'm more liberal when it comes to defining a computer.
why don't you recommend learning molecular biology? is it hard to grasp without the practical experiments? what's your thinking behind "i don't recommend it"?
why don't you recommend learning molecular biology? is it hard to grasp without the practical experiments? what's your thinking behind "i don't recommend it"?
Your comments will hold weight when you start paying your electrician 6 figures.
This video blew me away:
https://www.youtube.com/watch?v=wJyUtbn0O5Y
It's an animation of the bio-chemistry a lymphocyte uses to move between cells (maybe - that's a guess?). It covers what I can only assume is a few milliseconds, it's just a tiny slither of the cells behaviours, yet how much is going on to make that happen is off the charts. I showed in it a club consisting of mostly software engineers, and the universal reaction was "thank god I didn't do bio-chemistry".
https://www.youtube.com/watch?v=wJyUtbn0O5Y
It's an animation of the bio-chemistry a lymphocyte uses to move between cells (maybe - that's a guess?). It covers what I can only assume is a few milliseconds, it's just a tiny slither of the cells behaviours, yet how much is going on to make that happen is off the charts. I showed in it a club consisting of mostly software engineers, and the universal reaction was "thank god I didn't do bio-chemistry".
This reminds me of Tesler's law says something quite useful about how complexity works:
"Every application has an inherent amount of complexity that cannot be removed or hidden. Instead, it must be dealt with, either in product development or in user interaction."
"Every application has an inherent amount of complexity that cannot be removed or hidden. Instead, it must be dealt with, either in product development or in user interaction."
You can download the posters here:
https://www.roche.com/sustainability/philanthropy/science_ed...
Half-related: Reality has a surprising amount of detail
http://johnsalvatier.org/blog/2017/reality-has-a-surprising-...
https://news.ycombinator.com/item?id=16184255
https://news.ycombinator.com/item?id=22020495
http://johnsalvatier.org/blog/2017/reality-has-a-surprising-...
https://news.ycombinator.com/item?id=16184255
https://news.ycombinator.com/item?id=22020495
Related:
https://arxiv.org/abs/0708.1362
At some point the complexity of a system becomes great enough that it's impossible to simulate it perfectly.
The interesting question for me lately is how and under what circumstances can you ignore the complexity for some purpose.
For example, underlying what is represented in that poster is some set of physical processes that are ignored in that poster. The system at that physical level of description would be even more complex, probably too complex to represent on a poster. So why the level of analysis of the poster? Similarlly, at some point it's easier to talk about eating and fatigue than it is the citric acid cycle. How and why do you move from one level of analysis to another? Some of it probably depends on what is being explained, but some of it might not.
https://arxiv.org/abs/0708.1362
At some point the complexity of a system becomes great enough that it's impossible to simulate it perfectly.
The interesting question for me lately is how and under what circumstances can you ignore the complexity for some purpose.
For example, underlying what is represented in that poster is some set of physical processes that are ignored in that poster. The system at that physical level of description would be even more complex, probably too complex to represent on a poster. So why the level of analysis of the poster? Similarlly, at some point it's easier to talk about eating and fatigue than it is the citric acid cycle. How and why do you move from one level of analysis to another? Some of it probably depends on what is being explained, but some of it might not.
Interesting looking paper. From the abstract it seems that the claim is about about perfectly replicating a future state of the simulated system? Simulation with a less ambitious fidelity level still seems useful to understand these kind of systems.
Yes, that is called a model.
Usually people that study complexity will say that all models are wrong but some are useful.
In software, we usually prefer the term abstraction.
The trick is to remember that our model are wrong. This gap between "system-as-modeled" and "system-as-reality" can be inconsequential in a lot of time, but it is also a place in which failure mode can rise. It is important to always keep exploring it and change the model depending on situations.
Usually people that study complexity will say that all models are wrong but some are useful.
In software, we usually prefer the term abstraction.
The trick is to remember that our model are wrong. This gap between "system-as-modeled" and "system-as-reality" can be inconsequential in a lot of time, but it is also a place in which failure mode can rise. It is important to always keep exploring it and change the model depending on situations.
John’s blogs always set me off in one direction or another. Now he has me thinking:
Our human minds can only hold a finite amount of information in long term and short term memories. I don’t too much buy into direct brain augmentation in the sense of NeuralLink or my friends in the SingularityNET community.
What I am hoping for is external augmentation using smart glasses that record everything I study and that I hear, and sufficient AI (doesn’t have to be AGI) on the backend to learn when to prompt or remind. Ideally this would be paired with something like an iPad that the system could draw on top of what I am looking at.
This is the startup I would like to work for. Apple is supposed to be releasing smart glasses sometime. I hope the hardware and software support is hackable and extensible.
Our human minds can only hold a finite amount of information in long term and short term memories. I don’t too much buy into direct brain augmentation in the sense of NeuralLink or my friends in the SingularityNET community.
What I am hoping for is external augmentation using smart glasses that record everything I study and that I hear, and sufficient AI (doesn’t have to be AGI) on the backend to learn when to prompt or remind. Ideally this would be paired with something like an iPad that the system could draw on top of what I am looking at.
This is the startup I would like to work for. Apple is supposed to be releasing smart glasses sometime. I hope the hardware and software support is hackable and extensible.
I expect smart glasses to be the next UI paradigm. Very conveniently portable. AR style visual interfaces. Built in speakers for personal audio (bose already do this in sunglasses). The AR interface could be directly manipulated by hand gestures using cameras and other sensors. I know there have been a couple of false starts (Google Glass) and current products with bigger form factors (Hololens), but I expect spectacles size devices with good UI in the not too distant future, and I look forward to the development opportunities.
The problem that I see is getting systems where I own all of my data - my data never leaves my devices, or is encrypted on my servers.
I used to be a proponent of IoT until realizing that IoT is just a HUGE opportunity to gather the most economic and personal data on people stupid enough to pay for the devices that surveil them. Sort of like paying someone to kick you.
I used to be a proponent of IoT until realizing that IoT is just a HUGE opportunity to gather the most economic and personal data on people stupid enough to pay for the devices that surveil them. Sort of like paying someone to kick you.
That AI would need to learn what you find important though, and go beyond a "this user has an interest in subject x". So you need to bookmark / store and curate your own content first to get enough material. And even then, your interests will shift over time; this year for example I have an interest in the Go language, but for all I know next year I'm interested in Java again and need a refresher on stuff I've learned ten years ago + what I missed (because I can't keep up to date with language X if I'm working in language Y).
That’s really more of a “flying cars” startup than a “140 characters” one. You should pitch the idea :P
Someone just emailed me with the idea of making it open source. That would work if/when there is commodity and hackable hardware to support it.
There are some movements in this area with the inevitable and inexorable march of progress that drops the prices of high-speed low-rez cameras using mostly off-the-shelf components. If video can be lossy while key features from the feed are processed in-stream, one can get creative with high frame rates (>2000 fps, 256x256 monochrome).[1]
[1] https://journals.plos.org/plosone/article?id=10.1371/journal...
[1] https://journals.plos.org/plosone/article?id=10.1371/journal...
I love that it comes with a piece of paper reminding the viewer that this is a small selection of reality. Biochemistry is much, MUCH more complex than even these two 26 square foot posters.
I'd love to get a similar poster of the entire Linux kernel or, alternatively, the total supply chain for an everyday item like the proverbial pencil.
I'd love to get a similar poster of the entire Linux kernel or, alternatively, the total supply chain for an everyday item like the proverbial pencil.
What perfect timing: https://news.ycombinator.com/item?id=23452721
the complexity of the linux kernel (and in man-made things in general) is usually really small compared to biological systems.
I'm sure you could still fill a good-size poster if you make a nice inefficient representation, like every function call is a separate node in the graph and you include every single device driver even for decades old stuff like tape drives. I agree though, Mother Nature has an inordinate love for making everything a global variable and She definitely hates comments.
the analogy is good but i think it kinda breaks down when you consider the machine the kernel runs on.
so you would mostly represent the machine the software run on.
so you would mostly represent the machine the software run on.
I'd buy a similar poster of the Linux kernel.
I'm sure some astute HN reader could whip up a script that generates illustrative posters of static call graphs given a Git repo!
I'm sure some astute HN reader could whip up a script that generates illustrative posters of static call graphs given a Git repo!
https://makelinux.github.io/kernel/map/
my uni had some different, older posters about linux. one was radial with most non driver files represented. can’t seem to find them on the web though
my uni had some different, older posters about linux. one was radial with most non driver files represented. can’t seem to find them on the web though
http://www.visualcomplexity.com/vc/project_details.cfm?id=26... exists, not what the poster was, but WHOA way cooler. also, old.
I ran across an early version of this, on paper, in the university library at my undergraduate institution, and took the same lesson of humility and respect in the face of complexity that the article does.
The positive side is that we actually can and do effectively intervene in this system, and systems like it, to improve outcomes, so even very high levels of complexity don't mean analysis is impossible.
The positive side is that we actually can and do effectively intervene in this system, and systems like it, to improve outcomes, so even very high levels of complexity don't mean analysis is impossible.
As someone who is formally trained in computer science I was literally blown away by biochemistry. Once you see that any organism that involved organic chemistry is in fact an insanely complicated, biological computer you cannot unsee it. It's amazing.
In case someone is wondering what the author of the blog post zoomed in, that's the Krebs cycle - ie how we derive energy (ie ATP) from the food we eat (https://en.wikipedia.org/wiki/Citric_acid_cycle). See: https://www.youtube.com/watch?v=juM2ROSLWfw and https://www.youtube.com/watch?v=7J4LXs-oDCU