Alan Kay Answers “Is There a Successor to Smalltalk?”(quora.com)
quora.com
Alan Kay Answers “Is There a Successor to Smalltalk?”
https://www.quora.com/Is-there-a-programming-language-thats-effectively-a-successor-to-Smalltalk/answer/Alan-Kay-11?share=1
23 comments
To be fair, the poor "memory bandwidth" of CPUs has not been fought very hard and is not really a direct consequence of Moore's law.
Constantly growing caches just work very well, and we've grown to treat the DRAM level of the memory hierarchy as just another storage device, faster than your SSD but slower than your L4 cache. The industry has just emphasized capacity/$ and traditional, commodity DRAM protocols. Consider for example the potential, and near-absence[1], of CPU + DRAM MCMs.
[1] The latest rebranding of this obvious idea is "HBM" in GPUs. Though these don't really take advantage of the idea. But still CPUs have not been interested in even reaching this little distance.
Constantly growing caches just work very well, and we've grown to treat the DRAM level of the memory hierarchy as just another storage device, faster than your SSD but slower than your L4 cache. The industry has just emphasized capacity/$ and traditional, commodity DRAM protocols. Consider for example the potential, and near-absence[1], of CPU + DRAM MCMs.
[1] The latest rebranding of this obvious idea is "HBM" in GPUs. Though these don't really take advantage of the idea. But still CPUs have not been interested in even reaching this little distance.
There's a big difference between Smalltalk implementations being relatively slow and Smalltalk being intrinsically slow.
If Smalltalk VMs had the same resources thrown at them as other VMs/compilers/etc. they could be more competitive speed-wise. Noting, of course, that Smalltalk systems can be plenty fast-enough.
Smalltalk-as-glue is also perfectly reasonable, interfacing with multiple back-end systems, controlling business logic, etc.
If Smalltalk VMs had the same resources thrown at them as other VMs/compilers/etc. they could be more competitive speed-wise. Noting, of course, that Smalltalk systems can be plenty fast-enough.
Smalltalk-as-glue is also perfectly reasonable, interfacing with multiple back-end systems, controlling business logic, etc.
What I'm saying is that the "lots of little objects" model of Smalltalk goes against the grain of modern machines and how they access memory. I'm not sure what it takes for something to be _intrinsically slow_ but it appears physics is against us. If we could build machines that had fast uniform access to memory we probably would have.
Consider Concurrent Smalltalk on the http://cva.stanford.edu/projects/j-machine/ -- a processor-in-memory massively parallel architecture. This seems like a better match to physics than a Von Neumann architecture with a few fast processors separated from a giant memory, because physics is local. I suspect systems like the J-machine haven't gotten very far for historical path-dependent reasons more than intrinsic ones.
It's true though that a parallel object language like Concurrent Smalltalk isn't the historical Smalltalk we usually mean.
It's true though that a parallel object language like Concurrent Smalltalk isn't the historical Smalltalk we usually mean.
I think the current mindset of taking data layout as fixed is a big illusion that we just haven't broken out of. Optimizers should make data layout optimizations. For example, a JIT could do AoS->SoA optimizations on an array containing multi-field objects and vectorize loops that manipulate a small subset of the fields. Or replace object representations with limited, live data-specialized versions that consume a fraction of the space.
There may be some changes to language semantics needed of course, but doesn't preclude Smalltalky languages.
Another property of the current software/hardware world is that the vast majority of programs don't care about performance at this level. There is a vast size difference between the oceans of production code in the world, and the amount of "hot" code that people pay money to make faster.
(And the levels of "make faster" are usually more low hanging fruit than switch of programming language)
There may be some changes to language semantics needed of course, but doesn't preclude Smalltalky languages.
Another property of the current software/hardware world is that the vast majority of programs don't care about performance at this level. There is a vast size difference between the oceans of production code in the world, and the amount of "hot" code that people pay money to make faster.
(And the levels of "make faster" are usually more low hanging fruit than switch of programming language)
Citing performance (especially related to micro-optimization) everytime we discuss language design is not a right thing to do. Most applications don't require or benefit from such micro-optimization(tight control of layout).
> but it was designed in an era where hardware had very different performance characteristics from today's hardware.
Exactly, the era of 256k memory computers are over. Programmer time and correctness is more valuable than making sure everything fits in L1 cache.
> but it was designed in an era where hardware had very different performance characteristics from today's hardware.
Exactly, the era of 256k memory computers are over. Programmer time and correctness is more valuable than making sure everything fits in L1 cache.
You're missing my point. When we had 256k memory computers we didn't have caches because memory access was uniform. As time goes on, fitting in cache is getting more important, not less.
The difference between random access and linear access is easily 10x, possibly more.
Given how much time your average huge application (Word, Photoshop…) takes to even start up, I think this is still an important point.
Programmer time is important, but so is user time. And in many cases, there are many more users than programmers.
Given how much time your average huge application (Word, Photoshop…) takes to even start up, I think this is still an important point.
Programmer time is important, but so is user time. And in many cases, there are many more users than programmers.
Smalltalk inspired the languages I grew up with. I'd say that they're worthy successors.
I was introduced to programming at school using LOGO when I was 7 years old. My dad found a broken Mac Plus and told me I could keep it if I could fix it. So I learned basic hardware repair (swap out the motherboard). Having my own computer gave me a sense of responsibility and independence that I never had before.
When I was 11, he bought me an iBook. I also saved up my pocket money (allowance) for an iPod.
I wanted more apps on the iPod, and was interested in the Notes feature. I taught myself AppleScript to scrape websites and convert the HTML to 1000-character iPod Notes TXT files. This included my English textbooks - I read Shakespeare on an iPod in 2003 so I didn't need to carry a heavy book. I also wrote many scripts for iTunes.
All of this was self-taught, without textbooks or YouTube or tutorials online. It was even before StackOverflow. How did I learn?
The syntax is easy. There's a single button to compile & run. The code is auto-formatted in colour. And there are lots of APIs to every other app.
As a Computer Science class project, I spent months writing a database program for SMS messages using Java. But for myself, I spent half a day coding a converter AppleScript to put those SMS text files into blank MP3s and store them with iTunes. All the fancy features (smart playlists, playlist folders) were already done!
Programming is not scripting - you have to use libraries instead of leveraging other apps. But I see the successor to SmallTalk being a scripting language with a lot of APIs. Like AppleScript, but with better support for cross-platform and GUIs.
I was introduced to programming at school using LOGO when I was 7 years old. My dad found a broken Mac Plus and told me I could keep it if I could fix it. So I learned basic hardware repair (swap out the motherboard). Having my own computer gave me a sense of responsibility and independence that I never had before.
When I was 11, he bought me an iBook. I also saved up my pocket money (allowance) for an iPod.
I wanted more apps on the iPod, and was interested in the Notes feature. I taught myself AppleScript to scrape websites and convert the HTML to 1000-character iPod Notes TXT files. This included my English textbooks - I read Shakespeare on an iPod in 2003 so I didn't need to carry a heavy book. I also wrote many scripts for iTunes.
All of this was self-taught, without textbooks or YouTube or tutorials online. It was even before StackOverflow. How did I learn?
The syntax is easy. There's a single button to compile & run. The code is auto-formatted in colour. And there are lots of APIs to every other app.
As a Computer Science class project, I spent months writing a database program for SMS messages using Java. But for myself, I spent half a day coding a converter AppleScript to put those SMS text files into blank MP3s and store them with iTunes. All the fancy features (smart playlists, playlist folders) were already done!
Programming is not scripting - you have to use libraries instead of leveraging other apps. But I see the successor to SmallTalk being a scripting language with a lot of APIs. Like AppleScript, but with better support for cross-platform and GUIs.
> The syntax is easy
That made me chuckle. I've tried Applescript from time to time but always given up with it in frustration having failed after repeated attempts to find whatever particular Englishey construction might do what I want (or anything for that matter). Perhaps its approach doesn't work well for programmers (or just for me).
That made me chuckle. I've tried Applescript from time to time but always given up with it in frustration having failed after repeated attempts to find whatever particular Englishey construction might do what I want (or anything for that matter). Perhaps its approach doesn't work well for programmers (or just for me).
AppleScript makes me mental; it tries to hard to be the COBOL of scripting languages. It might be okay for some people, but if I never had to think about it again I'd be a happier person.
Not sure what you mean by "programming is not scripting". Scripting is most certainly programming, and whether or not programming-is-not-scripting depends on what you're doing.
I've never seen any truly meaningful distinction between the two.
Not sure what you mean by "programming is not scripting". Scripting is most certainly programming, and whether or not programming-is-not-scripting depends on what you're doing.
I've never seen any truly meaningful distinction between the two.
10000 lines for an os , plus gui, plus media player, plus... i suppose smalltalk was used as a glue over an api written with a lower level language ?
I don’t understand how one could build just a kernel with the required graphical drivers with that few number of lines... But maybe at that time the requirements for each part of the system were orders of magnitude simpler (memory protection, scheduling, fault isolation, graphical functionalities,...).
I don’t understand how one could build just a kernel with the required graphical drivers with that few number of lines... But maybe at that time the requirements for each part of the system were orders of magnitude simpler (memory protection, scheduling, fault isolation, graphical functionalities,...).
The only API was the machine instruction set, the lower level language was the microcode language used to define machine instructions. For example, the bitblt operation used to move graphics around on the screen was a machine instruction implemented in microcode. Ditto for network etc. At boot time, the Alto and later Xerox machines that ran Smalltalk loaded a different microcoded instruction set for
each language: Smalltalk, Lisp, Mesa, BCPL ... Then the whole system was written in that language. Details appear in the many papers and reports linked here:
http://xeroxalto.computerhistory.org/xerox_alto_file_system_...
I just noticed that site includes the code, apparently including the microcode, in addition to papers and reports! (I'm not sure if the microcode programs are included in Kay's 10 - 20,000 line estimate.)
http://xeroxalto.computerhistory.org/xerox_alto_file_system_...
I just noticed that site includes the code, apparently including the microcode, in addition to papers and reports! (I'm not sure if the microcode programs are included in Kay's 10 - 20,000 line estimate.)
The Alto only had 1024 words (4k bytes) of microcode, so it is included in the estimate. The NoteTaker used several 8086s (the first ones) and was entirely in Smalltalk except for 6k bytes of 8086 code.
i.e. "math wins"! ... when coupled into the design of the entire system.
And ... we would have loved to have had more microcode!
i.e. "math wins"! ... when coupled into the design of the entire system.
And ... we would have loved to have had more microcode!
Thanks for the info !
I'm still having a hard time understanding how having to ability to program your own "assembly" operation through microcodes could reduce the number of lines for huge programs that drastically. At worst you could still code those operations with the higher level language (smalltalk)... I can imagine how that would make things slower, but that's another issue.
Another thing i wonder as someone that hasn't coded in assembly since school : isn't the set of low level instructions required to build most regular programs pretty much settled now ? I'm not talking about high performance or 3D graphics, or AI. Only things personal computers have been doing for the last 30 years.
Put it differently: supposing intel gave use the possibility to reprogram a large set of its assembly instruction using microcodes, would the linux codebase be radically smaller ?
I'm still having a hard time understanding how having to ability to program your own "assembly" operation through microcodes could reduce the number of lines for huge programs that drastically. At worst you could still code those operations with the higher level language (smalltalk)... I can imagine how that would make things slower, but that's another issue.
Another thing i wonder as someone that hasn't coded in assembly since school : isn't the set of low level instructions required to build most regular programs pretty much settled now ? I'm not talking about high performance or 3D graphics, or AI. Only things personal computers have been doing for the last 30 years.
Put it differently: supposing intel gave use the possibility to reprogram a large set of its assembly instruction using microcodes, would the linux codebase be radically smaller ?
The main reason the Alto and other Xerox PARC machines provided so much functionality with so little code is the
designers were quite ruthless about simplification and providing just
one best way to do everything, which all programs then had to use. There weren't any huge programs - they didn't need
them. Those systems were very unlike Linux or any modern platform. For example Linux plus its typical userland provides perhaps seventeen dozen different ways to put text on a screen. The Alto solved that problem once and for all in microcode, and then all Alto software used that one solution and nothing else. That's where the savings came from: they decided in advance what they needed and then
designed the whole techology stack together to provide that in the most parsimoniious way: from the GUI through the
programming language and compiler and display hardware and
CPU hardware, machine instruction set down to microcode - it was all designed together from a bird's eye view of what the
entire finished system was supposed to look like. All done by a small group of people in a short time. It's a completely different process from the decades of accretion
in Linux and other modern platforms.
To respond to your technical questions about microcode. Microcode (and CISC architectures in general) are motivated by making the compiled binary programs as small as possible - each single machine instruction can do a lot because it runs a lot of microcode. The optimal set of machine instructions isn't settled once and for all, it depends on available technology. The Alto only had 128 kilobytes of RAM, so it was crucial to design an instruction set where compiled programs were as small as possible. So (I recall) their CPU used a stack rather than a lot of registers because then most operands could be implicit - that saves a couple of bytes in a typical instruction. Now that gigabyte memories are commonplace, the design trade-offs are quite different. But I think the microcoded instructions made the programs shorter at the source code level just because they made it unnecessary to re-implement the low level graphics operations (etc.) over and over in system and application code.
To respond to your technical questions about microcode. Microcode (and CISC architectures in general) are motivated by making the compiled binary programs as small as possible - each single machine instruction can do a lot because it runs a lot of microcode. The optimal set of machine instructions isn't settled once and for all, it depends on available technology. The Alto only had 128 kilobytes of RAM, so it was crucial to design an instruction set where compiled programs were as small as possible. So (I recall) their CPU used a stack rather than a lot of registers because then most operands could be implicit - that saves a couple of bytes in a typical instruction. Now that gigabyte memories are commonplace, the design trade-offs are quite different. But I think the microcoded instructions made the programs shorter at the source code level just because they made it unnecessary to re-implement the low level graphics operations (etc.) over and over in system and application code.
> the linux codebase be radically smaller ?
I believe Linux isn't built the same way the Smalltalk system was. One glaring difference is the use of C - with that you can't get very high level and are always programming at a very low level. You also force all integrations to be at a very low leve, given how the ABI and syscalls work.
You can do quite a bit more if you use a higher level language with very good meta programming, for instance Lisp or Smalltalk. E.g., you could imagine very high level abstractions that get translated to lower level logic automatically. Unfortunately, this is not an area that has been extensively adopted IMO, and the mainstream thinking is we need to use lower level languages for operating systems and such.
> supposing intel gave use the possibility to reprogram a large set of its assembly instruction using microcodes
Actually Alan Kay has been critical of Intel not doing this. It may not affect Linux much but it will definitely open the door for systems using more advanced techniques, specifically the ability to build high level emulators. See https://news.ycombinator.com/item?id=15282739 for more.
I believe Linux isn't built the same way the Smalltalk system was. One glaring difference is the use of C - with that you can't get very high level and are always programming at a very low level. You also force all integrations to be at a very low leve, given how the ABI and syscalls work.
You can do quite a bit more if you use a higher level language with very good meta programming, for instance Lisp or Smalltalk. E.g., you could imagine very high level abstractions that get translated to lower level logic automatically. Unfortunately, this is not an area that has been extensively adopted IMO, and the mainstream thinking is we need to use lower level languages for operating systems and such.
> supposing intel gave use the possibility to reprogram a large set of its assembly instruction using microcodes
Actually Alan Kay has been critical of Intel not doing this. It may not affect Linux much but it will definitely open the door for systems using more advanced techniques, specifically the ability to build high level emulators. See https://news.ycombinator.com/item?id=15282739 for more.
It sort-of depends on what you mean by "low level". You can play very sophisticated games with C and present a high-level interface to functionality.
For some ideas on how you can get fewer lines, see Nile (https://github.com/damelang/nile). Nile is a DSL that implemented many Cairo (https://cairographics.org/) primitives using a much more compact form, in a lot fewer lines. It was part of the STEPS project (http://www.vpri.org/pdf/tr2012001_steps.pdf)
The STEPS project was able to get an experimental system up and running with 20k lines. He's not talking out of his ass. When he has the dough he produces
He didn't mention some of the stuff Viewpoints was working on which seems to have petered out (although maybe "bad funders" was a dig at that). I'd like to know what happened to those projects.
https://en.wikipedia.org/wiki/COLA_(software_architecture)
https://en.wikipedia.org/wiki/COLA_(software_architecture)
The STEPS was more or less finished if I recall correctly. The last report took forever to get public, but they did get a proof of concept out.
Many people asked for something usable, but that wasn't really the goal, so they didn't do the polishing —it sure takes work, but it doesn't prove much.
We can still take the ideas, though. We could integrate some of them in the current ecosystem to write simpler programs, or even simpler ecosystems. How about something like squeak or pharo, only 10 times simpler?
Many people asked for something usable, but that wasn't really the goal, so they didn't do the polishing —it sure takes work, but it doesn't prove much.
We can still take the ideas, though. We could integrate some of them in the current ecosystem to write simpler programs, or even simpler ecosystems. How about something like squeak or pharo, only 10 times simpler?
I believe the pendulum is swinging back from dynamic languages to compiled languages with stronger static guarantees. Languages that are parsimonious with memory and give you control over the layout when you need it.
Don't get me wrong. Smalltalk is a hugely influential language and we're better for having it, but it was designed in an era where hardware had very different performance characteristics from today's hardware.