A brief interview with Tcl creator John Ousterhout (2023)(pldb.io)
pldb.io
A brief interview with Tcl creator John Ousterhout (2023)
https://pldb.io/blog/JohnOusterhout.html
70 comments
+1 for 'A Philosophy of Software Design.' I read that book a while ago and really liked his views on a lot of things. Off the top of my head some of them are: complexity being the greatest enemy of software engineers, the importance of strategic programming (as opposed to tacit 'somehow-get-the-job-done' programming), general-purpose abstractions, and having deep modules exposed through simple interfaces.
+1 One of the only books on design which actually is backed up by data.
I had been misled by the Ruby ecosystem earlier in my career to do clean code + refactoring + tdd and this book + actually looking at complex systems such as games made me realize that coding is not about abstractions but it is about data and what the computer needs to do.
I had been misled by the Ruby ecosystem earlier in my career to do clean code + refactoring + tdd and this book + actually looking at complex systems such as games made me realize that coding is not about abstractions but it is about data and what the computer needs to do.
In your view, what word(s) would you use to describe the following?
"a carefully designed API for interacting with piece of data which provides both a consistent technical interface and conceptual understanding of the meaning of said thing"
The reason I ask is because the word I've always used for that is "abstraction", but I'm getting the sense that over-abstracted and over-architected frameworks has soured many people to the word. I want to communicate the idea without conjuring up images of java classes with 90 character names. What would you recommend?
"a carefully designed API for interacting with piece of data which provides both a consistent technical interface and conceptual understanding of the meaning of said thing"
The reason I ask is because the word I've always used for that is "abstraction", but I'm getting the sense that over-abstracted and over-architected frameworks has soured many people to the word. I want to communicate the idea without conjuring up images of java classes with 90 character names. What would you recommend?
You are absolutely right. In that sense the right abstraction is what one should go after.
But what I mean is the more popular Java / OOP style abstractions like a repository which returns a builder which returns class which you new up and pass to some sort of super interface and its just such a mess.
As a technology I really like the JVM and hotspot but good Lord if I have to wade through 10 different things to get there.
Finally to answer your question I have started using the actual name of the thing in the language I'm using. Such as a "struct" in c and rust which I mostly use these days.
But what I mean is the more popular Java / OOP style abstractions like a repository which returns a builder which returns class which you new up and pass to some sort of super interface and its just such a mess.
As a technology I really like the JVM and hotspot but good Lord if I have to wade through 10 different things to get there.
Finally to answer your question I have started using the actual name of the thing in the language I'm using. Such as a "struct" in c and rust which I mostly use these days.
Also, a co-author of the Raft consensus algorithm.
He is an absolutely amazing and extremely humble person. He agreed to review my article (written under the influence of his book) in his spare time. He does tons of code reviews every week for his students as well.
I was especially impressed by his notion of "deep modules" in "A Philosophy of Software Design". The idea is extremely not-popular, but really sound and practical. I would definitely go for this book instead of "Clean Code" and such.
His book is so fundamental, and it goes far beyond the overused and vague "complexity" term.
I was especially impressed by his notion of "deep modules" in "A Philosophy of Software Design". The idea is extremely not-popular, but really sound and practical. I would definitely go for this book instead of "Clean Code" and such.
His book is so fundamental, and it goes far beyond the overused and vague "complexity" term.
Any tldr for “deep modules”?
Deep module - simple interface, complex functionality
Shallow module - interface is relatively complex to the small functionality it provides. Shallow modules don't hide as much complexity.
https://github.com/zakirullin/cognitive-load?#too-many-small...
But I highly recommend picking up the book :) It is an enjoyable read.
Shallow module - interface is relatively complex to the small functionality it provides. Shallow modules don't hide as much complexity.
https://github.com/zakirullin/cognitive-load?#too-many-small...
But I highly recommend picking up the book :) It is an enjoyable read.
IIRC, the deep/shallow thing is anout how much functionality there is, and he used wide/narrow for the size of the API?
So that the goal is to have deep, narrow modules: modules that have a lot of functionality hidden behibd a small API.
So that the goal is to have deep, narrow modules: modules that have a lot of functionality hidden behibd a small API.
I think a good example of this concept is the foreach command in Tcl: It's quite flexible compared to list iteration constructs found in most languages.
This is normal usage:
Many commands in Tcl give the impression that they can be used in many different ways, so writing a program in Tcl feels like building a structure by creatively assembling a limited set of legos.
This is normal usage:
foreach x {1 2} {puts $x} => 1 2
foreach x {1 2} y {3 4} {puts $x $y} => 1 3 2 4
foreach {x1 x2} {1 2 3 4} y {5 6} {puts $x1 $x2 $y} => 1 2 5 3 4 6
Many commands in Tcl give the impression that they can be used in many different ways, so writing a program in Tcl feels like building a structure by creatively assembling a limited set of legos.
Aha, I’d known some of the superpowers of TCL’s foreach but not all of those.
A ‘deep command’.
A ‘deep command’.
[deleted]
This google talk https://youtu.be/bmSAYlu0NcY?si=yHREamFoRG_fZopl is a condensed version of the (already small - 150 pags IIRC) book. I highly recommend both.
To the point made by the GP (and others here) about how useful the book is, I must add that I got a whole lot more from this small book than some other well known and oft suggested titles (like Clean Code, POSA etc).
My most recent peeve about the matter is that Kent Beck"s "Tidy, Fisrt?" was apparently motivated partly in response to this book and yet it falls way too short to matching up. Half of thst book is way too simplistic and almost patronizing and the other is handwavy abstract notions too scared to touch reality lest it spoil the narrative.
To the point made by the GP (and others here) about how useful the book is, I must add that I got a whole lot more from this small book than some other well known and oft suggested titles (like Clean Code, POSA etc).
My most recent peeve about the matter is that Kent Beck"s "Tidy, Fisrt?" was apparently motivated partly in response to this book and yet it falls way too short to matching up. Half of thst book is way too simplistic and almost patronizing and the other is handwavy abstract notions too scared to touch reality lest it spoil the narrative.
I tried learning Motif to write X11 graphical programs in 1994. Another student in the computer lab told me about Tcl / Tk. It was far easier. Then I got a job in the semiconductor design industry in 1997. The next year the leading EDA companies Cadence and Synopsys started using Tcl as the internal scripting language for the tools. So now it is 2024 and our CAD flows are tens of thousands of lines of Tcl code. Every week I still write Tcl code to generate reports or automate something.
What have been the pros and cons of Tcl for you, compared with other scripting languages if you have experience with them?
the big thing that's changed is that now we have html5, and enough ram to run it
Here's a transcript of a keynote I gave to the Toronto Usenix symposium, where I talked about the pros and cons of TCL/Tk, from developing multi-player SimCity with it around 1993.
A lot has changed since then, but in 1993 it was so much better than Motif and every other alternative in so many ways.
https://www.donhopkins.com/home/catalog/simcity/keynote.html
>Hello, my name is Don Hopkins, and I ported SimCity to Unix on top of X11 using the TCL/Tk toolkit, and also on top of the NeWS window system using HyperLook. Unfortunately I can't be in Toronto to demonstrate this stuff to you, but fortunately I have some video tapes that you can see that demonstrate the whole system.
>The X11 version of SimCity is multi player, and several people can join the game, and cooperate, and people draw roads, and put down buildings in different places, and then when you do something important, like change the tax rate, or build an airport, you have to get everybody else who's playing to agree where to put it, and which thing to put down. So it makes you slow down and rationalize what you're doing, and explain to people why it is you want to put it there. It's a really interesting addition to SimCity because it adds politics.
>I'm going to show first the X11 version, which is on top of TCL/Tk, and that's a free toolkit that's available, written by John Ousterhaut at Berkeley. There's a book that's coming out describing it, and you can just get TCL/Tk and use it to make products for free. You don't have to pay anything for it, and it's actually very good code. It really beats the pants off of Motif, and we were able to modify it in order to support things like colormaps and multiple displays the way we needed to, and add things to it like a sound server to make sound effects, and pie menus for quickly selecting city editing tools. TCL/Tk made alot of sense for SimCity.
[...]
>HyperLook has this scripting language in it, and TCL/Tk has the TCL scripting language in it, so I was able to translate the things I had written in PostScript into TCL, and have a very similar structure. It makes it much easier to develop a thing as complex as SimCity if you have an interpretive programming language there, so you can just fiddle around, and paste new function definitions into the running system.
>There are some user interface editors that have been developed for TCL/Tk, but I just wrote code by hand because I was doing some more specialized things, like multiple views and putting up views on different X11 displays over the net, that the user interface editors didn't support. HyperLook has a built in user interface editor, and I'll demonstrate HyperLook SimCity and the HyperLook environment in the following video tape.
[...]
>One of the things I developed for TCL/Tk and for HyperLook was this sound mixer server. And then we got an NCD X-Terminal, and just layered my sound mixer on top of their sound server. It can either use the built-in audio device and be a TCL sound server, or it can use the sound server that's in your X-Terminal or your Sun or SGI or whatever, if you're running NetAudio.
[...]
>For the pie menus and the multi player interaction stuff, like putting up windows on different screens and letting people have a shared view of this one environment, it was really nice to have the source code to the TCL/Tk toolkit, in order to support that kind of stuff.
>It wouldn't have been possible to port SimCity to X11 using Open Software Foundation's Motif toolkit. It just absolutely sucks. It's not open, and you have to pay for the source code, and it's not being maintained.
>But there's a really wonderful community that's grown around TCL/Tk, and people are allowed to use it in their products for free, and get the source code. I implemented pie menus with TCL/Tk for SimCity, and out of gratitude, I put the source code for the pie menu module out for other people to use, just to help to community. It's a nice positive feedback situation, as opposed to the cold war situation you have with COSE, Motif, and Open Software Foundation.
>We bundled the HyperLook runtime system with the NeWS version of SimCity. If you're running OpenWindows 3.0, you can get it and try it out.
[...]
A lot has changed since then, but in 1993 it was so much better than Motif and every other alternative in so many ways.
https://www.donhopkins.com/home/catalog/simcity/keynote.html
>Hello, my name is Don Hopkins, and I ported SimCity to Unix on top of X11 using the TCL/Tk toolkit, and also on top of the NeWS window system using HyperLook. Unfortunately I can't be in Toronto to demonstrate this stuff to you, but fortunately I have some video tapes that you can see that demonstrate the whole system.
>The X11 version of SimCity is multi player, and several people can join the game, and cooperate, and people draw roads, and put down buildings in different places, and then when you do something important, like change the tax rate, or build an airport, you have to get everybody else who's playing to agree where to put it, and which thing to put down. So it makes you slow down and rationalize what you're doing, and explain to people why it is you want to put it there. It's a really interesting addition to SimCity because it adds politics.
>I'm going to show first the X11 version, which is on top of TCL/Tk, and that's a free toolkit that's available, written by John Ousterhaut at Berkeley. There's a book that's coming out describing it, and you can just get TCL/Tk and use it to make products for free. You don't have to pay anything for it, and it's actually very good code. It really beats the pants off of Motif, and we were able to modify it in order to support things like colormaps and multiple displays the way we needed to, and add things to it like a sound server to make sound effects, and pie menus for quickly selecting city editing tools. TCL/Tk made alot of sense for SimCity.
[...]
>HyperLook has this scripting language in it, and TCL/Tk has the TCL scripting language in it, so I was able to translate the things I had written in PostScript into TCL, and have a very similar structure. It makes it much easier to develop a thing as complex as SimCity if you have an interpretive programming language there, so you can just fiddle around, and paste new function definitions into the running system.
>There are some user interface editors that have been developed for TCL/Tk, but I just wrote code by hand because I was doing some more specialized things, like multiple views and putting up views on different X11 displays over the net, that the user interface editors didn't support. HyperLook has a built in user interface editor, and I'll demonstrate HyperLook SimCity and the HyperLook environment in the following video tape.
[...]
>One of the things I developed for TCL/Tk and for HyperLook was this sound mixer server. And then we got an NCD X-Terminal, and just layered my sound mixer on top of their sound server. It can either use the built-in audio device and be a TCL sound server, or it can use the sound server that's in your X-Terminal or your Sun or SGI or whatever, if you're running NetAudio.
[...]
>For the pie menus and the multi player interaction stuff, like putting up windows on different screens and letting people have a shared view of this one environment, it was really nice to have the source code to the TCL/Tk toolkit, in order to support that kind of stuff.
>It wouldn't have been possible to port SimCity to X11 using Open Software Foundation's Motif toolkit. It just absolutely sucks. It's not open, and you have to pay for the source code, and it's not being maintained.
>But there's a really wonderful community that's grown around TCL/Tk, and people are allowed to use it in their products for free, and get the source code. I implemented pie menus with TCL/Tk for SimCity, and out of gratitude, I put the source code for the pie menu module out for other people to use, just to help to community. It's a nice positive feedback situation, as opposed to the cold war situation you have with COSE, Motif, and Open Software Foundation.
>We bundled the HyperLook runtime system with the NeWS version of SimCity. If you're running OpenWindows 3.0, you can get it and try it out.
[...]
If you want a more substantial interview with John Ousterhout, I highly recommend his hour-long talk at google A Philosophy of Software Design, which is essentially a synopsis of his book by the same title.
Edit: I guess not an interview, but he does answer some questions from the audience at the end.
[1] https://www.youtube.com/watch?v=bmSAYlu0NcY
Edit: I guess not an interview, but he does answer some questions from the audience at the end.
[1] https://www.youtube.com/watch?v=bmSAYlu0NcY
I remember early in my career working at an investment bank on the trading floor as a support analyst.
I was asked by a very frustrated trader to modify the behaviour of one of the trading apps.. basically he shouted..."just figure it out". Yea he was bit of a twat but then trading desks have changed a lot today.
The app was from a 3rd party provider and making changes would take time of course, then they told me I could do this myself with Tcl. So in a day I figured out Tcl, made the change and it was in production the next day. Of course doing that now would be impossible but I remember the buzz from being able to make a real world change so damn fast.
I was asked by a very frustrated trader to modify the behaviour of one of the trading apps.. basically he shouted..."just figure it out". Yea he was bit of a twat but then trading desks have changed a lot today.
The app was from a 3rd party provider and making changes would take time of course, then they told me I could do this myself with Tcl. So in a day I figured out Tcl, made the change and it was in production the next day. Of course doing that now would be impossible but I remember the buzz from being able to make a real world change so damn fast.
[deleted]
Tcl has been a staple on Mac for over 30 years at this point, since System 7 at least. It was installed in Mac OS X as standard, not sure if it still is these days.
I use it to wrangle text directory listings from old Macintosh CD-ROMs from pre-Unicode era. The character set support and conversion code seems to have been written by Apple themselves in the mid-1990s. https://blog.gingerbeardman.com/2022/03/31/working-with-clas...
I was pleasantly surprised by the syntax and capability of Tcl as well as the fact that it's still being developed and supported and that the maintainers took my feature requests and bug reports seriously and actioned them in what I consider a short time.
I use it to wrangle text directory listings from old Macintosh CD-ROMs from pre-Unicode era. The character set support and conversion code seems to have been written by Apple themselves in the mid-1990s. https://blog.gingerbeardman.com/2022/03/31/working-with-clas...
I was pleasantly surprised by the syntax and capability of Tcl as well as the fact that it's still being developed and supported and that the maintainers took my feature requests and bug reports seriously and actioned them in what I consider a short time.
> It was installed in Mac OS X as standard, not sure if it still is these days.
It is! Just type "tclsh"
It is! Just type "tclsh"
However, the Tk install that Macos ships with is so broken there's even a warning about it on the Tcl website (or was it the Python website, in case you try to use Tkinter on system Python). Invisible text, invisible buttons, Apple's got you covered....
Interesting, I am using a self-installed version for ...reasons.
Would love to read more.
Also:
> This site is powered by Jekyll (but, I absolutely definitely do not recommend it)
Can I recommend Scroll to you (https://scroll.pub/). It's not good yet, but should be great in 3 - 6 months.
P.S. Loving your blog
Also:
> This site is powered by Jekyll (but, I absolutely definitely do not recommend it)
Can I recommend Scroll to you (https://scroll.pub/). It's not good yet, but should be great in 3 - 6 months.
P.S. Loving your blog
Thanks! I've tamed Jekyll enough now that I don't mind it (but still wouldn't recommend it). Rebuilding my theme would be a boring job so I'm not keen to migrate.
Tcl was the scripting language in the mIRC IRC client, and as a 13 year old, my first foray into programming.
I started writing little scripts to boot annoying people off IRC, but ended up caring more about coding than IRC politics - and ended up writing a full blown Napster clone in Tcl.
I feel like I owe my career to Tcl opening these doors for me - so thank you John.
I started writing little scripts to boot annoying people off IRC, but ended up caring more about coding than IRC politics - and ended up writing a full blown Napster clone in Tcl.
I feel like I owe my career to Tcl opening these doors for me - so thank you John.
Wasn’t he also one of the creators of RAFT (https://en.m.wikipedia.org/wiki/Raft_(algorithm) )
Also responsible for Magic [1],[2] which is the basis (AFAIK) for the efabless route to getting your own chip designed these days...
I ported it to the Mac and played with it a while back with the intention of making my own chip, but priorities changed when my wife fell very ill.
[1] https://en.wikipedia.org/wiki/Magic_(software)
[2] http://opencircuitdesign.com/magic/
I ported it to the Mac and played with it a while back with the intention of making my own chip, but priorities changed when my wife fell very ill.
[1] https://en.wikipedia.org/wiki/Magic_(software)
[2] http://opencircuitdesign.com/magic/
TFA mentions the Sprite operating system, which brought us a few things, including (i recently discovered), pmake[0] (by way of Adam de Boor[1]) which was an early instance of parallelizing make(1), and begat BSD Make[2], which is lovely to use.
[0] https://web.stanford.edu/~ouster/cgi-bin/spriteRetrospective...
[1] https://www.linkedin.com/in/adam-de-boor-4293a1/
[2] https://man.netbsd.org/make.1
[0] https://web.stanford.edu/~ouster/cgi-bin/spriteRetrospective...
[1] https://www.linkedin.com/in/adam-de-boor-4293a1/
[2] https://man.netbsd.org/make.1
I just recently started learning tcl/tk out of frustration with the other options for gui toolkits. Liking it quite a bit now that I have gotten over the hurdle of figuring out the sense of tcl which often felt like nonsense at the start. TK is especially nice, very easy to work with.
tk is still excellent, tcl is mediocre
JO is giant in the industry; a longtime working professional with fantastic pedagogical chops. In my opinion, if he's talking, I'm listening.
Tcl "powers" the internet in a non-direct way since it is used in Cisco IOS.
And every proprietary EDA tool in the world too.
...and a10/F5 aFleX/iRules[0][1], and the scripting in Tealeaf[2][3], ...
[0] https://www.a10networks.com/blog/aflex-tutorial/
[1] https://community.f5.com/kb/technicalarticles/irules-concept...
[2] https://www.acoustic.com/tealeaf
[3] https://help.goacoustic.com/hc/en-us/articles/360043785173-E...
[0] https://www.a10networks.com/blog/aflex-tutorial/
[1] https://community.f5.com/kb/technicalarticles/irules-concept...
[2] https://www.acoustic.com/tealeaf
[3] https://help.goacoustic.com/hc/en-us/articles/360043785173-E...
...unfortunately.
I guess to be fair if it hadn't been TCL they would have chosen something even worse.
I guess to be fair if it hadn't been TCL they would have chosen something even worse.
I guess Lua wasn't available at the time.
Chez Scheme too then!
Perfect timing. I ordered a copy of "A Philosophy of Software Design" a week ago and it arrived yesterday. I'm going to start reading it tonight, but this is a nice preface, I bet.
Several months ago a friend encouraged me to look at Tcl again (independent of Tk.) Mixins, regexes, reflection. All stuff I didn't appreciate in the early 90s. And I had reason to look at Minix a few years back. Ousterhout definitely did some good work. I think he might have been just a little bit ahead of the rest of us.
Several months ago a friend encouraged me to look at Tcl again (independent of Tk.) Mixins, regexes, reflection. All stuff I didn't appreciate in the early 90s. And I had reason to look at Minix a few years back. Ousterhout definitely did some good work. I think he might have been just a little bit ahead of the rest of us.
it didn't have mixins in the early 90s
tcl's model of what data is doesn't accommodate general graphs, just trees. you can make a hash table but not a hash table of hash tables. on the plus side you won't have aliasing bugs like python's notorious
tcl's model of what data is doesn't accommodate general graphs, just trees. you can make a hash table but not a hash table of hash tables. on the plus side you won't have aliasing bugs like python's notorious
matrix = [[0] * width] * height
It is a phenomenal book for what it is. It was the required reading for the class I taught at UNC in 2019. I am teaching a workshop in September and bought a copy for every attendee because I believe it is that important for people to be exposed to his ideas and how he approaches things.
I was also impressed with "Righting Software" and the premise of volatility based decomposition but it is not nearly as approachable as Ousterhout's book.
I was also impressed with "Righting Software" and the premise of volatility based decomposition but it is not nearly as approachable as Ousterhout's book.
a lot of people here are wondering if tcl is something they should check out. tcl has some serious flaws, but some real merits as well. the best and most balanced answer to this question is https://yosefk.com/blog/i-cant-believe-im-praising-tcl.html, which explains some of the reasons i used to really enjoy tcl, before i gave it up for more capable languages
maybe the nucleus of kreinin's argument is
> So basically, pop infix languages (and I use the term in the most non-judgmental, factual way), pop infix languages are optimized for programming (duh, they are programming languages). Programming is definitions. Define a variable and it will be easy to use it, and computing hairy expressions from variables is also easy. Tcl is optimized for usage. Most of the time, users give simple commands. Command names and literal parameters are easy. If you are a sophisticated user, and you want to do pmem 0 bkpt [expr [pmem 0 pc] + 1], go ahead and do it. A bit ugly, but on the other hand, simple commands are really, really simple.
> And eventually, simple commands become all that matters for the user, because the sophisticated user grows personal shortcuts, which abstract away variables and expressions, so you end up with pmem 0 bkpt nextpc or something. Apparently, flat function calls with literal arguments is what interactive program usage is all about.
the stallman/ousterhout debate on usenet back in the 90s (from which guile sprang) may be interesting reading, but i think kreinin's post linked above has more deep insights
maybe the nucleus of kreinin's argument is
> So basically, pop infix languages (and I use the term in the most non-judgmental, factual way), pop infix languages are optimized for programming (duh, they are programming languages). Programming is definitions. Define a variable and it will be easy to use it, and computing hairy expressions from variables is also easy. Tcl is optimized for usage. Most of the time, users give simple commands. Command names and literal parameters are easy. If you are a sophisticated user, and you want to do pmem 0 bkpt [expr [pmem 0 pc] + 1], go ahead and do it. A bit ugly, but on the other hand, simple commands are really, really simple.
> And eventually, simple commands become all that matters for the user, because the sophisticated user grows personal shortcuts, which abstract away variables and expressions, so you end up with pmem 0 bkpt nextpc or something. Apparently, flat function calls with literal arguments is what interactive program usage is all about.
the stallman/ousterhout debate on usenet back in the 90s (from which guile sprang) may be interesting reading, but i think kreinin's post linked above has more deep insights
I never liked tcl but the implementation was definitely small, which mattered more back then than it does now. Does anyone still care about it?
These days what we really need is an easily embeddable Python. Guile is very nice if you want a somewhat fancy Scheme system, but it got out of control if all you wanted was something light and embeddable.
These days what we really need is an easily embeddable Python. Guile is very nice if you want a somewhat fancy Scheme system, but it got out of control if all you wanted was something light and embeddable.
Lots of people care about it, and it's alive and kicking in-industry and being developed (Tcl 9 is imminent[0]).
I've had fun w Guile over the years (though actually never considered its "heaviness"); fun.
I'm constantly amazed at the professional love for Python; its got it's network effect in full-effect: lots of people, lots of software - there's just no denying that. But for Joy of Development, Tcl (and C) is still my go-to.
You probably know this, but speaking of embeddable Python and Tcl, Tcl is indeed embedded in Python if you're running Tkinter[1].
And Larry Wall thought routers were weird places to be ported to[2].
[0] https://sourceforge.net/projects/tcl/postdownload
[1] https://docs.python.org/3/library/tkinter.html
[2] https://libquotes.com/larry-wall/quote/lbh5n5p
I've had fun w Guile over the years (though actually never considered its "heaviness"); fun.
I'm constantly amazed at the professional love for Python; its got it's network effect in full-effect: lots of people, lots of software - there's just no denying that. But for Joy of Development, Tcl (and C) is still my go-to.
You probably know this, but speaking of embeddable Python and Tcl, Tcl is indeed embedded in Python if you're running Tkinter[1].
And Larry Wall thought routers were weird places to be ported to[2].
[0] https://sourceforge.net/projects/tcl/postdownload
[1] https://docs.python.org/3/library/tkinter.html
[2] https://libquotes.com/larry-wall/quote/lbh5n5p
I believe lua is often used in this role these days. I've gotten the impression that it is light and easy to integrate, but I haven't tried it myself.
Yes, Lua is quite easy to embed and is popular in game development for that reason. It appears in a few other applications like Redis as well. But, in the big picture, it's a weird language that is not very popular. People want Python and/or Javascript, and JS is awful, so that leaves Python ;).
I do bot understand the downvotes. Maybe because of wording “weird”. Give me a break, I think it has a point: much more people can do Python as Lua.
I do like Lua, but if I just try to use it in my work, I would be paria.
I do like Lua, but if I just try to use it in my work, I would be paria.
I had figured people didn't like the dig at JS. Unfortunately the WAT video is no longer on youtube. It might be on destroyallsoftware.com.
I guess Lua remains the chief goto for light embedded scripting environment, doesn't it?
If only it had given us dict/apply/lmap earlier, embraced its prefix-ness more (no ugly expr nor https://core.tcl-lang.org/tips/doc/trunk/tip/676.md when we already can do Lisp-style arithmetic via `namespace path {::tcl::mathop ::tcl::mathfunc}`) and had better tools for metaprogramming (string based templating is suboptimal, let us say)...
It still was my gateway into Common Lisp and remains a very cool scripting language with builtin event loop, good Unicode handling and very practical/"batteries included" stdlib.
It still was my gateway into Common Lisp and remains a very cool scripting language with builtin event loop, good Unicode handling and very practical/"batteries included" stdlib.
To this day, TCL is still my go-to for internal scripting languages or, quite often, creating a simple configuration DSL (or an internal linter for configuration files).
Of course, I also do most of my scripting in TCL, so I'm a wee bit biased.
Of course, I also do most of my scripting in TCL, so I'm a wee bit biased.
>> what we really need is an easily embeddable Python. Guile is very nice if you want a somewhat fancy Scheme system, but it got out of control if all you wanted was something light and embeddable.
In my experience what goes out of control are python scripts, with many dependencies.
Also the changes in the language tend to be much more intrusive and often than in scheme.
In my experience what goes out of control are python scripts, with many dependencies.
Also the changes in the language tend to be much more intrusive and often than in scheme.
A huge bummer about Guile (and that is frankly shocking for a language designed to be embeddable) is that it relies on a bunch of global state, which means that you can't embed multiple interpreters a single process. This totally kills it as a embedded scripting language for many applications, and it's also just bad practice in general. Even Python, far less easily embeddable, supports multiple interpreters, though before Python 3.13 (I think) they all shared the GIL.
Aside from this issue, the Guile API is pretty good, but the best one by far is Lua's C API, which is just a thing of beauty. I'm curious to try it with Tcl.
Aside from this issue, the Guile API is pretty good, but the best one by far is Lua's C API, which is just a thing of beauty. I'm curious to try it with Tcl.
Tcl makes it very easy to use multiple interpreters in the same process. It is one of its strengths.
What I like about JO is that he acknowledges the deep flaws of Tcl. And there never developed an inward looking bubble of Tcl/Tk advocates - the users were largely driven by pragmatism; it just got the job done.
This contrasted with Larry Wall and ethos/self-delusion of the Perl community - anyone critical of the language or its design was disregarded as simply lacking the intellect to appreciate the "poetry" of Perl - and the the language warts (better described as big pus-weeping boils) were actually features.
This contrasted with Larry Wall and ethos/self-delusion of the Perl community - anyone critical of the language or its design was disregarded as simply lacking the intellect to appreciate the "poetry" of Perl - and the the language warts (better described as big pus-weeping boils) were actually features.
the implementation of tcl is not small; it's 1.9 megabytes stripped, 8.5 times the size of lua 5.2, which is a very popular embeddable scripting language:
really small would be a forth. forth is basically a scripting layer for assembly language with an interactive shell and ide, and there are plenty of full-featured forths that will run in 16k of ram. unfortunately forth code is hard to read and easy to crash
garbage-collected pointer-graph languages like lisp, python, js, and perl5 tend to use a lot more runtime memory for a given amount of functionality; in tcl (or the bourne shell) a list of 10 numbers like {8 2020 23806 102 7 3 11 5 36 86} is stored as a string which might occupy 31 bytes, plus an allocation header of probably another 16 bytes on a 64-bit system, for a total of maybe 48 or 64 bytes. by contrast, in a lisp system each list item occupies a 16-byte cons cell, so you end up paying 160 bytes; and storage is not reclaimed promptly, so you typically have another multiplier of 1½–3 to keep from spending all your time in the garbage collector. lua is the same; perl and python are different in detail (usually reclaiming storage more promptly due to reference counting, but wasting space on reference-count fields, boxing integers in python's case, and bloated svs in perl's case) but broadly similar in outline
tracing garbage collection (as opposed to the much slower reference counting used by perl and cpython) tends to diminish easy embeddability, both because it's very tricky to do in standard c, and because it kind of wants to own your program's entire memory space
this stuff still matters a lot due to things like icache misses, embedded systems, and standalone executable size. people complain about the size of golang and rust binaries but they don't hold a candle to the cpython interpreter. but it mattered a lot more in the 90s!
: ~; ls -l /lib/x86_64-linux-gnu/libtcl8.6.so /lib/x86_64-linux-gnu/liblua5.2.so.0.0.0
-rw-r--r-- 1 root root 216960 Dec 10 2022 /lib/x86_64-linux-gnu/liblua5.2.so.0.0.0
-rw-r--r-- 1 root root 1852968 Feb 1 2023 /lib/x86_64-linux-gnu/libtcl8.6.so
: bin; ls -l /lib/x86_64-linux-gnu/libguile-3.0.so.1.5.0
-rw-r--r-- 1 root root 1303112 Mar 5 2022 /lib/x86_64-linux-gnu/libguile-3.0.so.1.5.0
: bin; ls -l sh awk
-rwxr-xr-x 1 user user 46126 May 11 1979 awk
-rwxr-xr-x 1 user user 17310 May 5 1979 sh
: bin; ls -l /lib/x86_64-linux-gnu/libpython3.11.so.1.0 /lib/x86_64-linux-gnu/libperl.so.5.36.0
-rw-r--r-- 1 root root 3823936 Nov 25 2023 /lib/x86_64-linux-gnu/libperl.so.5.36.0
-rw-r--r-- 1 root root 7732544 May 2 08:59 /lib/x86_64-linux-gnu/libpython3.11.so.1.0
: Downloads; ls -l /usr/lib/x86_64-linux-gnu/libnode.so.108
-rw-r--r-- 1 root root 47166144 Jun 22 09:21 /usr/lib/x86_64-linux-gnu/libnode.so.108
: ~; du -sk /usr/lib/jvm/java-17-openjdk-amd64/lib/
187208 /usr/lib/jvm/java-17-openjdk-amd64/lib/
: Downloads; ls -l /usr/bin/tinyscheme
-rwxr-xr-x 1 root root 102648 Jun 8 2020 /usr/bin/tinyscheme
: Downloads; ls -l /usr/lib/tinyscheme/init.scm
-rw-r--r-- 1 root root 23806 Jun 8 2020 /usr/lib/tinyscheme/init.scm
really small would be a forth. forth is basically a scripting layer for assembly language with an interactive shell and ide, and there are plenty of full-featured forths that will run in 16k of ram. unfortunately forth code is hard to read and easy to crash
garbage-collected pointer-graph languages like lisp, python, js, and perl5 tend to use a lot more runtime memory for a given amount of functionality; in tcl (or the bourne shell) a list of 10 numbers like {8 2020 23806 102 7 3 11 5 36 86} is stored as a string which might occupy 31 bytes, plus an allocation header of probably another 16 bytes on a 64-bit system, for a total of maybe 48 or 64 bytes. by contrast, in a lisp system each list item occupies a 16-byte cons cell, so you end up paying 160 bytes; and storage is not reclaimed promptly, so you typically have another multiplier of 1½–3 to keep from spending all your time in the garbage collector. lua is the same; perl and python are different in detail (usually reclaiming storage more promptly due to reference counting, but wasting space on reference-count fields, boxing integers in python's case, and bloated svs in perl's case) but broadly similar in outline
tracing garbage collection (as opposed to the much slower reference counting used by perl and cpython) tends to diminish easy embeddability, both because it's very tricky to do in standard c, and because it kind of wants to own your program's entire memory space
this stuff still matters a lot due to things like icache misses, embedded systems, and standalone executable size. people complain about the size of golang and rust binaries but they don't hold a candle to the cpython interpreter. but it mattered a lot more in the 90s!
> tracing garbage collection (as opposed to the much slower reference counting used by perl and cpython) tends to diminish easy embeddability, both because it's very tricky to do in standard c, and because it kind of wants to own your program's entire memory space
Lua is living proof of the opposite: small, easily embeddable, with a tracing GC
Lua is living proof of the opposite: small, easily embeddable, with a tracing GC
yes, i agree. this is one of many ways lua is exceptional
i cloned https://github.com/rsdoiel/xlisp and it turns out that, although the original ast-walking xlisp was for the z80 under cp/m, this version is an r3rs scheme. the stripped executable is 303.6 kilobytes. historical versions of xlisp were easy to embed, and api.doc indicates that this is a priority for this version as well
amusingly, this xlisp package includes a tk interface—by embedding tcl
an earlier xlisp, i think for cp/m-86, is http://www.cpm.z80.de/download/xlisp.zip. it is written in c, about 2800 lines of c, which i haven't tried to compile, but i think the executable is under 64k. this might give the flavor of the language thus implemented:
still, it seems like pretty strong evidence that you can do an ergonomic embedded scripting language in 8 kilobytes or so, not 1900 kilobytes
amusingly, this xlisp package includes a tk interface—by embedding tcl
an earlier xlisp, i think for cp/m-86, is http://www.cpm.z80.de/download/xlisp.zip. it is written in c, about 2800 lines of c, which i haven't tried to compile, but i think the executable is under 64k. this might give the flavor of the language thus implemented:
; ::::::::::::
; :: Turtle ::
; ::::::::::::
; Define "Turtle" class
(setq Turtle (Class 'new))
; Define instance variables
(Turtle 'ivars '(xpos ypos char))
; Answer "isnew" by initing a position and char and displaying.
(Turtle 'answer 'isnew '() '(
(setq xpos (setq newx (+ newx 1)))
(setq ypos 12)
(setq char "*")
(self 'display)
self))
still, it seems like pretty strong evidence that you can do an ergonomic embedded scripting language in 8 kilobytes or so, not 1900 kilobytes
Yeah it is the de facto scripting language for EDA tools (everything about designing and manufacturing ASICs and FPGAs).
It's pretty terrible but honestly Python would be pretty bad too. What these tools really need is a proper API that you can hook into and then use any language you want.
It's pretty terrible but honestly Python would be pretty bad too. What these tools really need is a proper API that you can hook into and then use any language you want.
I can also very much recommend his book about programming methodology. Lots of good sane advice there for anyone wanting to learn more about structuring code and keeping everything readable and bug free.
Related:
A brief interview with Tcl creator John Ousterhout - https://news.ycombinator.com/item?id=34733120 - Feb 2023 (92 comments)
A brief interview with Tcl creator John Ousterhout - https://news.ycombinator.com/item?id=34733120 - Feb 2023 (92 comments)
I want something like a "RAMcloud" for home use. A computer with more RAM than secondary storage that boots from USB and no pre-installed corporate OS.
Thanks John, I've kicked many a people from IRC using TCL.
[deleted]
[deleted]
[1] RAMcloud talk by J. Ousterhout:https://www.youtube.com/watch?v=lcUvU3b5co8
[2] RAMcloud paper: https://web.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf
[3] A Philosophy of Software Design, https://www.amazon.com/Philosophy-Software-Design-John-Ouste...