Lisp in Web-Based Applications (2001)(sep.turbifycdn.com)
sep.turbifycdn.com
Lisp in Web-Based Applications (2001)
https://sep.turbifycdn.com/ty/cdn/paulgraham/bbnexcerpts.txt
21 comments
The productivity high folks get from using tools like Claude Code are the same as what one gets from learning a Lisp and using something like Emacs as the editor.
There is obviously a difference in capability, but do the cost differences scale linearly?
Maybe when the party is over, the lost will yearn for the parentheses to get just a shadow of that high back.
There is obviously a difference in capability, but do the cost differences scale linearly?
Maybe when the party is over, the lost will yearn for the parentheses to get just a shadow of that high back.
It's not "productivity" that sucks them in, it's the feeling of being seen by an apparently nice and subservient person.
Lol, come on now, you think it's not productivity and only some sort of feeling of being acquiesced to?
Yes, absolutely.
Tell that to all the AI users then because that's a ridiculous assertion that even some notable fraction use it for the purpose you're talking about, much less all. Most people simply use it for information, very few treat it like a person.
Why? The creator of ELIZA tried that, it didn't work then, why would it now?
We've had code generation tools for decades, abstracting over code generation has been a solved problem for a long time, and as it happens, this is not in itself a very rewarding practice except for a small segment of professional developers that get a kick out of the elegance of some technique or other.
The new thing about these chat bots is that they provide a feeling of being seen and appreciated, and unlike a coworker they seemingly carry very little risk of embarrassment, pain and so on while they also require very little effort compared to human relations.
We've had code generation tools for decades, abstracting over code generation has been a solved problem for a long time, and as it happens, this is not in itself a very rewarding practice except for a small segment of professional developers that get a kick out of the elegance of some technique or other.
The new thing about these chat bots is that they provide a feeling of being seen and appreciated, and unlike a coworker they seemingly carry very little risk of embarrassment, pain and so on while they also require very little effort compared to human relations.
No, it's that they work extremely well at finding the specific information you're looking for. The vast majority of for example developers aren't using it to bond with a chatbot but to get work done.
I contributed to a contemporaneous large and complex system in Lisp (Scheme) that used most of those.
It was one of those things where 1-3 programmers, working with the right tools and thinking, could out-perform all competitors, no matter how much resources competitors threw at the problems. (I can't take credit for the initial architecture and implementation: that was all someone else.)
The system actually outlived Viaweb by decades, and I suspect still uses Scheme for a lot, especially for the necessarily complex backend.
(Half the problem for Scheme uptake was that most students only saw it school, as presented by a CS professor in an intro class, and then they did annoying homework, so they hoped never to use it again. They should've seen what we did with it, making it fly in ways that a CS professor wouldn't in first-year classes, compared to how it would've been done in other languages at the time.)
> One way we used macros was to generate Html. There is a very natural fit between macros and Html, because Html is a prefix notation like Lisp, and Html is recursive like Lisp. So we had macro calls within macro calls, generating the most complicated Html, and it was all still very manageable.
We just used lists and functions for HTML. And often with quasiquote, when we wanted to splice "dynamic" bits into a substantial chunk of "static".
I didn't try to use macros for HTML until a decade after this talk. I liked the simple DSL, and leaning on buffered I/O ports and string ports, but nobody else seemed to like it as much as I did (maybe because they didn't have a prolific colleague doing 10x code-writing in a rather more delicate way of generating HTML):
https://www.neilvandyke.org/racket/html-template/
(Don't look at the code, though. I wrote it before Racket got submodules, which are great if you have a strict, non-CL-ish module system and syntax extension mechanism that otherwise make some things much harder than in CL. I have a TODO note in there to refactor it with submodules, but I had to redirect my time towards other languages, for reasons having nothing to do with language merit.)
It was one of those things where 1-3 programmers, working with the right tools and thinking, could out-perform all competitors, no matter how much resources competitors threw at the problems. (I can't take credit for the initial architecture and implementation: that was all someone else.)
The system actually outlived Viaweb by decades, and I suspect still uses Scheme for a lot, especially for the necessarily complex backend.
(Half the problem for Scheme uptake was that most students only saw it school, as presented by a CS professor in an intro class, and then they did annoying homework, so they hoped never to use it again. They should've seen what we did with it, making it fly in ways that a CS professor wouldn't in first-year classes, compared to how it would've been done in other languages at the time.)
> One way we used macros was to generate Html. There is a very natural fit between macros and Html, because Html is a prefix notation like Lisp, and Html is recursive like Lisp. So we had macro calls within macro calls, generating the most complicated Html, and it was all still very manageable.
We just used lists and functions for HTML. And often with quasiquote, when we wanted to splice "dynamic" bits into a substantial chunk of "static".
I didn't try to use macros for HTML until a decade after this talk. I liked the simple DSL, and leaning on buffered I/O ports and string ports, but nobody else seemed to like it as much as I did (maybe because they didn't have a prolific colleague doing 10x code-writing in a rather more delicate way of generating HTML):
https://www.neilvandyke.org/racket/html-template/
(Don't look at the code, though. I wrote it before Racket got submodules, which are great if you have a strict, non-CL-ish module system and syntax extension mechanism that otherwise make some things much harder than in CL. I have a TODO note in there to refactor it with submodules, but I had to redirect my time towards other languages, for reasons having nothing to do with language merit.)
I wonder what your take is on the current landscape --- do you see this approach still having an edge? If you wanted to outcompete for a current-day SaaS app, do you see an undervalued approach that would still confer an advantage?
Great question, in light of vibe coding.
Yes, and no.
I absolutely still believe in small teams of highly-effective and aligned people. And they should be working smarter, and thinking forward and more holistically, than anything LLM-ish based generation and reasoning can do.
Let's say, 5 years ago, you already had vibe coding. It wasn't implemented atop LLMs, but by VCs giving you full-time use of a thousand junior software engineers and product designers, from a country of very energetic and educated people, but with much lower cost of living.
(Not offshoring in general, but a particular thousand-juniors kind, for the analogy.)
Your team of founders spends all their time prompting the thousand-juniors to do things. The thousand-juniors work very rapidly, but they only do what you tell them to do. And they tend to copy heavily, but with a little obfuscation. They don't think holistically. They always speak confidently, regardless of how little they understand. Because of how they were taught to knock off tasks, they never have an original thought, or don't voice or act on it if they did. They're just trying to knock off sprint tasks by technically doing what you told them. There is also a lot of bloat like a thousand juniors pasting together 'solutions' they looked up on Stackoverflow, which make little sense (e.g., pulled in a whole new framework to do one thing, rather than do it with the existing framework). This is how they were trained.
With your founding team trying to herd this thousand-juniors army of Jira ticket closers, they will churn huge amounts of code, and the appearance of functionality, very rapidly. No one will have occasion for much insight into anything.
That just might work, if it was ZIRP, and you are just putting up a Potemkin Village to get enterprise sales and more investment, until an exit. But is a terrible way to get PMF, innovation, or anything sustainable.
That said...
One way that this vibe coding competes with, say, a stereotypical "10x" Lisp hacker, is when you're a "business" cofounder, and you talk over a new idea with the hacker cofounder, who then worked through it all weekend, and shows you a PoC on Monday. You don't understand the subtleties of what the hacker did, but you're impressed that they could deliver "functionality" so quickly.
But today, your thousand-juniors can also deliver this "functionality" over the weekend. You can't (initially) tell that they just mechanically interpreted the task they were given, without original thought. And they maybe they even started by cloning and obfuscating two similar-sounding bases of GPL code.
But to you, the business person, the effect looks the same as the hacker who thought about it and used skill. And you might not be able to tell, for months, that the thousand-juniors collective result just isn't very bright, nor sustainable. (Not to mention that sometimes they veer a little too close to plagiarism or IP theft in how they work, and you don't know when you're gambling on whether they'll/you'll be caught.)
But the existence of this thousand-juniors rapid turnaround of "functionality", and you not being able to tell the difference between smart product engineering and slop, means that you never even try to build a more effective team, nor go through the experiences that help you become one.
Yes, and no.
I absolutely still believe in small teams of highly-effective and aligned people. And they should be working smarter, and thinking forward and more holistically, than anything LLM-ish based generation and reasoning can do.
Let's say, 5 years ago, you already had vibe coding. It wasn't implemented atop LLMs, but by VCs giving you full-time use of a thousand junior software engineers and product designers, from a country of very energetic and educated people, but with much lower cost of living.
(Not offshoring in general, but a particular thousand-juniors kind, for the analogy.)
Your team of founders spends all their time prompting the thousand-juniors to do things. The thousand-juniors work very rapidly, but they only do what you tell them to do. And they tend to copy heavily, but with a little obfuscation. They don't think holistically. They always speak confidently, regardless of how little they understand. Because of how they were taught to knock off tasks, they never have an original thought, or don't voice or act on it if they did. They're just trying to knock off sprint tasks by technically doing what you told them. There is also a lot of bloat like a thousand juniors pasting together 'solutions' they looked up on Stackoverflow, which make little sense (e.g., pulled in a whole new framework to do one thing, rather than do it with the existing framework). This is how they were trained.
With your founding team trying to herd this thousand-juniors army of Jira ticket closers, they will churn huge amounts of code, and the appearance of functionality, very rapidly. No one will have occasion for much insight into anything.
That just might work, if it was ZIRP, and you are just putting up a Potemkin Village to get enterprise sales and more investment, until an exit. But is a terrible way to get PMF, innovation, or anything sustainable.
That said...
One way that this vibe coding competes with, say, a stereotypical "10x" Lisp hacker, is when you're a "business" cofounder, and you talk over a new idea with the hacker cofounder, who then worked through it all weekend, and shows you a PoC on Monday. You don't understand the subtleties of what the hacker did, but you're impressed that they could deliver "functionality" so quickly.
But today, your thousand-juniors can also deliver this "functionality" over the weekend. You can't (initially) tell that they just mechanically interpreted the task they were given, without original thought. And they maybe they even started by cloning and obfuscating two similar-sounding bases of GPL code.
But to you, the business person, the effect looks the same as the hacker who thought about it and used skill. And you might not be able to tell, for months, that the thousand-juniors collective result just isn't very bright, nor sustainable. (Not to mention that sometimes they veer a little too close to plagiarism or IP theft in how they work, and you don't know when you're gambling on whether they'll/you'll be caught.)
But the existence of this thousand-juniors rapid turnaround of "functionality", and you not being able to tell the difference between smart product engineering and slop, means that you never even try to build a more effective team, nor go through the experiences that help you become one.
Great answer, with many thought-provoking parts and some gems ... I liked this thousand-juniors army of Jira ticket closers the best.
Paul Graham glosses over the details: writing LISP in continuation passing style is not for the faint hearted. For a modern implementation that hides the gnarly stuff, see https://docs.racket-lang.org/web-server/stateless.html
The disadvantage of this kind of stateless server is that you need to keep around a hash full of continuations, basically one for every page visit. Each of these continuations contains an entire environment and stack, and you have to manage all that memory allocation somehow. You need to serialize the continuations in order to cache them. Then you need to figure out which ones to keep around.
Authentication is potentially also an issue because absent some other security mechanism, anyone who knows the hash of the continuation can visit the page with the same permissions as the original user. https://docs.racket-lang.org/web-server/faq.html (10.7)
Apparently this website runs on LISP so I would be interested to know how it works under the hood. For me, anything more than a SPA seemed like too much pain.
The disadvantage of this kind of stateless server is that you need to keep around a hash full of continuations, basically one for every page visit. Each of these continuations contains an entire environment and stack, and you have to manage all that memory allocation somehow. You need to serialize the continuations in order to cache them. Then you need to figure out which ones to keep around.
Authentication is potentially also an issue because absent some other security mechanism, anyone who knows the hash of the continuation can visit the page with the same permissions as the original user. https://docs.racket-lang.org/web-server/faq.html (10.7)
Apparently this website runs on LISP so I would be interested to know how it works under the hood. For me, anything more than a SPA seemed like too much pain.
Using delimited continuations in a web framework is magic. In remember the first time I saw things like pharo and wodered how it was even possible. Doing that stuff in Python is still hard. Doing it in racket is easy peasy.
A BEAM process per user would have similar advantages, is pretty memory efficient, and does not need a lot of extra work.
love this on keyword parameters:
> Rtml even depended heavily on keyword parameters, which up to that time I had always considered one of the more dubious features of Common Lisp. Because of the way Web-based software gets released, you have to design the software so that it's easy to change. And Rtml itself had to be easy to change, just like any other part of the software. Most of the operators in Rtml were designed to take keyword parameters, and what a help that turned out to be. If I wanted to add another dimension to the behavior of one of the operators, I could just add a new keyword parameter, and everyone's existing templates would continue to work. A few of the Rtml operators didn't take keyword parameters, because I didn't think I'd ever need to change them, and almost every one I ended up kicking myself about later. If I could go back and start over from scratch, one of the things I'd change would be that I'd make every Rtml operator take keyword parameters.
> Rtml even depended heavily on keyword parameters, which up to that time I had always considered one of the more dubious features of Common Lisp. Because of the way Web-based software gets released, you have to design the software so that it's easy to change. And Rtml itself had to be easy to change, just like any other part of the software. Most of the operators in Rtml were designed to take keyword parameters, and what a help that turned out to be. If I wanted to add another dimension to the behavior of one of the operators, I could just add a new keyword parameter, and everyone's existing templates would continue to work. A few of the Rtml operators didn't take keyword parameters, because I didn't think I'd ever need to change them, and almost every one I ended up kicking myself about later. If I could go back and start over from scratch, one of the things I'd change would be that I'd make every Rtml operator take keyword parameters.
"Closures Simulate Subroutines" - I really love this way of coding for the web, it's what I did when I created FastScala: https://www.fastscala.com/ (same logic)
I found out recently that CloudFlare uses a dialect of Lisp for managing there DNS records:
https://blog.cloudflare.com/topaz-policy-engine-design/
https://blog.cloudflare.com/topaz-policy-engine-design/
Anyone know how redundant this is with the pg essays on his website? Not sure I've seen much about 'Rtml' or other technical details of Viaweb before.
This is one of the essays linked on his site: https://paulgraham.com/lwba.html
Which just links to the same ASCII-text link at https://sep.turbifycdn.com/ty/cdn/paulgraham/bbnexcerpts.txt, so it's exactly the same link as the one posted on HN here.
Which just links to the same ASCII-text link at https://sep.turbifycdn.com/ty/cdn/paulgraham/bbnexcerpts.txt, so it's exactly the same link as the one posted on HN here.
Here you will find some screenshots of some of today's web applications built in CL: http://lisp-screenshots.org/ and here an opinionated tutorial: https://web-apps-in-lisp.github.io/ One example: ScreenShotBot https://screenshotbot.io/ a successful open-source product and company. It now replaced Facebook's automatic screenshot testing tool (source: their blog). So, web apps in Lisp are possible -with a right amount of learning time and elbow grease.
The incremental development and interactive top-level are still precious and unmatched.