A Road to Common Lisp(stevelosh.com)
stevelosh.com
A Road to Common Lisp
http://stevelosh.com/blog/2018/08/a-road-to-common-lisp/
98 comments
My personal problem with CL libraries is not that, but rather the lack of documentation. More often than not, there is no documentation at all, not even a readme file.. It feels like some library authors simply don't care. I'd say this attitude has a negative impact on how people perceive viability of those libraries -- and by extension, of the language.
A lot of libraries that don't have separate documentation in the form of HTML/PDF/README.. are actually well-documented at source level in the form of docstrings.
Since Common Lisp is an interactive programming language and is meant to be used interactively (think Smalltalk, not Python) it is common practice to (interactively) load a library in a Common Lisp image and explore it (interactively). One can see what symbols are exported from packages, what these symbols are used for and (interactively) retrieve their documentation. All of this takes place within the editing environment (ideally Emacs) in a rapid feedback loop (again think Smalltalk, not Python) that feels seamless and tremendously empowering.
Since Common Lisp is an interactive programming language and is meant to be used interactively (think Smalltalk, not Python) it is common practice to (interactively) load a library in a Common Lisp image and explore it (interactively). One can see what symbols are exported from packages, what these symbols are used for and (interactively) retrieve their documentation. All of this takes place within the editing environment (ideally Emacs) in a rapid feedback loop (again think Smalltalk, not Python) that feels seamless and tremendously empowering.
I agree with this -- it's a real problem. My only solution has been to try to be the change I want to see in the world and document all of my own libraries before I officially release them.
In Freenode's #mercurial, sjl said this about common lisp:
http://stevelosh.com/blog/2012/04/volatile-software/
I kind of have the feeling that fighting bitrot is sjl's main motivation for CL.
<sjl_> CL code from the early 90's runs just fine on SBCL from a month ago
<sjl_> But trying to run five-year-old python/ruby/scala makes me hate life.
http://stevelosh.com/blog/2012/04/volatile-software/
I kind of have the feeling that fighting bitrot is sjl's main motivation for CL.
I don't know that I'd go so far as to call it my main motivation, but it is absolutely one of the strongest motivators, yes.
Running decades-old code on modern implementations is also a characteristic shared by other languages (e.g. QBasic with FreeBASIC) so I'd hope it's not anyone's main motivation for Lisp. Still, it's a big plus. This is a great writeup to see how to start tackling the elephant that is CL...
Elephant? Bit of a mixed metaphor, almost a malaphor, no? I'd say "tackle the behemoth" or "slay the dragon" instead. Elephants aren't usually tackled, they merely reside in rooms.
The phrase I was probably looking for was "eating the elephant", from Creighton Abrams' "When eating an elephant take one bite at a time."
Maybe "wrestling the alien" would work best for CL. Nose-against-arm.
Maybe "wrestling the alien" would work best for CL. Nose-against-arm.
[deleted]
Put the elephant to bed? "How the elephant got in my pajamas, I'll never know!"
It's a worthy effort. It really tires you after a while with languages that constantly break the language, the libraries, the ecosystem and the tooling in the ao called "minor versions". Ain't nobody got time to deal with that stuff. Then they are surprised that people still run "ancient" versions of compilers or libraries. Yes scala it's about you.
I stopped using Hugo for this reason. Some point release broke my ability to rebuild my site, and I was just donezo. Better to write like 100 lines of go to write the files myself.
[deleted]
[deleted]
I'm curious about python being on that list. I worked with python on multiple applications, some fairly large (500kloc), over 10+ years with multiple version upgrades without issue. I always considered python's backwards compatibility to be one of its strengths. That was why the python-3 was such a big deal, it would break backwards compatibility which had always been a top priority.
I am missing something?
I am missing something?
I think the Python problem is mostly about
https://xkcd.com/1987/
Which is to say, core Python may be fairly backwards compatible, but once you pip anything, well...
https://xkcd.com/1987/
Which is to say, core Python may be fairly backwards compatible, but once you pip anything, well...
Yep. Yep. Yep.
I once rewrote a Python program into Java just to avoid those problems. 10/10 would do again.
I once rewrote a Python program into Java just to avoid those problems. 10/10 would do again.
It's all much better now, especially since pip can (and does) install per-user. Of course, for anything non-trivial, you'd want a virtualenv anyway, and then you're dealing with a single place for all dependencies, and (usually) pinned version numbers.
I'm afraid I must disagree: Python packaging is as bad as its ever been; but now with more nuances to trip one up and occasionally work.
Gez, they should try running Swift from a few months ago
I haven't read the whole thing, but, I gotta say, seeing the "Ugliness" section (http://stevelosh.com/blog/2018/08/a-road-to-common-lisp/#ugl...) in the preamble gives me high hope for this writeup. It's so lovely to see writing on Lisp that isn't just hagiography.
Yes. I don't mind the parens (syntax highlighting can subdue them), but every time I actually fire up Common Lisp to try it, I see all caps somewhere, like someone even older than me is shouting at me, or I stumbled across a box of my Fortran punched cards from before I switched to APL in the 1970's. That's an ugliness not addressed by this guide.
There is a good reason for this "ugliness": it makes it easier to distinguish the things that you typed in vs the things that Lisp printed out. Nowadays we can make these distinctions with fonts, but back in the teletype days this was a really useful feature. Yes, the teletype days are over, but teriminal.app still exists and there are situations where interacting with Lisp directly through a terminal is useful, and so this feature can still be useful even today. Like the parens, it takes a little getting used to, but once you're used to it, you'll miss it when it's not there.
[deleted]
Yes, the same reason why all Wirth languages use uppercase for keywords.
And not a big deal on the age of automatic code formatters.
And not a big deal on the age of automatic code formatters.
That’s just the default. You can always put
And yeah, the default upcasing is ugly and inelegant, but they were trying to support the caps-only machines & code still in use at the time. It’s one of the key things I’d change in a modern Lisp standard.
But you have to admit: when the default print-case is one of the worst things in a language, that language is doing pretty well.
(setf *print-case* :downcase)
And yeah, the default upcasing is ugly and inelegant, but they were trying to support the caps-only machines & code still in use at the time. It’s one of the key things I’d change in a modern Lisp standard.
But you have to admit: when the default print-case is one of the worst things in a language, that language is doing pretty well.
TIL! Thanks for pointing this out; it makes me feel much better than my previous assumption that all these people running around proclaiming the greatness of CL were just _okay_ with the allcaps!
You can try to make it less shoutey by messing with readtable-case[1] but in my experience that's more trouble than it's worth. I found the uppercase a bit obnoxious at first too, but it's grown on me over time and now I find it quaint.
[1]: http://clhs.lisp.se/Body/f_rdtabl.htm
[1]: http://clhs.lisp.se/Body/f_rdtabl.htm
Because you can just turn it off?
https://www.cliki.net/Case%20sensitivity
[deleted]
Allegro CL can be configured to be case sensitive [1].
[1] https://franz.com/support/documentation/10.0/doc/case.htm
[1] https://franz.com/support/documentation/10.0/doc/case.htm
Thanks for this guide, I'll keep referring to it often.
My current road to Common Lisp is working through Peter Norvig's book Paradigms of Artificial Intelligence Programming [0]. It's not a direct route to the kind of programming most people do nowadays, but I hope to at least get a taste of what it was to be a researcher in classical AI.
[0] https://github.com/norvig/paip-lisp
My current road to Common Lisp is working through Peter Norvig's book Paradigms of Artificial Intelligence Programming [0]. It's not a direct route to the kind of programming most people do nowadays, but I hope to at least get a taste of what it was to be a researcher in classical AI.
[0] https://github.com/norvig/paip-lisp
Not only classical AI. If you bake in probability into logic-based Lisp & Prolog AI you end up in probabilistic programming. This is a very hot research topic.
You can also then add deep-learning based samplers. It's all connected.
You can also then add deep-learning based samplers. It's all connected.
I'm wondering what is people's opinion on modern Scheme dialects like Racket vs. Common Lisp? It seems to be pretty stable, with active development community, and comes with batteries installed.
Well, first off Racket is a different language from Scheme, although it's very similar. It's definitely very cool, and the folks involved have invested a tremendous amount of effort into it.
At the end of the day, though, I prefer Lisp. I like that it's standarised; I like that so much code runs in just about every implementation; I like that — as someone noted elsethread — in Lisp it's not uncommon for libraries to be done.
I like that Lisp is much more complete than Scheme. Standarised places are great. Standardised extensible types are wonderful. I don't like that so many in the Scheme community are so very opposed to adding to the language, no matter how painful the lack (witness the abject failure of R6RS).
CLOS is amazingly good, better than any object system in any other language I've used. Scheme doesn't have a standard version.
I think that multiple namespaces is a huge feature. A lot of Schemers disagree, but I don't see a good reason for functions, macros, classes, tags &c. to share a namespace, and it makes programs more obtuse.
I don't care for Scheme's separate Boolean types, nor for the way it splits NIL, () & #f. They make code less concise, for no terribly good reason IMHO. Maybe that's a matter of taste, but I think it reflects the pragmatism of Lisp vice the idealism of Scheme.
Lisp has standardised compiler macros. Lisp's normal macros are, I believe, more powerful than Scheme's (as I understand it, one can implement Scheme macros in Lisp but not Lisp macros in Scheme).
Scheme's dynamic-wind is broken, while UNWIND-PROTECT isn't.
Scheme's continuations in general are really awesome, but make it slightly too difficult to optimise code. I think it's great to have them available in an educational language, but not so great to have them in a general-purpose industrial language meant for real programs.
Generally, when I come across some corner of the Lisp standard I don't understand, some years later I'll recognise how incredibly valuable it is to be able to have it, and how great it is that every implementation has it. I never come across any corner of the Scheme standards, because they have no corners. Whenever I write Scheme I'm not really writing Scheme — I'm really writing guile or whatever.
Scheme's a wonderful language for teaching C.S. concepts like continuations & computer science in general — it's not IMHO a good language for industrial-strength software.
Racket is a single implementation of what used to be a Scheme but has now grown to be something else entirely different. It's really cool — I just wish everyone involved had spent that time on SBCL & portable Common Lisp libraries instead. It's a free world, of course!
At the end of the day, though, I prefer Lisp. I like that it's standarised; I like that so much code runs in just about every implementation; I like that — as someone noted elsethread — in Lisp it's not uncommon for libraries to be done.
I like that Lisp is much more complete than Scheme. Standarised places are great. Standardised extensible types are wonderful. I don't like that so many in the Scheme community are so very opposed to adding to the language, no matter how painful the lack (witness the abject failure of R6RS).
CLOS is amazingly good, better than any object system in any other language I've used. Scheme doesn't have a standard version.
I think that multiple namespaces is a huge feature. A lot of Schemers disagree, but I don't see a good reason for functions, macros, classes, tags &c. to share a namespace, and it makes programs more obtuse.
I don't care for Scheme's separate Boolean types, nor for the way it splits NIL, () & #f. They make code less concise, for no terribly good reason IMHO. Maybe that's a matter of taste, but I think it reflects the pragmatism of Lisp vice the idealism of Scheme.
Lisp has standardised compiler macros. Lisp's normal macros are, I believe, more powerful than Scheme's (as I understand it, one can implement Scheme macros in Lisp but not Lisp macros in Scheme).
Scheme's dynamic-wind is broken, while UNWIND-PROTECT isn't.
Scheme's continuations in general are really awesome, but make it slightly too difficult to optimise code. I think it's great to have them available in an educational language, but not so great to have them in a general-purpose industrial language meant for real programs.
Generally, when I come across some corner of the Lisp standard I don't understand, some years later I'll recognise how incredibly valuable it is to be able to have it, and how great it is that every implementation has it. I never come across any corner of the Scheme standards, because they have no corners. Whenever I write Scheme I'm not really writing Scheme — I'm really writing guile or whatever.
Scheme's a wonderful language for teaching C.S. concepts like continuations & computer science in general — it's not IMHO a good language for industrial-strength software.
Racket is a single implementation of what used to be a Scheme but has now grown to be something else entirely different. It's really cool — I just wish everyone involved had spent that time on SBCL & portable Common Lisp libraries instead. It's a free world, of course!
Agree with all points above.
I think the most important plus for CL is that CL is image-based, and that every important thing is already standardized, either on the ANSI standard, or de-facto via mature libraries.
I think the most important plus for CL is that CL is image-based, and that every important thing is already standardized, either on the ANSI standard, or de-facto via mature libraries.
> CL is image-based
CL is not image based. That's a popular implementation detail.
There are several implementations which are not image based. If we look closely we would find probably ten CL implementations which are not image-based of which maybe five are still somewhat in use. The Common Lisp standard also says absolutely nothing about programs/applications/images/libraries...
A prominent example for a CL implementation which is not image-based is ECL, Embeddable Common-Lisp:
https://common-lisp.net/project/ecl/static/manual/ch26.html
--- Traditionally, Common Lisp implemenations have provided a function to save the dump all data from a running Lisp process into a file. The result was called the Lisp image and could be shipped to other version compatible implementations.Nowadays, having less control of the systems it runs in, a Lisp implementation must work very hard to dump memory images and be able to load and execute them afterwards.
ECL has chosen to avoid this process entirely. Instead, we conceive five different portable models for building and shippin your programs. The models, described in Table 1.1, enumerate the different kinds of files that ECL can portably produce. To get one or more of the products mentioned in the table, you may resort to a low level API described in Part III. However, we recommend a simpler way based on using System Definition Files to describe the structure of your project and let ECL build the desired target for you. This approach is described in the following sections. ---
CL is not image based. That's a popular implementation detail.
There are several implementations which are not image based. If we look closely we would find probably ten CL implementations which are not image-based of which maybe five are still somewhat in use. The Common Lisp standard also says absolutely nothing about programs/applications/images/libraries...
A prominent example for a CL implementation which is not image-based is ECL, Embeddable Common-Lisp:
https://common-lisp.net/project/ecl/static/manual/ch26.html
--- Traditionally, Common Lisp implemenations have provided a function to save the dump all data from a running Lisp process into a file. The result was called the Lisp image and could be shipped to other version compatible implementations.Nowadays, having less control of the systems it runs in, a Lisp implementation must work very hard to dump memory images and be able to load and execute them afterwards.
ECL has chosen to avoid this process entirely. Instead, we conceive five different portable models for building and shippin your programs. The models, described in Table 1.1, enumerate the different kinds of files that ECL can portably produce. To get one or more of the products mentioned in the table, you may resort to a low level API described in Part III. However, we recommend a simpler way based on using System Definition Files to describe the structure of your project and let ECL build the desired target for you. This approach is described in the following sections. ---
You forgot arguably the most important bit: Racket is not an image-based interactive language (every time you evaluate code in DrRacket the entire state is wiped) and its designers do not think that's worth doing right. Of course there are some hacky solutions (geiser) that are full of gotchas and, compared to Common Lisp, can only be seen as entirely broken.
Just recently searched for an IMAP library but only found one for Allegro Common Lisp. The lack of libraries is one of my biggest concerns when it comes to using Lisp in production.
There are two more libs on Cliki:
https://www.cliki.net/email
And note that CLiki does not cover all that it's out there.
https://www.cliki.net/email
And note that CLiki does not cover all that it's out there.
I know from experience that clonsigna is broken :) I don’t think it would be difficult to fix, but, as is, it doesn’t work very well.
It isn't that bad:
- https://github.com/CodyReichert/awesome-cl - http://quickdocs.org/
- https://github.com/CodyReichert/awesome-cl - http://quickdocs.org/
Would you recommend learning Common Lisp over Clojure?
The author? Most likely
https://twitter.com/stevelosh/status/1034147772440760320?s=1...
I have a similar experience. Started out learning Clojure because it was 'practical', stayed with Common Lisp because I had less tooling setup to deal with. To be fair, that was because the Emacs integration for clojure depended on a specific CVS revision of Swank and I was starting out with Emacs. The situation has greatly improved regarding Clojure tooling. Still not as good as CL.
With time I've found CL to be much more flexible than Clojure and less opinionated so you'll be able to explore different paradigms.
That said if you want to write an SPA, go with ClojureScript. It has a good dev UX story (Figwheel <3) and Webpack will have already lowered your expectations regarding build systems to so setting up a ClojureScript project will seem less of a hassle.
https://twitter.com/stevelosh/status/1034147772440760320?s=1...
I have a similar experience. Started out learning Clojure because it was 'practical', stayed with Common Lisp because I had less tooling setup to deal with. To be fair, that was because the Emacs integration for clojure depended on a specific CVS revision of Swank and I was starting out with Emacs. The situation has greatly improved regarding Clojure tooling. Still not as good as CL.
With time I've found CL to be much more flexible than Clojure and less opinionated so you'll be able to explore different paradigms.
That said if you want to write an SPA, go with ClojureScript. It has a good dev UX story (Figwheel <3) and Webpack will have already lowered your expectations regarding build systems to so setting up a ClojureScript project will seem less of a hassle.
Clojure is really your better bet. It is highly practical with a good community and excellent Java interop. You never have to worry about finding a good library. The syntax is also a bit more easily parsable than CL. If you don't touch the Java interop stuff, it's just as elegant as CL.
No, Clojure is not as elegant (or flexible) as Common Lisp, not even close. Also, how can Clojure be highly practical when it ties you to the JVM? When it forces its worldview on state down your throat? When it prematurely optimizes with concepts like STM?
Common Lisp is all about choices and flexibility. Do you want pattern matching? You can get it but it's not shoved down your throat. Same for STM. Same for immutable data structures. Same for Java interop.
On the other hand, Clojure is lacking fundamental features of Common Lisp that are extremely powerful: conditions & restarts, programmable debugger, programmable reader, programmable compiler through compiler macros, dumping images & native code compilation not to mention advanced code analysis and optimization capabilities.
Common Lisp is all about choices and flexibility. Do you want pattern matching? You can get it but it's not shoved down your throat. Same for STM. Same for immutable data structures. Same for Java interop.
On the other hand, Clojure is lacking fundamental features of Common Lisp that are extremely powerful: conditions & restarts, programmable debugger, programmable reader, programmable compiler through compiler macros, dumping images & native code compilation not to mention advanced code analysis and optimization capabilities.
>Also, how can Clojure be highly practical when it ties you to the JVM?
The JVM allows you to use a ton of libraries, on a very performant platform. CL libraries just aren't anywhere near as feature-complete as in more popular languages, and this matters when you actually need to be productive. I don't have time to reinvent the wheel constantly. This practicality is what motivated Clojure in the first place, and why it is far and away the most popular lisp.
The JVM allows you to use a ton of libraries, on a very performant platform. CL libraries just aren't anywhere near as feature-complete as in more popular languages, and this matters when you actually need to be productive. I don't have time to reinvent the wheel constantly. This practicality is what motivated Clojure in the first place, and why it is far and away the most popular lisp.
I guess your definition of practical differs from mine. For me practicality is tied directly to usability in solving a widely disparate set of problems. This means that flexibility is key. The language I use should not only be able to bend and adapt to the problem at hand but also _not constrain my thinking_. The latter is also known as 'metalinguistic abstraction' and is strongly expressed in SICP as the philosophy of Lisp. Clojure mostly bypasses that since it has a very opinionated but also very constrained view on the problem-solving design space.
As an example of metalinguistic abstraction, some of the tasks I've successfully deployed Common Lisp at include creating a high-performance network stack that runs on ARMv8 and is based on JIT compilation and an entire assortment of solutions that sit on opposite ends of the highlevel-lowlevel spectrum. Concepts such as tight control over memory, stack allocation and one-to-one mapping with generated instructions were critical. Common Lisp allowed me not only to successfully investigate the domain but also to use the resulting code in production. For these tasks, Clojure would have been a total miss both due to implementation decisions (JVM) and its constraining nature -- in this case, immutability, memory and compilation model -- not allowing me to come up with a 'language' that will let me think the right thoughts.
As an example of metalinguistic abstraction, some of the tasks I've successfully deployed Common Lisp at include creating a high-performance network stack that runs on ARMv8 and is based on JIT compilation and an entire assortment of solutions that sit on opposite ends of the highlevel-lowlevel spectrum. Concepts such as tight control over memory, stack allocation and one-to-one mapping with generated instructions were critical. Common Lisp allowed me not only to successfully investigate the domain but also to use the resulting code in production. For these tasks, Clojure would have been a total miss both due to implementation decisions (JVM) and its constraining nature -- in this case, immutability, memory and compilation model -- not allowing me to come up with a 'language' that will let me think the right thoughts.
I don’t know of any collected data to support the claims in this Quora answer, but I think it does a decent job highlighting the different interpretations of “popularity”: https://www.quora.com/Which-is-the-most-popular-Lisp-dialect...
What’s your definition of popularity when you say that Clojure “is by far and away the most popular lisp”?
What’s your definition of popularity when you say that Clojure “is by far and away the most popular lisp”?
Number of github repos, which likely correlates to number of users. It also is probably a better gauge of the number of maintained, relevant libraries available. Clojure is used in production quite a bit nowadays, and there are excellent libraries for all kinds of modern tasks.
https://redmonk.com/sogrady/2018/03/07/language-rankings-1-1...
https://redmonk.com/sogrady/2018/03/07/language-rankings-1-1...
I've spent a few months learning Clojure - never gave CL a try. Here are a few thoughts on Clojure:
1. Excellent syntax and library support. It is my first Lisp, but I can't how I programmed without macros and persistent data structures.
2. I hate the stack traces. I've used both Clojure and Clojurescript (mainly cljs), and the stack traces for errors are nearly indecipherable. To be fair I am using React (not Reagent), but I don't find it too much better with other libraries.
3. I hate the build system. It is fractured and there are too many mediocre options. shadow-cljs is the best one I've used, and it works okay not great. I also hate that I can't distribute standalone binaries. I have used pkg, which distributes Node project as binaries, but the binaries are huge (something like 85MB+)
4. The core functional language is excellent, but protocols, types etc. seem much more ad-hoc and not well-designed. I'm used to object oriented (Python) and I find the lack of focus on an object-system a bit unsettling.
I have no clue if Common Lisp fixes these issues or brings new ones.
1. Excellent syntax and library support. It is my first Lisp, but I can't how I programmed without macros and persistent data structures.
2. I hate the stack traces. I've used both Clojure and Clojurescript (mainly cljs), and the stack traces for errors are nearly indecipherable. To be fair I am using React (not Reagent), but I don't find it too much better with other libraries.
3. I hate the build system. It is fractured and there are too many mediocre options. shadow-cljs is the best one I've used, and it works okay not great. I also hate that I can't distribute standalone binaries. I have used pkg, which distributes Node project as binaries, but the binaries are huge (something like 85MB+)
4. The core functional language is excellent, but protocols, types etc. seem much more ad-hoc and not well-designed. I'm used to object oriented (Python) and I find the lack of focus on an object-system a bit unsettling.
I have no clue if Common Lisp fixes these issues or brings new ones.
I'm not sure it's fair to bring up a fractured build system when talking about ClojureScript, and comparing it to Common Lisp, which you would not be compiling to JavaScript and interop'ing with NodeJS etc.
With (JVM) Clojure, leiningen has to be 90+% of the Clojure projects in the wild. And in that world, the equivalent to a native binary is an uberjar, which is rather easy and pretty much standardized.
The stack traces could be improved though.
With (JVM) Clojure, leiningen has to be 90+% of the Clojure projects in the wild. And in that world, the equivalent to a native binary is an uberjar, which is rather easy and pretty much standardized.
The stack traces could be improved though.
> Common Lisp, which you would not be compiling to JavaScript and interop'ing with NodeJS etc.
https://jscl-project.github.io
https://common-lisp.net/project/parenscript/
https://jscl-project.github.io
https://common-lisp.net/project/parenscript/
That's great, are you using these? How is the interop? How does that compare with ClojureScript and shadow-cljs? How can you make a single binary?
Maybe I worded wrongly the first time. I definitely didn't mean to imply that Common Lisp can't do something. That would be as foolish as saying Emacs can't do something ;)
I was just pointing out that we aren't comparing like-with-like once you bring in a totally different ecosystem such as JavaScript's (which is historically neither Common Lisp's nor Clojure's main focus).
Maybe I worded wrongly the first time. I definitely didn't mean to imply that Common Lisp can't do something. That would be as foolish as saying Emacs can't do something ;)
I was just pointing out that we aren't comparing like-with-like once you bring in a totally different ecosystem such as JavaScript's (which is historically neither Common Lisp's nor Clojure's main focus).
> That's great, are you using these?
Not often. In Common Lisp I don't have to use them.
Not often. In Common Lisp I don't have to use them.
To date I have not come up with a situation needing protocols or objects in Clojure that could not be just done with functional programming.
The real killer superpowers come from extend-type and extend-protocol. You can modify existing java/clojure libraries functionality like magic. The functional equivalent is with-binding if I remember correctly.
Some Clojure libraries use protocols heavily and IMHO not just necessary.
The real killer superpowers come from extend-type and extend-protocol. You can modify existing java/clojure libraries functionality like magic. The functional equivalent is with-binding if I remember correctly.
Some Clojure libraries use protocols heavily and IMHO not just necessary.
> The core functional language is excellent, but protocols, types etc. seem much more ad-hoc and not well-designed. I'm used to object oriented (Python) and I find the lack of focus on an object-system a bit unsettling.
I guess CLOS was well designed.
What Clojure offers is a subset of CLOS.
I guess CLOS was well designed.
What Clojure offers is a subset of CLOS.
Clojure multimethods actually can do more than CL multiple dispatch, because they can dispatch by value rather than just type.
You can implement Clojure multimethods as an extension of CLOS: https://github.com/pcostanza/filtered-functions
CLOS can dispatch by value with eql specifier...
> 2. I hate the stack traces. I've used both Clojure and Clojurescript (mainly cljs), and the stack traces for errors are nearly indecipherable. To be fair I am using React (not Reagent), but I don't find it too much better with other libraries.
Stack traces are already the wrong approach. Error handling is first. Stack traces may serve some purpose, but are secondary. Common Lisp was designed for interactive error handling.
An example. Let's say we have a function FAK and it returns the wrong result "0" for 0.
But we also get another REPL, but this time one level down and the original REPL is still there, one level up. This means we can do some computations in the error REPL:
So we decide to go one error level up and explore the problem there further. :c 3 chooses the restart to go to debug level 1.
Sure we can also get a stack trace - but the stacktrace is active in the context of the error - it's now a post-error stack trace - it's an in-error stack trace. Means we can, while we are in the error, see the stack trace and use it: change variables, restart start frames, return from stack frames, set break points to stack frames, ...
Let's say we are in the error again. Let's get a quick backtrace:
There are lots of ways to work with a stack trace while we are in the error - the display of the stack trace is only a minor feature.
Stack traces are already the wrong approach. Error handling is first. Stack traces may serve some purpose, but are secondary. Common Lisp was designed for interactive error handling.
An example. Let's say we have a function FAK and it returns the wrong result "0" for 0.
CL-USER 21 > (defun fak (n)
(if (zerop n)
"0"
(* n (fak (1- n)))))
FAK
CL-USER 22 > (fak 10)
Error: In * of (1 "0") arguments should be of type NUMBER.
1 (continue) Return a value to use.
2 Supply a new second argument.
3 (abort) Return to top loop level 0.
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
But we also get another REPL, but this time one level down and the original REPL is still there, one level up. This means we can do some computations in the error REPL:
CL-USER 23 : 1 > (fak 0)
"0"
CL-USER 24 : 1 > (fak 1)
Error: In * of (1 "0") arguments should be of type NUMBER.
1 (continue) Return a value to use.
2 Supply a new second argument.
3 (abort) return to debug level 1.
4 Return a value to use.
5 Supply a new second argument.
6 Return to top loop level 0.
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
So we decide to go one error level up and explore the problem there further. :c 3 chooses the restart to go to debug level 1.
CL-USER 25 : 2 > :c 3
CL-USER 26 : 1 > :c 1
Supply a form to be evaluated and used: 1
3628800
Sure we can also get a stack trace - but the stacktrace is active in the context of the error - it's now a post-error stack trace - it's an in-error stack trace. Means we can, while we are in the error, see the stack trace and use it: change variables, restart start frames, return from stack frames, set break points to stack frames, ...
Let's say we are in the error again. Let's get a quick backtrace:
CL-USER 58 : 1 > :bq
ERROR <- * <- FAK <- FAK <- FAK <- FAK <- FAK <- FAK <- FAK <- FAK <- FAK <- FAK <- EVAL
<- CAPI::CAPI-TOP-LEVEL-FUNCTION <- CAPI::INTERACTIVE-PANE-TOP-LOOP <- MP::PROCESS-SG-FUNCTION
CL-USER 59 : 1 > :n
Call to *
CL-USER 60 : 1 > :n
Interpreted call to FAK
CL-USER 61 : 1 > :n
Interpreted call to FAK
CL-USER 62 : 1 > :v
Interpreted call to FAK:
N : 2
CL-USER 63 : 1 > :ret 2
3628800
There are lots of ways to work with a stack trace while we are in the error - the display of the stack trace is only a minor feature.
I agree with this. The error messages in most Common Lisp environments are just so much better than you get in languages like Clojure or C# or Python.
I even have a good example to illustrate this. A while back someone had emailed me because he was trying to implement a terrain generation algorithm called Diamond Square, which I had written a blog post about. It was mostly working, but would crash with an IndexOutOfRangeException at one point. He had tried debugging it and emailed me to see if I could point him in the right direction.
Getting an array out of bounds exception didn't surprise me, because in Diamond Square the algorithm will try to read outside of the bounds of the array by default -- part of implementing the algorithm correctly is detecting that case and making sure to handle it, either by just ignoring that cell or by wrapping it around to the other side of the array. So I assumed he had just forgotten to do this. But then I read his code, and no, he had definitely added a check for `if (x >= array.length)`.
This code was using Unity, so it was written in C#, but the point applies to most languages. Look at the errors given back by C# and Clojure for this problem:
If I had been able to get this running in a debugger I probably could have figured it out, sure. But Common Lisp's nice error messages made that unnecessary -- instead of telling me "You tried to access an invalid index" it says "You tried to access X, but I was expecting Y". It's so much nicer, and in a lot of cases (like this one) instantly makes the error completely obvious.
I don't know exactly why Common Lisp's error messages tend to be so much better than most other dynamic languages. Maybe it's a culture of good error writing from the very start. Maybe it's because the language itself makes it easy to provide good messages, thanks to things like CHECK-TYPE, ECASE, ETYPECASE, etc (and their `C` variants, which is a whole extra layer of goodness on top of all that). I don't know. I just know that hitting an error in most languages feels like hitting a brick wall (especially in JVM languages, good god are those stack traces awful) but in Common Lisp it feels like the system is at least trying to do its best to help me.
I even have a good example to illustrate this. A while back someone had emailed me because he was trying to implement a terrain generation algorithm called Diamond Square, which I had written a blog post about. It was mostly working, but would crash with an IndexOutOfRangeException at one point. He had tried debugging it and emailed me to see if I could point him in the right direction.
Getting an array out of bounds exception didn't surprise me, because in Diamond Square the algorithm will try to read outside of the bounds of the array by default -- part of implementing the algorithm correctly is detecting that case and making sure to handle it, either by just ignoring that cell or by wrapping it around to the other side of the array. So I assumed he had just forgotten to do this. But then I read his code, and no, he had definitely added a check for `if (x >= array.length)`.
This code was using Unity, so it was written in C#, but the point applies to most languages. Look at the errors given back by C# and Clojure for this problem:
user=> (nth [:a :b :c] x)
IndexOutOfBoundsException clojure.lang.PersistentVector.arrayFor (PersistentVector.java:153)
float[] fs = new float[5];
Console.WriteLine(fs[x]);
Unhandled Exception:
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at MainClass.Main (System.String[] args) [0x00009] in <fdc0f271de194e269b9b6c61b644f47b>:0
[SBCL] CL-USER> (aref #(:a :b :c) *x*)
debugger invoked on a SB-INT:INVALID-ARRAY-INDEX-ERROR in thread
#<THREAD "main thread" RUNNING {10005505B3}>:
Invalid index -1 for (SIMPLE-VECTOR 3), should be a non-negative integer below 3.
If I had been able to get this running in a debugger I probably could have figured it out, sure. But Common Lisp's nice error messages made that unnecessary -- instead of telling me "You tried to access an invalid index" it says "You tried to access X, but I was expecting Y". It's so much nicer, and in a lot of cases (like this one) instantly makes the error completely obvious.
I don't know exactly why Common Lisp's error messages tend to be so much better than most other dynamic languages. Maybe it's a culture of good error writing from the very start. Maybe it's because the language itself makes it easy to provide good messages, thanks to things like CHECK-TYPE, ECASE, ETYPECASE, etc (and their `C` variants, which is a whole extra layer of goodness on top of all that). I don't know. I just know that hitting an error in most languages feels like hitting a brick wall (especially in JVM languages, good god are those stack traces awful) but in Common Lisp it feels like the system is at least trying to do its best to help me.
I'm not very deep into Clojure - but I find that you end up having to use Java libraries which feels kinda clumsy. Java is a stateful language and Clojure has it's own ideas about how state is managed - so I'm never quite sure how to mix the two (based on some very limited experience... so I might be off-base)
I'm also not very deep into Clojure, but I've had the opposite experience so far. I've used it for some real-world projects and the most Java I've ever needed to pull in was a BufferedReader that I wrapped and forgot about.
Is it reasonable to do Clojure and not have to touch Java?
It's very possible. I use Clojure on a daily basis, and I only rarely need Java interop to use pretty obscure libraries. It's not that bad, it's not any worse than using Java directly.
An excellent guide, thanks!
One suggestion: checked again just now and SBCL is still not production-ready on Windows (for the understandable reason of insufficient volunteers); perhaps the recommendation for that platform should be changed to CCL?
One suggestion: checked again just now and SBCL is still not production-ready on Windows (for the understandable reason of insufficient volunteers); perhaps the recommendation for that platform should be changed to CCL?
>One suggestion: checked again just now and SBCL is still not production-ready on Windows (for the understandable reason of insufficient volunteers); perhaps the recommendation for that platform should be changed to CCL?
It only has a warning for threading code that has been left there for years. But I use it on windows with no problems.
On the other hand Clozure CL is a very very good implementation with a loooooooooooooooooooooooooooooooooooooooooooooooooong history (emphasis added) being used in production stuff.
But don't limit yourself to SBCL and CCL -- take a look also at ECL, ABCL, CLASP, etc.
It only has a warning for threading code that has been left there for years. But I use it on windows with no problems.
On the other hand Clozure CL is a very very good implementation with a loooooooooooooooooooooooooooooooooooooooooooooooooong history (emphasis added) being used in production stuff.
But don't limit yourself to SBCL and CCL -- take a look also at ECL, ABCL, CLASP, etc.
The Windows x86 version of SBCL crashes immediately on startup, so I figured the warning on the x64 version should be taken seriously, but if you are saying it is solid after all, that is good news.
Going to give CCL a try.
Going to give CCL a try.
> If you’re like me and already have Vim burned too deeply into your fingers to ever get it out, I’d recommend Vim with Vlime.
Anyone has tried both Slimv and Vlime for Vim? What are the differences? Which one gives an experience closer to that of SLIME?
Anyone has tried both Slimv and Vlime for Vim? What are the differences? Which one gives an experience closer to that of SLIME?
I've tried both. Slimv was always very buggy for me, especially the REPL buffer. I think this was because it was made long before any of the new Vim async stuff existed, and so had to do a lot of ugly hacks to get a reasonable REPL.
Vlime was made after Neovim gave Vim a kick in the ass to add async, and it takes advantage of all of it. This lets its implementation be a lot cleaner and more stable, at least from what I've experienced.
One thing I do that makes the REPL a lot nicer: I run the actual SBCL process inside a Neovim terminal split (with rlwrap). This gives me an actual REPL like you would expect, not just Vlime's "REPL" (which is essentially two separate buffers, one for input and one for output).
Vlime was made after Neovim gave Vim a kick in the ass to add async, and it takes advantage of all of it. This lets its implementation be a lot cleaner and more stable, at least from what I've experienced.
One thing I do that makes the REPL a lot nicer: I run the actual SBCL process inside a Neovim terminal split (with rlwrap). This gives me an actual REPL like you would expect, not just Vlime's "REPL" (which is essentially two separate buffers, one for input and one for output).
I've been using slimv for hobby hacking for a while, but not professionally so you can take my commentary with a big grain of salt. You should try both, and if 'stevelosh likes vlime better you might start with that.
I like slimv a lot better myself, and prior to that I made due with a gnu screen split-window terminal with a vim plugin that would send stuff from one screen panel to the other (used that for Python, Clojure, and Node sometimes too). I tried using vlime somewhat recently, but it just felt off, hard to express everything I didn't like but maybe the experience of having to launch your REPL separately was the beginning (slimv just finds your lisp on the path). You're encouraged to compile whole files at once rather than bit by bit (perhaps sensible for Real Work), the REPL buffer is read-only which is quite bizarre to me, and the default key bindings make less sense. Feature-wise it seemed comparable since they both use Swank. The tutorial at https://kovisoft.bitbucket.io/tutorial.html which follows a classic SLIME demo vid is nicer than the vim-tutor for vlime.
I like slimv a lot better myself, and prior to that I made due with a gnu screen split-window terminal with a vim plugin that would send stuff from one screen panel to the other (used that for Python, Clojure, and Node sometimes too). I tried using vlime somewhat recently, but it just felt off, hard to express everything I didn't like but maybe the experience of having to launch your REPL separately was the beginning (slimv just finds your lisp on the path). You're encouraged to compile whole files at once rather than bit by bit (perhaps sensible for Real Work), the REPL buffer is read-only which is quite bizarre to me, and the default key bindings make less sense. Feature-wise it seemed comparable since they both use Swank. The tutorial at https://kovisoft.bitbucket.io/tutorial.html which follows a classic SLIME demo vid is nicer than the vim-tutor for vlime.
> the REPL buffer is read-only which is quite bizarre to me
The way I work around this is to run the SBCL process inside a Neovim terminal split (with rlwrap). That way I get a vanilla SBCL REPL plus the stability of Vlime.
The way I work around this is to run the SBCL process inside a Neovim terminal split (with rlwrap). That way I get a vanilla SBCL REPL plus the stability of Vlime.
I just wish common lisp people would embrace static typing so that I could ask the reply/compiler: After all these changes, is the code still formally correct (as far as types were specified)? Typed racket/clojure show it's possible.
I do quite like common lisp, but:
> In Common Lisp you can certainly choose to panic on or ignore errors, but there’s a better way to work. When an error is signaled in Common Lisp, it doesn’t unwind the stack. The Lisp process will pause execution at that point and open a window in your editor showing you the stack trace. Your warrior’s sword is hovering over the monster, waiting for you. At this point you can communicate with the running process at the REPL to see what’s going on. You can examine variables in the stack, or even run any arbitrary code you want.
This doesn't seem like something that's particularly difficult to do with c/gdb.
> In Common Lisp you can certainly choose to panic on or ignore errors, but there’s a better way to work. When an error is signaled in Common Lisp, it doesn’t unwind the stack. The Lisp process will pause execution at that point and open a window in your editor showing you the stack trace. Your warrior’s sword is hovering over the monster, waiting for you. At this point you can communicate with the running process at the REPL to see what’s going on. You can examine variables in the stack, or even run any arbitrary code you want.
This doesn't seem like something that's particularly difficult to do with c/gdb.
> This doesn't seem like something that's particularly difficult to do with c/gdb.
Well, why don't you do it, then? Not trying to be snarky(not much anyway), but I feel you are seriously under-estimating the work involved. Either that, or the description does not make it clear enough.
It's a full blown REPL. We are not talking about just printing out the code at that point, or maybe mutating a variable on the stack. We can execute arbitrary code and even replace entire sections of code with new code, and then hit continue and it works as if nothing happened, only now running the new (and hopefully corrected) code. Just the memory management implications of doing this in C would be very interesting.
Note that you can also do this over a network connection. NASA folks used this to great effect to patch Deep Space-1. Lisp machines also used this capability over the entire OS. I have also used a similar mechanism (with Chicken Scheme) to do incremental development on iOS, without having to do the whole xcode deploy/run cycle.
The closest analog I can think of today is a browser's "console", which lets you do similar things for javascript code.
Well, why don't you do it, then? Not trying to be snarky(not much anyway), but I feel you are seriously under-estimating the work involved. Either that, or the description does not make it clear enough.
It's a full blown REPL. We are not talking about just printing out the code at that point, or maybe mutating a variable on the stack. We can execute arbitrary code and even replace entire sections of code with new code, and then hit continue and it works as if nothing happened, only now running the new (and hopefully corrected) code. Just the memory management implications of doing this in C would be very interesting.
Note that you can also do this over a network connection. NASA folks used this to great effect to patch Deep Space-1. Lisp machines also used this capability over the entire OS. I have also used a similar mechanism (with Chicken Scheme) to do incremental development on iOS, without having to do the whole xcode deploy/run cycle.
The closest analog I can think of today is a browser's "console", which lets you do similar things for javascript code.
The closet analog is Smalltalk.
Java and .NET allow for partial edit-and-continue, but they aren't as powerful as Lisp/Smalltalk.
Java and .NET allow for partial edit-and-continue, but they aren't as powerful as Lisp/Smalltalk.
You can put your code in shared objects and then reload them. Well, I've never done it myself as I haven't seen the need but it seems easy enough, you'd simply have to define a macro for defining functions (you'd just have to write them as FUN(void, bla)(int x, int y) instead of void bla(int x, int y)), which would add the function to a symbol table so then you can say reload("bla") from within gdb and then keep running. I'm not quite sure what you mean by 'the memory management implications of doing this in C would be very interesting'? The only thing, really, that you'd expect to stop working static variables.
You might be interested to read more about this not particularly difficult problem, known in general as "dynamic software updating". You can get a general overview on Wikipedia, but I'd recommend having a browse through the work of Michael Hicks and colleagues: https://www.cs.umd.edu/~mwh/papers/mwh-cat.html
They've worked on a number of iterations on this idea, of which these are probably the most interesting and relevant to C:
Original DSU paper (2001): https://www.cs.umd.edu/~mwh/papers/HicksMN01a.html
Proteus (2005/2007): https://www.cs.umd.edu/~mwh/papers/StoyleHBSN06.html
Ginseng (2006): https://www.cs.umd.edu/~mwh/papers/neamtiu06dsu.html
Kitsune (2012): https://www.cs.umd.edu/~mwh/papers/hayden12kitsune.html
I'd caution you that "I've never done it, but it seems easy" is the first symptom of early stage Dunning-Kruger. Don't ignore the warning signs!
They've worked on a number of iterations on this idea, of which these are probably the most interesting and relevant to C:
Original DSU paper (2001): https://www.cs.umd.edu/~mwh/papers/HicksMN01a.html
Proteus (2005/2007): https://www.cs.umd.edu/~mwh/papers/StoyleHBSN06.html
Ginseng (2006): https://www.cs.umd.edu/~mwh/papers/neamtiu06dsu.html
Kitsune (2012): https://www.cs.umd.edu/~mwh/papers/hayden12kitsune.html
I'd caution you that "I've never done it, but it seems easy" is the first symptom of early stage Dunning-Kruger. Don't ignore the warning signs!
There are a few ways in which the shared objects method you suggest does not match the full power of Lisp debugging.
First of all, when a C program crashes, it just crashes. There is no REPL. There is only a core dump. So any live-debugging you plan to do is after the fact. After you have seen a crash, you would now begin to prepare for the next crash by launching your process via GDB or restarting your process and attaching a GDB to it. Whether a similar crash would occur again or not or when it would occur again depends on the nature of the bug. Now contrast this with Lisp debugging when your program crashes, it stops there and offers you an REPL to interact with the program right then. There is no need to wait for the next crash.
Secondly, when you debug with GDB, you would be dealing with syntaxes: The syntax of C that we are so familiar with. The GDB syntax to investigate the problem that we may be less familiar with. When the Lisp debugger offers the REPL to you, you are working with Lisp again. Your compiler, debugger, program, etc. all are part of the same unified environment where you just execute Lisp code to debug your issue.
Finally, putting your code in shared objects and reloading them requires you to go through the complete write-build-test-debug cycle. And then what do you do if your shared object itself crashes? With Lisp you skip the write-build-test part when all you want to do is debug an error. You jump straight to the debug part of the cycle and begin investigating the runtime state. And it works the same in a uniform manner whether your main program crashes or a dependency crashes.
First of all, when a C program crashes, it just crashes. There is no REPL. There is only a core dump. So any live-debugging you plan to do is after the fact. After you have seen a crash, you would now begin to prepare for the next crash by launching your process via GDB or restarting your process and attaching a GDB to it. Whether a similar crash would occur again or not or when it would occur again depends on the nature of the bug. Now contrast this with Lisp debugging when your program crashes, it stops there and offers you an REPL to interact with the program right then. There is no need to wait for the next crash.
Secondly, when you debug with GDB, you would be dealing with syntaxes: The syntax of C that we are so familiar with. The GDB syntax to investigate the problem that we may be less familiar with. When the Lisp debugger offers the REPL to you, you are working with Lisp again. Your compiler, debugger, program, etc. all are part of the same unified environment where you just execute Lisp code to debug your issue.
Finally, putting your code in shared objects and reloading them requires you to go through the complete write-build-test-debug cycle. And then what do you do if your shared object itself crashes? With Lisp you skip the write-build-test part when all you want to do is debug an error. You jump straight to the debug part of the cycle and begin investigating the runtime state. And it works the same in a uniform manner whether your main program crashes or a dependency crashes.
A while back, I remember running samba with the configuration option "panic action = /usr/bin/X11/xterm -display :0 -e gdb ...". On a segfault, you'd get a happy little xterm ready to go. Never managed to recover an operation from that window, though.
I have caught a condition in StumpWM (a window manager written in Common Lisp) and done something useful with it. A completely different beast. Practical Common Lisp has a solid example that may be a bit enlightening (http://www.gigamonkeys.com/book/beyond-exception-handling-co...).
I have caught a condition in StumpWM (a window manager written in Common Lisp) and done something useful with it. A completely different beast. Practical Common Lisp has a solid example that may be a bit enlightening (http://www.gigamonkeys.com/book/beyond-exception-handling-co...).
I think this might give a more in-depth introduction to CL’s conditions/restarts: http://www.gigamonkeys.com/book/beyond-exception-handling-co...
And for more on the history and philosophy behind it: http://www.nhplace.com/kent/Papers/Condition-Handling-2001.h...
Live editing the running code without a restart may be a mite harder in C.
It's very hard (corner cases are nasty). That said, there are IDEs out there that support that sort of thing, e.g. https://docs.microsoft.com/en-us/visualstudio/debugger/edit-...
> https://docs.microsoft.com/en-us/visualstudio/debugger/suppo...
Looks like it has some severe limitations when it comes to interactive object-oriented programming:
> Changes to a data type that affect the layout of an object, such as data members of a class.
The Common Lisp Object System does not have such a limitation. The price to pay is some indirection. You don't even need to create new instances from your class. The existing instances will be updated for runtime changes: different inheritance, new slot, removed slot, different class, ...
Looks like it has some severe limitations when it comes to interactive object-oriented programming:
> Changes to a data type that affect the layout of an object, such as data members of a class.
The Common Lisp Object System does not have such a limitation. The price to pay is some indirection. You don't even need to create new instances from your class. The existing instances will be updated for runtime changes: different inheritance, new slot, removed slot, different class, ...
I have implemented live coding via DLLs for C , it’s not hard to do and can easily replace gdb. I don’t think that gdb offers live coding outside the box. Inspecting live state and to an extend REPL abilities do exist in GDB.
This is a godsend. Probably the best introductory article on Lisp to date.
I have found this to be true in my own experience. The perception of stagnation is, however, a common initial objection to folks working in CL for the first time.