Why Does Elisp Suck(emacswiki.org)
emacswiki.org
Why Does Elisp Suck
https://www.emacswiki.org/emacs/WhyDoesElispSuck
143 comments
> Certain points like the fact that Emacs isn't multithreaded are thrown around by people who don't have the intuition that multithreading isn't the right thing for a lot of applications. The added complexity that multithreading would add to Emacs would seriously outweigh the usefulness it would provide. Emacs already has a decent process model, and having to deal with only a single shared state makes programs much cleaner.
I disagree with this point. Emacs does not have a useful process model, or at least it cannot be used to the full extend, because of shared global mutable state. This is exactly the issue, not a practical thing. If it were not for all the usage of mutable (and mutated) global state, Emacs could actually benefit from running many things concurrently, without annoying users with blocking behavior.
This is of course a huge task to refactor Emacs like that. I can only hope, that some day we will get there, because I see it as one fatal flaw, that could over time kill Emacs. Yes, Emacs as it is is super useful, but more and more things come up, that will be better off running concurrently and in parallel. Just to name a few: Package compilation, waiting for LSP stuff, anything that waits for network, like tramp, running source blocks in org-mode (org-babel), syntax highlighting of big files, magit stuff like rendering huge diffs in a magit status buffer ...
All of those things could benefit from being able to use multiple cores and improve Emacs' snappiness and speed. All of these come before even changing anything about elisp the language.
I disagree with this point. Emacs does not have a useful process model, or at least it cannot be used to the full extend, because of shared global mutable state. This is exactly the issue, not a practical thing. If it were not for all the usage of mutable (and mutated) global state, Emacs could actually benefit from running many things concurrently, without annoying users with blocking behavior.
This is of course a huge task to refactor Emacs like that. I can only hope, that some day we will get there, because I see it as one fatal flaw, that could over time kill Emacs. Yes, Emacs as it is is super useful, but more and more things come up, that will be better off running concurrently and in parallel. Just to name a few: Package compilation, waiting for LSP stuff, anything that waits for network, like tramp, running source blocks in org-mode (org-babel), syntax highlighting of big files, magit stuff like rendering huge diffs in a magit status buffer ...
All of those things could benefit from being able to use multiple cores and improve Emacs' snappiness and speed. All of these come before even changing anything about elisp the language.
I would say that proper threads with shared mutable state would make things much more fragile and high-ceremony, with locks, atomics, and potential races everywhere. For real multi-core performance gains, I would much prefer a model like JavaScript's workers, or Python's multiprocessing, with easy but explicit shared memory interface where message passing with serde does not suffice.
I'm pretty sure out of the box, Emacs can native compile packages in parallel ( https://www.jamescherti.com/emacs-native-compilation-config-... ) unless I'm misunderstanding something. Elpaca, which is the package manager I use, can download and install packages in parallel. lsp-bridge intercepts Emacs' LSP client and in my experience significantly reduces perceived latency for clangd.
[deleted]
> Emacs can native compile packages in parallel
via external "jobs"
via external "jobs"
I have always found this view interesting. Emacs has been able to run external tools with sentinels for a long time. Compilation mode being the easy example to show this.
It is somewhat cumbersome, no doubt, but for all of the code that didn't decide to use that and can easily cause emacs to stall, I see no reason to think we won't have as much or more problems with multiple threads.
It is somewhat cumbersome, no doubt, but for all of the code that didn't decide to use that and can easily cause emacs to stall, I see no reason to think we won't have as much or more problems with multiple threads.
If I understand correctly, sentinels (https://www.gnu.org/software/emacs/manual/html_node/elisp/Se...) are about external processes, like running a shell command concurrently. So for example one can run a `git status` in a separate process and have a sentinel for that. Now if I think about magit, it might very well run some git status, but what about rendering a huge diff in the magit status buffer? Can such a thing even be done in an external process? Or does that necessarily stall Emacs?
There are a few things here. You are correct that this is for external processes. You are further correct that you can push the heavy part of a diff and such to an external process.
At that point, you have the textual output in a buffer and you want emacs to simply render that buffer in a "pretty" way. This is done for many things, grep being one people are most familiar with in emacs. It can recognize each match with enough detail to let you jump to the file it is in. I don't know how magit renders diffs, but I'd imagine it should work roughly the same way. Probably leaning on some other elisp to allow common editing.
My point holds that more and more languages have server protocols that would be akin to an external process, now. In this way, there is little benefit in reimplementing a ton of stuff in the emacs process. Especially with the standardization of leaning on json, the link to the external process can be a textual buffer just fine.
At that point, you have the textual output in a buffer and you want emacs to simply render that buffer in a "pretty" way. This is done for many things, grep being one people are most familiar with in emacs. It can recognize each match with enough detail to let you jump to the file it is in. I don't know how magit renders diffs, but I'd imagine it should work roughly the same way. Probably leaning on some other elisp to allow common editing.
My point holds that more and more languages have server protocols that would be akin to an external process, now. In this way, there is little benefit in reimplementing a ton of stuff in the emacs process. Especially with the standardization of leaning on json, the link to the external process can be a textual buffer just fine.
> Certain points like the fact that Emacs isn't multithreaded are thrown around by people who don't have the intuition that multithreading isn't the right thing for a lot of applications.
GNU Emacs has a lot of applications where it is the right thing. It's no longer the simple editor. It comes with more than a million lines of Lisp code implementing all kinds of complex features: like various network client applications, IDEs, ...
> Common Lisp+SLIME shares some of this convenience, but not to the extent that Elisp truly does
SLIME is mostly implemented in Emacs Lisp. A lot of other Lisp systems can locate all source code and all documentation. It's not so much a special feature of Emacs Lisp, but its development environment. It's also not necessary that the IDE and Lisp runs in the same process / same machine to be able to look up documentation and code. It may be convenient in GNU Emacs, but any such editor can provide such features for programming language implementations.
GNU Emacs has a lot of applications where it is the right thing. It's no longer the simple editor. It comes with more than a million lines of Lisp code implementing all kinds of complex features: like various network client applications, IDEs, ...
> Common Lisp+SLIME shares some of this convenience, but not to the extent that Elisp truly does
SLIME is mostly implemented in Emacs Lisp. A lot of other Lisp systems can locate all source code and all documentation. It's not so much a special feature of Emacs Lisp, but its development environment. It's also not necessary that the IDE and Lisp runs in the same process / same machine to be able to look up documentation and code. It may be convenient in GNU Emacs, but any such editor can provide such features for programming language implementations.
> GNU Emacs has a lot of applications where it is the right thing. It's no longer the simple editor. It comes with more than a million lines of Lisp code implementing all kinds of complex features: like various network client applications, IDEs, ...
Multi-threading has landed in emacs anyway, by the way: https://www.gnu.org/software/emacs/manual/html_node/elisp/Th...
Multi-threading has landed in emacs anyway, by the way: https://www.gnu.org/software/emacs/manual/html_node/elisp/Th...
I experimented with that, but it doesn't use more than one CPU core, so it's more of a proof of concept at the moment.
You could use it to make non-blocking user experience perhaps, but you could also do it using `while-no-input` or a library like deferred.el or my https://github.com/meedstrom/asyncloop.
Anyway, there is `make-process` for multicore work. Takes some boilerplate and studying, but no more than with threads.
You could use it to make non-blocking user experience perhaps, but you could also do it using `while-no-input` or a library like deferred.el or my https://github.com/meedstrom/asyncloop.
Anyway, there is `make-process` for multicore work. Takes some boilerplate and studying, but no more than with threads.
see the word "limited" in the documentation. It's some kind of "cooperative thread" mechanism...
We can also use SBCL's multi threading in Elisp
https://git.sr.ht/~hajovonta/glue/tree/master/item/README.or...
https://git.sr.ht/~hajovonta/glue/tree/master/item/README.or...
Yes, that runs Common Lisp code in an external Common Lisp instance via a SLIME / SLY connection, using the CL thread features. It can use Bordeaux-Threads as API for threads in various CL implementations.
https://github.com/sionescu/bordeaux-threads
> Certain points like the fact that Emacs isn't multithreaded are thrown around by people who don't have the intuition that multithreading isn't the right thing for a lot of applications. The added complexity that multithreading would add to Emacs would seriously outweigh the usefulness it would provide. Emacs already has a decent process model, and having to deal with only a single shared state makes programs much cleaner.
I theoretically understand this but having to deal with company-mode providers that, when they start performing badly, start hanging everything is not fun!
Obviously thread-count and async-codestyle-ease are not the same thing, but given the world of LSPs and plugins that do I/O on the main thread (this is a big enough reason to have async interfaces that are actually used) it would be good to have some effort spent there
I theoretically understand this but having to deal with company-mode providers that, when they start performing badly, start hanging everything is not fun!
Obviously thread-count and async-codestyle-ease are not the same thing, but given the world of LSPs and plugins that do I/O on the main thread (this is a big enough reason to have async interfaces that are actually used) it would be good to have some effort spent there
Yes, on the particular issue of company/corfu, I'm actually tempted to see if we can't just specify that the work of `completion-at-point-functions` must be done in an external process. Then calling `completion-at-point` would never directly call third-party code that may block the main thread.
Maybe for speed, `completion-at-point` would keep a fairly recent `dump-emacs-portable` image, and spin up external processes using that, with has the bonus that the completion-at-point-functions get to inspect current state.
Maybe for speed, `completion-at-point` would keep a fairly recent `dump-emacs-portable` image, and spin up external processes using that, with has the bonus that the completion-at-point-functions get to inspect current state.
I'm curious what makes you so sure those plugins will actually use multithreading appropriately? Emacs already has methods to communicate with processes off the main thread, right?
So some of this stuff is "communicate with an external process", and I feel like many things in that space do the right thing (if only because if you don't it's immediately visible).
I have a lot of issues with "pure elisp" plugins though. Things where these plugins are doing a lot of string munging, and it's fast in testing because the tests aren't that big. But you can easily hit some edge case where suddenly you're looking for occurrences of `e` in Moby Dick so now each letter press takes a second (fortunately computers are still fast!)
If completion frameworks were async-by-default, at least that stuff wouldn't hang everything for me. At least that's my theory.
I have a lot of issues with "pure elisp" plugins though. Things where these plugins are doing a lot of string munging, and it's fast in testing because the tests aren't that big. But you can easily hit some edge case where suddenly you're looking for occurrences of `e` in Moby Dick so now each letter press takes a second (fortunately computers are still fast!)
If completion frameworks were async-by-default, at least that stuff wouldn't hang everything for me. At least that's my theory.
Makes sense. Auto complete things are increasingly using language servers, such that I would think they should be in the "communicate with an external process" category. Right?
For pure elisp, the worst I have seen have been around gigantic org buffers. Which, isn't too hard to split up into smaller buffers.
By far the worst case of slowdown is in gigantic log files with absurdly long lines. They made a lot of headway there in detecting the case, as I recall. I also don't think anyone ever said multithreaded would help there.
For pure elisp, the worst I have seen have been around gigantic org buffers. Which, isn't too hard to split up into smaller buffers.
By far the worst case of slowdown is in gigantic log files with absurdly long lines. They made a lot of headway there in detecting the case, as I recall. I also don't think anyone ever said multithreaded would help there.
[deleted]
> The language being integrated into Emacs makes writing Elisp programs a very fluid and intuitive experience: things like the fact that the documentation and the place where a function was defined is available at any time makes /understanding/ the system easy, and the actual documentation itself is often quite well written, and the Info manuals will most often explain everything you need to know about a package. Common Lisp+SLIME shares some of this convenience, but not to the extent that Elisp truly does.
I think that all (or a lot of?) the Emacs niceties could be grafted into a hypothetical EMACS package in Common Lisp. There’s no reason why an Emacs in Common Lisp wouldn’t still be able to jump to a definition, or to documentation, or whatever. It could still implement advice.
And the entire thing would be hackable at runtime, unlike the current Emacs situation where everything but the core runtime is hackable. And there would be packages instead of having to prefix every package’s function with the package name.
Common Lisp is definitely not perfect, but it would probably be a better base for Emacs than Elisp is. Elisp is not bad — at least it’s a Lisp, not Lua or Javascript — but it’s really not great.
I think that all (or a lot of?) the Emacs niceties could be grafted into a hypothetical EMACS package in Common Lisp. There’s no reason why an Emacs in Common Lisp wouldn’t still be able to jump to a definition, or to documentation, or whatever. It could still implement advice.
And the entire thing would be hackable at runtime, unlike the current Emacs situation where everything but the core runtime is hackable. And there would be packages instead of having to prefix every package’s function with the package name.
Common Lisp is definitely not perfect, but it would probably be a better base for Emacs than Elisp is. Elisp is not bad — at least it’s a Lisp, not Lua or Javascript — but it’s really not great.
> There’s no reason why an Emacs in Common Lisp wouldn’t still be able to jump to a definition, or to documentation, or whatever. It could still implement advice.
Sorry if you were confused by my wording; SLIME is able to do all of this--and I am not intending to badmouth SLIME either, it's a very great and well-featured system. However, it plays the odd role of having to bridge the gap between Emacs, and the various CL implementations it's meant to support. Things like documentation of functions really come down to the implementation you're using. The de-facto source of documentation for Common Lisp is the HyperSpec, which is a collection of HTML documents. If it doesn't already exist I would like to write a hack to make these documents render inside Emacs itself.
I think I should clarify the idea that Common Lisp itself is merely a /language/. Unless you're only writing strictly ANSI CL, real programs depend on the facilities provided by the implementation; it isn't a complete programming system until you give it an environment. SBCL+SLIME, or Allegro or LispWorks or a Lisp machine constitute full programming systems in the same way Emacs+Elisp does by itself.
>And the entire thing would be hackable at runtime, unlike the current Emacs situation where everything but the core runtime is hackable.
Ever since Elisp has begun to be compiled into native code, I've become hopeful of the idea that eventually the core components written in C will eventually be subsumed by Elisp itself. However, I am also concerned that this will also degrade the overall performance of the system; not because compiled Elisp wouldn't be performant enough but because hackers might be likely to rewrite these components in a negligent way (the same way they have written modules which are the source of many of the relevant complaints, like LSP and company-mode)--see my other comment in this thread about the internals of the Lisp machine system.
However, having said that, I think a good enough majority of the runtime is hackable that it escapes being a problem. Much of the window system, for example is exposed to Elisp: one of my projects recently has been to write a drop-in replacement for the X window context menus that spawn text frames instead.
>And there would be packages instead of having to prefix every package’s function with the package name.
Packages with namespaces are one thing that would have been nice to have, but it's lack of them is well entrenched enough at this point to remain, and it isn't really a problem once you get used to it.
Sorry if you were confused by my wording; SLIME is able to do all of this--and I am not intending to badmouth SLIME either, it's a very great and well-featured system. However, it plays the odd role of having to bridge the gap between Emacs, and the various CL implementations it's meant to support. Things like documentation of functions really come down to the implementation you're using. The de-facto source of documentation for Common Lisp is the HyperSpec, which is a collection of HTML documents. If it doesn't already exist I would like to write a hack to make these documents render inside Emacs itself.
I think I should clarify the idea that Common Lisp itself is merely a /language/. Unless you're only writing strictly ANSI CL, real programs depend on the facilities provided by the implementation; it isn't a complete programming system until you give it an environment. SBCL+SLIME, or Allegro or LispWorks or a Lisp machine constitute full programming systems in the same way Emacs+Elisp does by itself.
>And the entire thing would be hackable at runtime, unlike the current Emacs situation where everything but the core runtime is hackable.
Ever since Elisp has begun to be compiled into native code, I've become hopeful of the idea that eventually the core components written in C will eventually be subsumed by Elisp itself. However, I am also concerned that this will also degrade the overall performance of the system; not because compiled Elisp wouldn't be performant enough but because hackers might be likely to rewrite these components in a negligent way (the same way they have written modules which are the source of many of the relevant complaints, like LSP and company-mode)--see my other comment in this thread about the internals of the Lisp machine system.
However, having said that, I think a good enough majority of the runtime is hackable that it escapes being a problem. Much of the window system, for example is exposed to Elisp: one of my projects recently has been to write a drop-in replacement for the X window context menus that spawn text frames instead.
>And there would be packages instead of having to prefix every package’s function with the package name.
Packages with namespaces are one thing that would have been nice to have, but it's lack of them is well entrenched enough at this point to remain, and it isn't really a problem once you get used to it.
Posted from eww in emacs as a test :) Yes, I tend to think this is more "It is not like...
> I think that for what it was designed to be (a complete scripting language for a text editor) it does everything that it needs to do very well, and much more.
Performance is something that needs to be done well in this target domain, and it fails
Performance is something that needs to be done well in this target domain, and it fails
emacs lisp has a gccjit backend by default now that is quite performant
About the multithreaded thing: it's not as if everything was single-threaded. There are cases where Emacs shall offload the work to something working in the background. Be it a LSP server or generating thumbnails for a directory with thousands of pictures seen from image-dired: Emacs is totally usable while other processes / async calls are doing their things in the background.
> the base GNU Emacs provides 100000000x more utility in its packages in a fraction of the memory space than VS code or Vim ever will. Emacs has its fair share of killer apps like Magit that provide such a clean interface to something that it makes using it worth it for that alone
And now that Microsoft created LSP (IMO not because they're kind hearted but as a totally dick move to piss off JetBrains who was gaining too much developers momentum and becoming a threat to the "developers, developers, developers" mantra) and that Emacs supports LSP, suddenly those things an IDE did really better than Emacs are much smaller. I'm not saying there aren't a few things an IDE from JetBrains shall be better at than Emacs for a particular language: but on average the gap on those things got way narrower. And Emacs can do a shitload of things an IDE doesn't: configuration/tailoring to your way of working comes to mind. I wrote an IntelliJ plugin back in the days: not fun. I much prefer to write elisp to enhance Emacs.
I was using IntelliJ IDEA back before some commenting here were born: I was an early IntelliJ IDEA adopter (when it was barely usable on Linux btw). But I never stopped using Emacs.
Every other IDE became slower and "piggyer" at a rate outpacing the now dog-slow advances in CPU/RAM on personal computers (I'm running an AMD 7700X atm, so not too bad of a machine). Meanwhile at every single release Emacs became more efficient. The switch to "native complication" was a gigantic speed boost.
As I like to point out: the "Emacs meaning Eight Megabytes And Constantly Swapping" joke was true back in the frigging nineties. Yup, on a 486 with 8 MB of RAM it was quite something to run Emacs.
But nowadays we all have about 16 to 64 GB of RAM on our systems. And Emacs is still Emacs.
When something survives decades and has stuff like Magit and org-mode which many revere, it has to do something right.
If there's a problem with elisp is that it's... Old? But it did survive. It is usable. It's problems and how to work around them are well known.
And it's rock stable. My custom Emacs/elisp code configuration is about 3 000 lines of elisp. I think I made one change/commit in six months to my Emacs configuration. It's that solid.
> At the very least if it all comes down to a dick measuring contest ...
Oh also... At times I have Emacs' uptime surpassing that possible of a Windows consumer OS (which needs to be rebooted regularly due to mandatory updates).
Basically the only time when I reinstall Emacs is when I upgrade to either the new Debian stable distro or when I upgrade to a new PC: when it comes out, I install it (I prefer to install from scratch, YMMV) and then use the opportunity to fetch the latest Emacs and compile it from source. I don't remember the last time I did it: it was when Debian 12 stable came out.
> the base GNU Emacs provides 100000000x more utility in its packages in a fraction of the memory space than VS code or Vim ever will. Emacs has its fair share of killer apps like Magit that provide such a clean interface to something that it makes using it worth it for that alone
And now that Microsoft created LSP (IMO not because they're kind hearted but as a totally dick move to piss off JetBrains who was gaining too much developers momentum and becoming a threat to the "developers, developers, developers" mantra) and that Emacs supports LSP, suddenly those things an IDE did really better than Emacs are much smaller. I'm not saying there aren't a few things an IDE from JetBrains shall be better at than Emacs for a particular language: but on average the gap on those things got way narrower. And Emacs can do a shitload of things an IDE doesn't: configuration/tailoring to your way of working comes to mind. I wrote an IntelliJ plugin back in the days: not fun. I much prefer to write elisp to enhance Emacs.
I was using IntelliJ IDEA back before some commenting here were born: I was an early IntelliJ IDEA adopter (when it was barely usable on Linux btw). But I never stopped using Emacs.
Every other IDE became slower and "piggyer" at a rate outpacing the now dog-slow advances in CPU/RAM on personal computers (I'm running an AMD 7700X atm, so not too bad of a machine). Meanwhile at every single release Emacs became more efficient. The switch to "native complication" was a gigantic speed boost.
As I like to point out: the "Emacs meaning Eight Megabytes And Constantly Swapping" joke was true back in the frigging nineties. Yup, on a 486 with 8 MB of RAM it was quite something to run Emacs.
But nowadays we all have about 16 to 64 GB of RAM on our systems. And Emacs is still Emacs.
When something survives decades and has stuff like Magit and org-mode which many revere, it has to do something right.
If there's a problem with elisp is that it's... Old? But it did survive. It is usable. It's problems and how to work around them are well known.
And it's rock stable. My custom Emacs/elisp code configuration is about 3 000 lines of elisp. I think I made one change/commit in six months to my Emacs configuration. It's that solid.
> At the very least if it all comes down to a dick measuring contest ...
Oh also... At times I have Emacs' uptime surpassing that possible of a Windows consumer OS (which needs to be rebooted regularly due to mandatory updates).
Basically the only time when I reinstall Emacs is when I upgrade to either the new Debian stable distro or when I upgrade to a new PC: when it comes out, I install it (I prefer to install from scratch, YMMV) and then use the opportunity to fetch the latest Emacs and compile it from source. I don't remember the last time I did it: it was when Debian 12 stable came out.
Can't have it both ways.
Elisp is either a scripting language for a text editor,
OR
a fully-featured programming language for doing All The Things.
Elisp is either a scripting language for a text editor,
OR
a fully-featured programming language for doing All The Things.
It's a jack of all trades, master of one.
This wiki page raises some good points, for example, the stateful nature of some of the APIs:
Instead of returning a data structure of the match results (eg from a call to match-string), the match results are mutated in global memory and accessed in separate function calls. This is both less functional in style and more error prone. For example, if save-match-data is not used appropriately, then library functions can trample on the match data which a higher level function is in the middle of using.
EmacsLisp Isn't Scheme
This, from my highly unscientific sample, is far and away the most popular reason EmacsLisp sucks. Well, that’s good to know. EmacsLisp is also not Perl, or COBOL, or IBM 1130 assembler, or a bicycle, or an orange. Thanks for the help!Fairly amusing that just today I was looking at some elisp code I wrote a few years ago. A comment I had inserted:
;; Also, I need the ~save-match-data~ because apparently
;; ~split-string~ messes up the match information. I spent hours
;; debugging this.That’s the charm of EmacsWiki. It really is community-edited!
It used to function as a semi permanent archive of all the babblings on #emacs on freenode. Those were the days...
Reminds me I often wished (and tried~) to make a purely functional regex lib in elisp
The problem with that statement isn't that some APIs use state but that the reason those APIs need state wasn't documented.
This seems to be missing the elephant in the room - elisp doesn't suck, it is just a lisp. It is annoying having to learn a specific lisp just to edit Emacs, but whatever. Other lisps are better if you want to add lisp to your application.
The Emacs data model sucks. There are APIs that do who-knows-what combined with your personal extensions that modify behaviour who-knows-how, figuring out how to describe UI components is no fun, text editing is surprisingly hard from a programmers perspective, there are decades of accumulated different ways of doing things and it is unclear where to start or what has been obsoleted. The names made sense in the 70s but unfortunately it is 2024 and "Windows" means something different now. I wouldn't even exactly blame Emacs for any of this since some of it seems to be essential complexity of the domain.
EDIT Example, picked completely at random:
https://github.com/clojure-emacs/cider/tree/master/dev
EDITEDIT Tramp isn't an extension, it is part of core emacs. We live and learn.
EDITEDITEDIT Spoke too quickly, it looks like it is a GNU-supported extension. This is the sort of trivia that got me using Doom Emacs.
The Emacs data model sucks. There are APIs that do who-knows-what combined with your personal extensions that modify behaviour who-knows-how, figuring out how to describe UI components is no fun, text editing is surprisingly hard from a programmers perspective, there are decades of accumulated different ways of doing things and it is unclear where to start or what has been obsoleted. The names made sense in the 70s but unfortunately it is 2024 and "Windows" means something different now. I wouldn't even exactly blame Emacs for any of this since some of it seems to be essential complexity of the domain.
EDIT Example, picked completely at random:
https://github.com/clojure-emacs/cider/tree/master/dev
(defun cider-tramp-prefix (&optional buffer)
"Use the filename for BUFFER to determine a tramp prefix.
Defaults to the current buffer. Return the tramp prefix, or nil
if BUFFER is local."
(let* ((buffer (or buffer (current-buffer)))
(name (or (buffer-file-name buffer)
(with-current-buffer buffer
default-directory))))
(when (tramp-tramp-file-p name)
(with-parsed-tramp-file-name name v
(with-no-warnings
(cider-make-tramp-prefix v-method v-user v-host v-port))))))
EDITEDIT Tramp isn't an extension, it is part of core emacs. We live and learn.
EDITEDITEDIT Spoke too quickly, it looks like it is a GNU-supported extension. This is the sort of trivia that got me using Doom Emacs.
> Then on top of that I have to go look up what let* is and why it is presumably different from let.
I don't think this is a problem for elisp; isn't `let*` in Common Lisp as well?
I don't think this is a problem for elisp; isn't `let*` in Common Lisp as well?
Your complaints, while being understandable can be addressed with simply reading of the manual and source code. There are various lips out there and as soon as I know Emacs has its own, I would seek the manual for that. There are wealth of documentation on emacs and its ecosystem, and it predates a lot of standards. The terminology can be strange, but you can readily find some glossary.
> Your complaints, while being understandable can be addressed with simply reading of the manual and source code.
I don't think they can, my complaints are that I have to spend too much time reading the manual and source code. Reading the manual and source code will not help resolve that.
I don't think they can, my complaints are that I have to spend too much time reading the manual and source code. Reading the manual and source code will not help resolve that.
> This seems to be missing the elephant in the room - elisp doesn't suck, it is just a lisp.
The article lists a few defaults of elisp that aren't shared by all lisps. What do you mean by this sentence?
The article lists a few defaults of elisp that aren't shared by all lisps. What do you mean by this sentence?
There isn't anything profoundly wrong with Elisp. Emacs needs a programming language, Elisp is a programming language and that is the end of the story, more or less.
You can level complaints against Elisp. You can level complaints against anything. They aren't a big deal - Elisp supports variables, loops and if statements and that is enough to implement a text editor to a first approximation.
If the slate was wiped clean, the Emacs community would probably prefer that Emacs was written with Common Lisp. Then Emacs wouldn't need its own lisp reference manual.
You can level complaints against Elisp. You can level complaints against anything. They aren't a big deal - Elisp supports variables, loops and if statements and that is enough to implement a text editor to a first approximation.
If the slate was wiped clean, the Emacs community would probably prefer that Emacs was written with Common Lisp. Then Emacs wouldn't need its own lisp reference manual.
I can't speak for the Emacs community, but like to caution that Common Lisp is a large language and IMHO would be too much a burden to learn for someone just to customize or extend the editor. I'd rather think it's the Common Lisp community who would prefer Emacs to be written using CL ;-)
Emacs Lisp gets there anyway, somehow. Just decades later.
Examples:
Lexical binding. SCHEME in 1975, Common Lisp in 1984. GNU Emacs in 2012.
Native compiled code. LISP 1 got that in 1960. There are now native compiled code builds of GNU Emacs since a few years.
Cooperative threading. Common Lisp had that in the 80s. Other Lisp dialects probably earlier. Now concurrent native threads would be a thing, so that Emacs Lisp too can take advantage of preemptive single and multi-core threading.
Examples:
Lexical binding. SCHEME in 1975, Common Lisp in 1984. GNU Emacs in 2012.
Native compiled code. LISP 1 got that in 1960. There are now native compiled code builds of GNU Emacs since a few years.
Cooperative threading. Common Lisp had that in the 80s. Other Lisp dialects probably earlier. Now concurrent native threads would be a thing, so that Emacs Lisp too can take advantage of preemptive single and multi-core threading.
But (setq variable var) looks the same in CL and ELisp, so it takes the same amount of learning to customize, right? It is not more to learn just because there exist more possibilities in the language.
Have I reached Lisp enlightenment (TM) if I read that and felt that the code was fine?
> Then on top of that I have to go look up what let* is and why it is presumably different from let. One more thing I don't want to do. Emacs lisp sucks.
Don't modern languages require you to also look up definitions of functions. Learning peculiarities like this in Common Lisp (eg equality) really opened my eyes to how much "magic" there is modern high level languages. Personally I dont like magic and lisps tend to be the most magic free high level languages
Don't modern languages require you to also look up definitions of functions. Learning peculiarities like this in Common Lisp (eg equality) really opened my eyes to how much "magic" there is modern high level languages. Personally I dont like magic and lisps tend to be the most magic free high level languages
Sorry but picking out let* as a problem just shows that you don't know some of the most basic Lisp forms. You say that you don't want to learn a specific Lisp, but starred let is in both Common Lisp and Scheme. So what Lisp have you actually taken the time to learn? The impression you give is precisely ().
> The Emacs data model sucks
It's just one damned anachronism after another.
The language used to manipulate the data model is almost irrelevant. It's probable that the initiates within the priesthood know it all, but the hoi-polloi certainly don't; and we probably don't have the time to find and read all the docs just to use an editor with a simple customization. It's exclusionary and there's an immense barrier to entry, whether it's elisp or bananas-lang.
It's just one damned anachronism after another.
The language used to manipulate the data model is almost irrelevant. It's probable that the initiates within the priesthood know it all, but the hoi-polloi certainly don't; and we probably don't have the time to find and read all the docs just to use an editor with a simple customization. It's exclusionary and there's an immense barrier to entry, whether it's elisp or bananas-lang.
I’m fanatical about having and using the best tools. I take a great deal of pride in it.
I take great care with my VSCode extensions and their configuration. I likewise take great care with neovim and having it set up properly.
Modern Emacs Lisp is easily the best environment among my daily drivers for interacting with my editor in real time.
Of anything modern I only know Zed a little: Zed in theory might be better than Emacs.
Outside of maybe Zed, which I haven’t mastered, Emacs Lisp is bar nothing the best tool-making tool for serious hackers.
I take great care with my VSCode extensions and their configuration. I likewise take great care with neovim and having it set up properly.
Modern Emacs Lisp is easily the best environment among my daily drivers for interacting with my editor in real time.
Of anything modern I only know Zed a little: Zed in theory might be better than Emacs.
Outside of maybe Zed, which I haven’t mastered, Emacs Lisp is bar nothing the best tool-making tool for serious hackers.
Why is zed better ? (Real question I don't know it)
In terms of GUI interaction speed and input latency, Zed imho is better. Just see for yourself the first 2 minutes of their showcase video [1]. They have written their own abstraction layer (GPUI) over Apple's Metal (I'm not sure whether that is portable to Vulkan, DirectX 12 or not). And what is Emacs's most performant rendering backend? PGTK is recently gaining traction.
[1]: https://zed.dev/
[1]: https://zed.dev/
I haven’t tried hard with Zed because last time I took a serious go at it (roughly 4-6 months ago) it didn’t have a great story for remote machines.
If you’re doing production work, sooner or later your laptop won’t have the same GPU as the target. At that point an editor without a remote facility becomes an accessory.
If you’re on a Mac, there is an NVIDIA target these days. You have to be able to edit code next to the card.
If you’re doing production work, sooner or later your laptop won’t have the same GPU as the target. At that point an editor without a remote facility becomes an accessory.
If you’re on a Mac, there is an NVIDIA target these days. You have to be able to edit code next to the card.
You sound like you'd enjoy giving the Helix editor a try. It replaced neovim completely for me.
Helix is really great. It’s so nice to not require screenfuls of configuration. And Emacs Lisp fans will feel right at home when the Scheme plugin system lands
How will emacs fans feel at home, in an editor that has mandatory modal editing?
It sounds like the parent is proposing Helix as an alternative to nvim for modal use cases. I’ll have to check it out.
It's a good editor, if modal is your thing. Requests in the past to add immediate-style editing capabilities have basically stated it won't/can't happen.
I keep coming back to Helix ... but... modal editing.
Nope.
Nope.
Do you have your configuration available somewhere? Would love to see some VSCode inspiration.
I’m actually kind of a newb on sharing dots in the VSCode world, I auth with GitHub to get then on a new machine. Is there a way to publish that?
You can open your settings using cmd/ctrl+comma and there is a button in top right corner that opens the JSON file itself.
That procedure gives this opaque link: https://vscode.dev/editor/profile/github/e5b2f79ed7fbed47d95...
I'm reticent to give you that garbage that I can't diff or know if it's worth anything. Life in the VSCode era.
I'm reticent to give you that garbage that I can't diff or know if it's worth anything. Life in the VSCode era.
Feels odd to actually like elisp. :(
There are some surprises. And I do think common lisp has a lot of better parts. But, at the end of the day I find elisp fine for a scripting like language. And I love how accessible every line of elisp in my machine is.
There are some surprises. And I do think common lisp has a lot of better parts. But, at the end of the day I find elisp fine for a scripting like language. And I love how accessible every line of elisp in my machine is.
I also find a lot of the user code well written. Helps to have various IDE support from the Editor and built-in documentation.
This actually reminds me that one thing that helps a lot, is to actively explore some of the code you are reliant on. I found elisp (and, well, lisp) a lot more intimidating before I started perusing code. The pragmatic approach so much lisp code goes for is refreshing and goes a long way to justify why some choices that may feel odd actually work really really well.
Yeah, that's really good advice.
With regards to lexical scope, yes elisp has that now, but it didn't for so long that a ton of existing code, including the standard library and editor interfaces was written without it, and there is a TON of global state.
> and there is a TON of global state.
And that's usually good, because the whole point of Emacs is to make it easy to inspect and manipulate it. It's one case where information hiding and encapsulation - including lexical scope - get annoying very quickly.
And that's usually good, because the whole point of Emacs is to make it easy to inspect and manipulate it. It's one case where information hiding and encapsulation - including lexical scope - get annoying very quickly.
It would probably require a lot more thought about how to make things so modifiable, yet avoid the global mutable state. Yet I think that is the work that will need to be done sooner or later, step by step, to keep the platform relevant.
I dream of combining wlroots and GNU Guile to make a graphic-first and multi-threaded by default Emacsen that does not aim to be source compatible with Emacs Lisp.
Why wlroots? Isn't it better to target an abstraction api over the various GPU libraries (vulkan, metal etc)?
And I would have asked about Guile too but that is another matter.
I think LEM's approach is better here in utilizing SDL and common lisp. If they write an ELisp interpreter and somehow translate Emacs editor api so plugins would work in LEM too, that would be ideal.
And I would have asked about Guile too but that is another matter.
I think LEM's approach is better here in utilizing SDL and common lisp. If they write an ELisp interpreter and somehow translate Emacs editor api so plugins would work in LEM too, that would be ideal.
Lem is cool and is shipping features fast. We don't have an Elisp interpreter, but we already have a small Magit-like interface: see changes, stash files and hunks (no line precision yet), interactive rebase…
https://lem-project.github.io/usage/usage/
(development notes: https://lisp-journey.gitlab.io/blog/oh-no-i-started-a-magit-...)
Lem has:
- syntax highlighting and language-specific commands (linter etc) for many languages: Common Lisp of course, but also Python, Rust, C, Erlang… JSON, HTML, markdown (including interactive preview in a browser)… a directory mode…
- integrated terminal (libvterm)
- project-related commands, interactive grep, tabs, a side-bar project explorer, keyboard macros, multiple cursors, built-in help (to a certain extent), mouse support, transparent background…
https://lem-project.github.io/usage/usage/
(development notes: https://lisp-journey.gitlab.io/blog/oh-no-i-started-a-magit-...)
Lem has:
- syntax highlighting and language-specific commands (linter etc) for many languages: Common Lisp of course, but also Python, Rust, C, Erlang… JSON, HTML, markdown (including interactive preview in a browser)… a directory mode…
- LSP should just work
- treesitter is a WIP
- integrated terminal (libvterm)
- project-related commands, interactive grep, tabs, a side-bar project explorer, keyboard macros, multiple cursors, built-in help (to a certain extent), mouse support, transparent background…
Interesting. Is Lem the editor you use most for editing code?
For editing text that is not code?
For editing text that is not code?
wlroots because I have little experience with GPU things, and they have already done a lot of the heavy lifting that I would do. I specifically want it to be a window manager, so it can script graphical applications in the same way that GNU Emacs can script text applications.
Regarding Guile - I am familiar with it since I use GNU Guix. I think it is one of the more practical Scheme implementations with its current ecosystem.
Regarding Guile - I am familiar with it since I use GNU Guix. I think it is one of the more practical Scheme implementations with its current ecosystem.
[deleted]
Also on Guile - it is meant to be embedded in C programs (like wlroots). It is one of the best options in that domain of Scheme implementations.
Guile is the GNU extension language. Seems like the best candidate for an emacs replacement.
There was already a project to port emacs to run on guile. Guile supports both emacs lisp and scheme for this reason. The project seems to have run out of steam but it was mostly working at one point AFAIK.
Well, an Emacsen that hopes achieving the same degree of popularity as Emacs should be usable in Windows too. Last I checked, Guile had problems in Windows...
I wouldn’t consider running directly on Windows a hard requirement. You can run it under WSL[1|2] for instance, or bundle a very minimal WSL distribution that only runs your software.
It would also simplify the codebase, to scrap the native Windows support and outsource that responsibility to WSL.
Many have tried, but few have succeeded at creating an Emacs-replacement.
Good luck!
Good luck!
I would say, that with Enrich Gamma at the steering wheel, and Microsoft's money, VSCode is succeeding pretty well.
I'd think that VSCode eats into the 'market share' of Eclipse and the like, but I can't see it as replacement for Emacs at all.
Better look at the shape of Emacs market share evolution since VSCode exists.
Turns out JavaScript is more appealing than Elisp, plus the graphical capabilities of having a browser engine.
Turns out JavaScript is more appealing than Elisp, plus the graphical capabilities of having a browser engine.
VS Code is nothing like emacs at all except in the most basic sense that they’re both text editors popular with programmers.
I am old enough to have used XEmacs when the paint was still drying.
On similar lines I've wondered about reimplementing the C parts of Emacs in Common Lisp. At that point Elisp would be a hosted language, letting you benefit from all the existing packages.
Lem is an Emacs using CL (also instead of Emacs Lisp): https://lem-project.github.io/
That was the goal of CEDAR: https://gitlab.com/sasanidas/cedar (staled, too big of a project). But yeah, Lem!
I'm actually writing some Emacs Lisp right now, because I want $FEATURE in Emacs. I've been away from Emacs Lisp for awhile in the land of Clojure and I'm reimporting certain things the way they are done there.
Libraries that help with that, even though the major underlying structure isn't immutable data structures make Emacs Lisp bearable are: s.el, ht.el, dash.el It would be great if there was namespacing and less global state pollution.
Of course if you want to integrate with the rest of the Emacs ecosystem and you want some libraries that help you out make things nice means learning a whole different ballgame. Like wanting to use the great transient.el. That's super stateful and based on eieio.el. (Although transient.el is relatively good about hiding that behind a good API surface.)
Overall, even though I wish it was based on a functional core and Clojure's immutable data structure, concurrency, namespacing and its great EDN reader, I just accept the pragmatism on the human scale that if I want a great text based interface that interacts with the rest of my text driven workflow Emacs is a good choice and Emacs Lisp is acceptable.
Libraries that help with that, even though the major underlying structure isn't immutable data structures make Emacs Lisp bearable are: s.el, ht.el, dash.el It would be great if there was namespacing and less global state pollution.
Of course if you want to integrate with the rest of the Emacs ecosystem and you want some libraries that help you out make things nice means learning a whole different ballgame. Like wanting to use the great transient.el. That's super stateful and based on eieio.el. (Although transient.el is relatively good about hiding that behind a good API surface.)
Overall, even though I wish it was based on a functional core and Clojure's immutable data structure, concurrency, namespacing and its great EDN reader, I just accept the pragmatism on the human scale that if I want a great text based interface that interacts with the rest of my text driven workflow Emacs is a good choice and Emacs Lisp is acceptable.
Its regex dialect is horrendous because of the sheer number of additional backslashes you end up having to use in string literals compared to other languages, e.g. "\\(a\\|b\\)" vs "(a|b)".
At least there's the "rx" DSL as an alternative.
At least there's the "rx" DSL as an alternative.
Exactly! When I saw that rx exists, I was delighted, that I don't actually have to write terrible regexes. Something like rx should exist in any serious lisp imo. Well, one should avoid regexes as much as possible, of course, but when you have to use them, rx is a great tool to have!
Personally I don't really have a problem with 'normal' regex. Learned it to a decent level, have gotten many years of value from that, reading and writing, generally no issues.
I don't like Emacs Lisp a whole lot. It's probably my second least favorite Lisp to interact with. (Least favorite is Autolisp.) But it's a Lisp, it's gotten much better over the years, and it has lots of features that make automating what Emacs does really stinking convenient. Automating a task to smooth out a workflow usually takes maybe a page or two of Emacs Lisp, tops. And I can piece it together and try it out, live, in the editor session I'm doing my work in. Visual Studio Code has nothing comparable.
> But I believe there is a difference between making Emacs read mail or news or have a shell because you needed to in the 1970s and making Emacs browse web sites or act as an httpd because you can in the 2000s. I wonder if perhaps the people who want to fix all of EmacsLisp’s problems want to do so more for their own benefit – so it will be a better general-purpose development base and toolkit – than to make Emacs better for everyone. I wonder if they have lost sight of the primary application: the editor[5].
I actually get a lot of utility out of eww, it's not just a novelty for me. I can browse online documentation without a DM, or if I do have a DM, without moving my hand to the mouse. It's not just a novelty.
It is getting increasingly difficult to use; much like emacs the web has also transitioned from its original intention (hypertext documents) to a bloated app development platform and it's increasingly unfriendly and difficult to use with simple browsers. Even so, there are still websites worth visiting that are usable from trivial browsers as long as you avoid the many apps masquerading as webpages.
I actually get a lot of utility out of eww, it's not just a novelty for me. I can browse online documentation without a DM, or if I do have a DM, without moving my hand to the mouse. It's not just a novelty.
It is getting increasingly difficult to use; much like emacs the web has also transitioned from its original intention (hypertext documents) to a bloated app development platform and it's increasingly unfriendly and difficult to use with simple browsers. Even so, there are still websites worth visiting that are usable from trivial browsers as long as you avoid the many apps masquerading as webpages.
> I actually get a lot of utility out of eww
I do too. And the underlying shr rendering library is invaluable when so much of email traffic is text/html. A bunch of other stuff like mastodon.el and devdoc.el rely on it too.
> It is getting increasingly difficult to use;
In my experience it's not that much worse than it was 10 years ago. By the time eww came about the JS-less browsing ship has long sailed. The sort of stuff that was working back then still tends to work now. We lost some good sites like the BBC text-only mode but also gained some nice text proxies like 68k.news and brutalist.report.
I do too. And the underlying shr rendering library is invaluable when so much of email traffic is text/html. A bunch of other stuff like mastodon.el and devdoc.el rely on it too.
> It is getting increasingly difficult to use;
In my experience it's not that much worse than it was 10 years ago. By the time eww came about the JS-less browsing ship has long sailed. The sort of stuff that was working back then still tends to work now. We lost some good sites like the BBC text-only mode but also gained some nice text proxies like 68k.news and brutalist.report.
> but the former indicates a stigmatization of side-effecting in the Scheme community
Such poppycock. Clearly marking functions with side effects as such is not stigmatization, it's common sense. Don't take the sloppiness of other language cultures for the norm, although statistically it may well be. Too many people already think procedural object soup is how things should be.
Such poppycock. Clearly marking functions with side effects as such is not stigmatization, it's common sense. Don't take the sloppiness of other language cultures for the norm, although statistically it may well be. Too many people already think procedural object soup is how things should be.
So far as “APIs suck” I think about Guile scripting in the GIMP. I wouldn’t say “it sucks” but Guile seems nonsuperior to Lua, Visual Basic for Applications, and other languages built for embeddable scripting.
Is a MOP (like CLOS) going to bring substantial advantage to extensibility? My intuition says yes, but I can't test this hypothesis without thousands of plugin developers. Has any (meta) study even been done?
I think Elisp already implements a lot of things that CLOS would give you, like around method combinators (with defadvice, if I remember correctly). It's probably more ad-hoc, and you might want some extra customizations on top, but I'm not sure.
Elisp already has a (partial, but for most purposes complete) implementation of CLOS as part of the cl-lib package.
The only reason I immagine people might not be happy with elisp is when they reach a point and wish their emacs was their operating system - ie they wish elisp was a systems language - and dont want to learn C
I would think that people might wish Emacs Lisp would be a more capable application programming language. For example for an editor with such extensive capabilities, it would be useful that the language would provide preemptive scheduled threads and related constructs.
On my old Lisp Machine, Zmacs ran in a multi-threaded Lisp roughly in the early 80s. Multiple Zmacs windows each had their own thread. Applications had their own thread. GC has its own threads. Even the mouse has its own thread. It still works that way.
But you don't need a Lisp Machine OS with threaded Lisp for that. Many current implementations of Lisp support multi-threading. An example is SBCL, a natively compiled Common Lisp, which supports native threads on its platforms. Thus an editor written on top of SBCL could make use of that. IDEs of LispWorks and Allegro CL are also multi-threaded and they have their own implementations of Emacs, written in Common Lisp.
Compare with GNU Emacs, where Emacs Lisp lacks this feature. Wouldn't you want an mail thread not block the rest of Lisp code? Some Lisp action in the REPL not to block the rest of Lisp code?
Multi-threaded Lisp architectures are more complex. Emacs Lisp tried to be simple. But the stuff written in it long ago was outgrowing the provided simplicity. For a capable extension language it was "good enough" for a long time. For an "application development platform" it's not. Lack of multi-threading in Emacs Lisp and the GNU Emacs architecture is an example.
On my old Lisp Machine, Zmacs ran in a multi-threaded Lisp roughly in the early 80s. Multiple Zmacs windows each had their own thread. Applications had their own thread. GC has its own threads. Even the mouse has its own thread. It still works that way.
But you don't need a Lisp Machine OS with threaded Lisp for that. Many current implementations of Lisp support multi-threading. An example is SBCL, a natively compiled Common Lisp, which supports native threads on its platforms. Thus an editor written on top of SBCL could make use of that. IDEs of LispWorks and Allegro CL are also multi-threaded and they have their own implementations of Emacs, written in Common Lisp.
Compare with GNU Emacs, where Emacs Lisp lacks this feature. Wouldn't you want an mail thread not block the rest of Lisp code? Some Lisp action in the REPL not to block the rest of Lisp code?
Multi-threaded Lisp architectures are more complex. Emacs Lisp tried to be simple. But the stuff written in it long ago was outgrowing the provided simplicity. For a capable extension language it was "good enough" for a long time. For an "application development platform" it's not. Lack of multi-threading in Emacs Lisp and the GNU Emacs architecture is an example.
>On Lisp Machine, Zmacs ran in a multi-threaded Lisp
I wonder what the processing architecture of those Lisp machines were like. Did their "cpu"(?) had special circuitry for implementing lists, CAR and CDR? Was this multi-threaded-ness something built-in in hardware?
I wonder what the processing architecture of those Lisp machines were like. Did their "cpu"(?) had special circuitry for implementing lists, CAR and CDR? Was this multi-threaded-ness something built-in in hardware?
There is nothing special about CAR and CDR on the Lisp Machine, which should be considered a virtual machine for all intents and purposes that was implemented in microcode.
Multi processing was implemented mostly in Lisp, but stack group handling (how the system manages "threads") was done in microcode.
The hardware for the CADR is very simple, and has no knowledge of Lisp or the Lisp Machine.
Multi processing was implemented mostly in Lisp, but stack group handling (how the system manages "threads") was done in microcode.
The hardware for the CADR is very simple, and has no knowledge of Lisp or the Lisp Machine.
> Did their "cpu"(?) had special circuitry for implementing lists, CAR and CDR?
It was implemented in Microcode on the CPU.
> Was this multi-threaded-ness something built-in in hardware?
Some. The process scheduler was written in Lisp. See for an early example on the MIT Lisp Machine: https://github.com/mietek/mit-cadr-system-software/blob/mast...
It was implemented in Microcode on the CPU.
> Was this multi-threaded-ness something built-in in hardware?
Some. The process scheduler was written in Lisp. See for an early example on the MIT Lisp Machine: https://github.com/mietek/mit-cadr-system-software/blob/mast...
I think you should clarify that the Lisp machines (at least the early ones that you speak of) were not 'multi-threaded' in the sense of multiprocessing--the only multiprocessing they did was running independent Lisp systems on each of their processors, if they had more than one.
From the 1981 4th edition Lisp Machine Manual, page 428ff
> 25. Processes
> The Lisp Machine supports multi-processing; several computations can be executed "concurrently" by placing each in a separate process. A process is like a processor, simulated by software. Each process has its own "program counter", its own stack of function calls and its own special-variable binding environment in which to execute its computation. (This is implemented with stack groups, see chapter 12, page 163.)
These computers could send/receive network traffic, handle the GUI, read from disk, while the user would have processes for each application and window.
On my mid 1980s Symbolics 3640 I could compile a Lisp application from scratch in a Lisp REPL for half an hour, while I was using Zmacs to edit code. The practical limitations were CPU speed, memory size, disk speed and the Garbage collector. A full GC would halt everything else. Better incremental GCs helped a lot.
In GNU Emacs last I tried I could not use two IELM repls executing Lisp code at the same time. One would block the rest of GNU Emacs Lisp code.
> 25. Processes
> The Lisp Machine supports multi-processing; several computations can be executed "concurrently" by placing each in a separate process. A process is like a processor, simulated by software. Each process has its own "program counter", its own stack of function calls and its own special-variable binding environment in which to execute its computation. (This is implemented with stack groups, see chapter 12, page 163.)
These computers could send/receive network traffic, handle the GUI, read from disk, while the user would have processes for each application and window.
On my mid 1980s Symbolics 3640 I could compile a Lisp application from scratch in a Lisp REPL for half an hour, while I was using Zmacs to edit code. The practical limitations were CPU speed, memory size, disk speed and the Garbage collector. A full GC would halt everything else. Better incremental GCs helped a lot.
In GNU Emacs last I tried I could not use two IELM repls executing Lisp code at the same time. One would block the rest of GNU Emacs Lisp code.
There is a bit to much simplification here, and confusing terminology going on here. Lisp Machines did not have "processes" (as in, Unix processes); a Lisp Machine process just a function - with some special properties.
The Lisp Machine only had one processor for the Lisp Machine, and it did absolutely support multi-processing and multi-threading.
A good explanation can be found in the Lisp Machine manual:
The Lisp Machine only had one processor for the Lisp Machine, and it did absolutely support multi-processing and multi-threading.
A good explanation can be found in the Lisp Machine manual:
https://tumbleweed.nu/r/lm-3/uv/chinual.html#Processes
https://tumbleweed.nu/r/lm-3/uv/chinual.html#Stack-Groups
All current work ?
> Compare with GNU Emacs, where Emacs Lisp lacks this feature. Wouldn't you want an mail thread not block the rest of Lisp code? Some Lisp action in the REPL not to block the rest of Lisp code?
Emacs Lisp does have threads; https://www.gnu.org/software/emacs/manual/html_node/elisp/Th...
Emacs Lisp does have threads; https://www.gnu.org/software/emacs/manual/html_node/elisp/Th...
Even more simply, `while-no-input` has existed since 2007. And start-process since 1985. It has always been possible to make a mail "thread".
If the developer didn't use the old tricks, it doesn't seem likely they will use the thread trick either.
If the developer didn't use the old tricks, it doesn't seem likely they will use the thread trick either.
ZMail on the Lisp Machine would stall the ZMail thread when fetching mail. So you wouldn't be able to compose an email if you'd be fetching 10MiB of mbox.
But it isn't so much about tricks, sometimes it is nice if software just works reliably, GNU Emacs has a great track record on that.
But it isn't so much about tricks, sometimes it is nice if software just works reliably, GNU Emacs has a great track record on that.
These "tricks" still don't make two GNU Emacs Lisp REPLs in the same GNU Emacs execute arbitrary Lisp code concurrently.
But why does that matter? Help me understand.
To be fair I haven't used an elisp REPL, unless it counts that I use commands. Any command used in Emacs is an eval step in a REPL.
But you mean a REPL as in a console-style interaction, right? It seems an impoverished interaction model, only necessary with other languages since they are not also the editor process.
Ok I can't have multiple REPLs but neither can I type on multiple keyboards at the same time. So it seems to only matter when you are going to spin up some noticeable chunk of work to be done in the background, while the user interacts with the editor. And that is totally doable with `while-no-input`.
In what situation are you running two or more noticeably slow call stacks that need to happen inside the editor process?
To be fair I haven't used an elisp REPL, unless it counts that I use commands. Any command used in Emacs is an eval step in a REPL.
But you mean a REPL as in a console-style interaction, right? It seems an impoverished interaction model, only necessary with other languages since they are not also the editor process.
Ok I can't have multiple REPLs but neither can I type on multiple keyboards at the same time. So it seems to only matter when you are going to spin up some noticeable chunk of work to be done in the background, while the user interacts with the editor. And that is totally doable with `while-no-input`.
In what situation are you running two or more noticeably slow call stacks that need to happen inside the editor process?
> But why does that matter? Help me understand.
It doesn't really.
ZMacs and the Listener on the Lisp Machine is just as single threaded as GNU Emacs. One ZMacs frame cannot run the above code without locking up, one Listener instance cannot run the above code without locking up, if you open a file in ZMacs it will lock up while reading up the file.
The only difference is that when the ZMacs background process is working, you can still talk to the Lisp Machine in other ways.
The closest you might get with GNU Emacs is running multiple versions, one for each "program" (Emacs, mail, web browser, ...) -- still not exactly the same thing since you can't easily poke into the mail program from another instance.
At the end of the day .. none of this matters much, as we can evidently see since nobody has really tried to fix this.
It doesn't really.
ZMacs and the Listener on the Lisp Machine is just as single threaded as GNU Emacs. One ZMacs frame cannot run the above code without locking up, one Listener instance cannot run the above code without locking up, if you open a file in ZMacs it will lock up while reading up the file.
The only difference is that when the ZMacs background process is working, you can still talk to the Lisp Machine in other ways.
The closest you might get with GNU Emacs is running multiple versions, one for each "program" (Emacs, mail, web browser, ...) -- still not exactly the same thing since you can't easily poke into the mail program from another instance.
At the end of the day .. none of this matters much, as we can evidently see since nobody has really tried to fix this.
I think Emacs' process model is generic enough that it could adopt a better method of concurrency in the future. Functions for processes are run based on 'sentinels' which fire on interrupts from foreign processes. It's a simple model which works fine, and I would greatly prefer it to be augmented with a more clever scheduler than to dragging mutexes and condition variables into the system.
> At the end of the day .. none of this matters much, as we can evidently see since nobody has really tried to fix this.
Something that's easy to take for granted is how fast Emacs actually is. Anyone who's taken the time to try out an emulated Lisp machine will quickly observe how slow and clunky it actually is most of the time. And it's worth mentioning that a lot of the crucial components of the Lisp machine system, such as the windowing system, the garbage collector and the network stack work because they are written in a more structured and low-level manner so that they run efficiently and avoid consing. It is all Lisp, sure; but there's a bit of a bias that seems to portray that its all 'plain lisp' that 'just works' concurrently because it's magic.
> At the end of the day .. none of this matters much, as we can evidently see since nobody has really tried to fix this.
Something that's easy to take for granted is how fast Emacs actually is. Anyone who's taken the time to try out an emulated Lisp machine will quickly observe how slow and clunky it actually is most of the time. And it's worth mentioning that a lot of the crucial components of the Lisp machine system, such as the windowing system, the garbage collector and the network stack work because they are written in a more structured and low-level manner so that they run efficiently and avoid consing. It is all Lisp, sure; but there's a bit of a bias that seems to portray that its all 'plain lisp' that 'just works' concurrently because it's magic.
The VLM I'm using has the CPU emulated in assembler. Runs okay on Apple Silicon. It uses one core.
> And it's worth mentioning that a lot of the crucial components of the Lisp machine system, such as the windowing system, the garbage collector and the network stack work because they are written in a more structured and low-level manner so that they run efficiently and avoid consing.
Not all levels are low-level, there is a large higher level which is written using Flavors. Even transmitting IP packets is a Flavor method.
> And it's worth mentioning that a lot of the crucial components of the Lisp machine system, such as the windowing system, the garbage collector and the network stack work because they are written in a more structured and low-level manner so that they run efficiently and avoid consing.
Not all levels are low-level, there is a large higher level which is written using Flavors. Even transmitting IP packets is a Flavor method.
On my Symbolics VLM installation (on a MacBook Pro) I can run things in one Listener, let it running its code and switch to the next listener. I can also use the GUI to create a new Listener. I can switch back to the busy Listener and force it into a break by control-suspend. I can also reshape a busy Listener window.
Something like the LispWorks IDE, with its editor-based listener tool, does the same.
Something like the LispWorks IDE, with its editor-based listener tool, does the same.
People seem to use Lisp like it is some impoverished programming model. Why would I want to do things concurrently in one Lisp? Because that's how I use a Lisp system all the time. That's my normal way to do things.
> But you mean a REPL as in a console-style interaction, right?
I mean > 1 REPL as an example how I would use a Lisp concurrently. Yeah, I can switch between REPLs in micro seconds and run > 1 second activities in each of them.
> Any command used in Emacs is an eval step in a REPL
And you can't run simple Lisp code in two commands concurrently, neither in a REPL nor in a command.
> `while-no-input`
Type (while-no-input (dotimes (i 1000000) (print i))) in two Emacs Lisp repls and have them running concurrently.
I understand that GNU Emacs can run external processes, external instances of GNU Emacs, external other Lisp systems, that it can provide some limited form of cooperative threads, ...
I have lived through this stuff 30 years ago and onwards, when many Lisp implementations used cooperative threading (and some had cooperative threads deeply integrated in their Lisp code, other than GNU Emacs in 2024) and over the years switched to native preemptive threading. I also remember when we got Symmetric Multiprocessing in several Lisp systems roughly 15 years ago.
> But you mean a REPL as in a console-style interaction, right?
I mean > 1 REPL as an example how I would use a Lisp concurrently. Yeah, I can switch between REPLs in micro seconds and run > 1 second activities in each of them.
> Any command used in Emacs is an eval step in a REPL
And you can't run simple Lisp code in two commands concurrently, neither in a REPL nor in a command.
> `while-no-input`
Type (while-no-input (dotimes (i 1000000) (print i))) in two Emacs Lisp repls and have them running concurrently.
I understand that GNU Emacs can run external processes, external instances of GNU Emacs, external other Lisp systems, that it can provide some limited form of cooperative threads, ...
I have lived through this stuff 30 years ago and onwards, when many Lisp implementations used cooperative threading (and some had cooperative threads deeply integrated in their Lisp code, other than GNU Emacs in 2024) and over the years switched to native preemptive threading. I also remember when we got Symmetric Multiprocessing in several Lisp systems roughly 15 years ago.
That'd be better as
EDIT: OK, you explained what you call the normal way to do things. What kind of code is this that take >1 second in each repl? Out of curosity.
(defvar foo-i 0)
(defun foo ()
(while-no-input
(while (< foo-i 10000)
(print (cl-incf foo-i))))
(when (< foo-i 10000)
(run-with-idle-timer 1 nil #'foo)))
EDIT: OK, you explained what you call the normal way to do things. What kind of code is this that take >1 second in each repl? Out of curosity.
> Both loops will complete eventually.
Thanks for your effort!
But I need none of this. I want to open a REPL and type (dotimes (i 1000000) (print i)) - or anything morally equivalent ;-) - and I want it to do its job and the user interface to be responsive!! There were Lisp systems, which did this 40 years ago and there are some which can do it today: Execute simple plain Lisp code concurrently without any special instrumentation. Each REPL executes by default in its own thread, concurrently. I want the same for the other tools, too.
Execute (dotimes (i 1000000) (print i)) in the Emacs Lisp REPL. The UI is dead then. I find it strange that it is dead and I find it strange that there are people who don't find that strange. ;-)
> What kind of code is this that take >1 second in each repl? Out of curosity.
All kinds of stuff.
> Out of curosity.
Have you ever used a multithreaded Lisp environment as a primary work tool?
Thanks for your effort!
But I need none of this. I want to open a REPL and type (dotimes (i 1000000) (print i)) - or anything morally equivalent ;-) - and I want it to do its job and the user interface to be responsive!! There were Lisp systems, which did this 40 years ago and there are some which can do it today: Execute simple plain Lisp code concurrently without any special instrumentation. Each REPL executes by default in its own thread, concurrently. I want the same for the other tools, too.
Execute (dotimes (i 1000000) (print i)) in the Emacs Lisp REPL. The UI is dead then. I find it strange that it is dead and I find it strange that there are people who don't find that strange. ;-)
> What kind of code is this that take >1 second in each repl? Out of curosity.
All kinds of stuff.
> Out of curosity.
Have you ever used a multithreaded Lisp environment as a primary work tool?
> EDIT: OK, you explained what you call the normal way to do things. What kind of code is this that take >1 second in each repl? Out of curosity.
One easy example -- listing all the files under the GNU Emacs tree to index them -- you might not want to do it many times but even `find` takes a bit of time. Looping over such a list might also take a bit of time.
One easy example -- listing all the files under the GNU Emacs tree to index them -- you might not want to do it many times but even `find` takes a bit of time. Looping over such a list might also take a bit of time.
>sbcl
SBCL (and it's one of the fastest Common Lisps out there) ran MCCLIM at glacial speeds in my n270 netbook, kinda like a busy Pentium MMX running Windows 98 where you could see Win32 controls redrawing themselves. OTOH, GNU Emacs' multithreading sucks, but at least the UI it's miles ahead on responsiveness.
SBCL (and it's one of the fastest Common Lisps out there) ran MCCLIM at glacial speeds in my n270 netbook, kinda like a busy Pentium MMX running Windows 98 where you could see Win32 controls redrawing themselves. OTOH, GNU Emacs' multithreading sucks, but at least the UI it's miles ahead on responsiveness.
> but at least the UI it's miles ahead on responsiveness.
One Lisp REPL doing stuff and GNU Emacs is completely dead as long as the Lisp REPL runs that Emacs Lisp code. It's not even slow, it just is completely non-responsive.
One Lisp REPL doing stuff and GNU Emacs is completely dead as long as the Lisp REPL runs that Emacs Lisp code. It's not even slow, it just is completely non-responsive.
This seems a bug in the design of that REPL. Unless there's some reason the process has to be the same as the editor process.
I forgot where in the manual they explained that elisp was mostly influenced by maclisp (thus earlier than CL). Makes me curious how maclisp systems were designed now..
I never used MAC Lisp, but I read the documentation for it. CL is more similar to it than ELisp. It also didn't come particularly later. It's common to associate the date of ANSI CL standard publication with the "birth" of the language, however, the standard in this case was the attempt to unify and codify the existing language that was implemented in slightly different ways by various groups. You could even think about it as incorporating MAC Lisp as one of the components that went into standardization. Eg. the loop macro in CL is almost exact copy, whereas ELisp only has it in cl package. There are few more things like that.
If memory serves, the idea for ELisp was to eventually become the "proper" Lisp for system programming. At the time, people used the word "Lisp" very liberally, in a way how we today might use "high-level programming language". And, in a way, Guile was supposed to become that, but as is the case with some such initiatives (eg. Hurd) it never gathered enough momentum to replace the predecessor that was "good enough" and was continuously kept afloat with more and more upgrades.
As for the larger set of complaints about ELisp... it was good for a very long time, but it's getting old. There are many programming practices that are difficult to incorporate into the language. There are what we collectively came to believe to be programming anti-patterns that are encoded into the core of the language. And it's hard to do a proper face-lift because it will inevitably break a lot of old code. Also, doing such a face-lift you'd always be perplexed by the idea of what if a few years from now you could do an even better job of rewriting it into something even better? Knowing that one such change will put a ring on you, perhaps for the rest of your life, it's hard to make the step. Also, there are and have been many attempts of moving away from ELisp, and none really gained much traction among Emacs users. So, it's hard to imagine that the next such attempt will succeed.
My fear in this regard is that one day, instead of evolving and replacing with something better, we'll be reduced to using junk like VSCode due to some underlying system aging out completely. Similar to what happened to Firefox when it lost its ability to house extensions. I pray this day never comes, but, in practical terms, it means that at some point the community of ELisp users needs to cut losses and embrace the loss of older libraries to move to a revamped ELisp2 or w/e it will be called.
If memory serves, the idea for ELisp was to eventually become the "proper" Lisp for system programming. At the time, people used the word "Lisp" very liberally, in a way how we today might use "high-level programming language". And, in a way, Guile was supposed to become that, but as is the case with some such initiatives (eg. Hurd) it never gathered enough momentum to replace the predecessor that was "good enough" and was continuously kept afloat with more and more upgrades.
As for the larger set of complaints about ELisp... it was good for a very long time, but it's getting old. There are many programming practices that are difficult to incorporate into the language. There are what we collectively came to believe to be programming anti-patterns that are encoded into the core of the language. And it's hard to do a proper face-lift because it will inevitably break a lot of old code. Also, doing such a face-lift you'd always be perplexed by the idea of what if a few years from now you could do an even better job of rewriting it into something even better? Knowing that one such change will put a ring on you, perhaps for the rest of your life, it's hard to make the step. Also, there are and have been many attempts of moving away from ELisp, and none really gained much traction among Emacs users. So, it's hard to imagine that the next such attempt will succeed.
My fear in this regard is that one day, instead of evolving and replacing with something better, we'll be reduced to using junk like VSCode due to some underlying system aging out completely. Similar to what happened to Firefox when it lost its ability to house extensions. I pray this day never comes, but, in practical terms, it means that at some point the community of ELisp users needs to cut losses and embrace the loss of older libraries to move to a revamped ELisp2 or w/e it will be called.
[deleted]
I know enough Elisp to assign a key combo to a kbd-macro fwiw
I have hacked a considerable amount in both Elisp and CL and I can state that I actually prefer the way Elisp does some things over the way Common Lisp does--this is not to say that CL does things the wrong way--the matter is that they have different purposes. Emacs combined with Elisp in itself constitutes a complete programming system; all of its components are designed around it.
The language being integrated into Emacs makes writing Elisp programs a very fluid and intuitive experience: things like the fact that the documentation and the place where a function was defined is available at any time makes /understanding/ the system easy, and the actual documentation itself is often quite well written, and the Info manuals will most often explain everything you need to know about a package. Common Lisp+SLIME shares some of this convenience, but not to the extent that Elisp truly does.
Certain points like the fact that Emacs isn't multithreaded are thrown around by people who don't have the intuition that multithreading isn't the right thing for a lot of applications. The added complexity that multithreading would add to Emacs would seriously outweigh the usefulness it would provide. Emacs already has a decent process model, and having to deal with only a single shared state makes programs much cleaner.
At the very least if it all comes down to a dick measuring contest, the base GNU Emacs provides 100000000x more utility in its packages in a fraction of the memory space than VS code or Vim ever will. Emacs has its fair share of killer apps like Magit that provide such a clean interface to something that it makes using it worth it for that alone.