Writing your own BEAM(martin.janiczek.cz)
martin.janiczek.cz
Writing your own BEAM
https://martin.janiczek.cz/2025/11/09/writing-your-own-beam.html
92 comments
I like to think the B stands for BEAM.
It obviously stands for "Benoit B Mandelbrot"
Very cliché.
Hey it’s a classic for a reason.
I, for one, appreciate the classic sensibilities.
Same here! I also don't want "the facts to come in the way of a good story", and want to believe B is BEAM.
Sounds like it’s time for your nap.
The BEAM is fascinating for many reasons, including being register-based.
I really just wish the BEAM was portable in the way the JVM is. The BEAM hooks into so many system libraries, you must compile it on every flavor of linux instead of just unpacking a tarball.
This means you either must use your distro package manager's version, or compile from scratch. If you want to control the exact version that's being used across your team (via `asdf` or similar), this practically means you'll end up compiling the BEAM over and over...
I really just wish the BEAM was portable in the way the JVM is. The BEAM hooks into so many system libraries, you must compile it on every flavor of linux instead of just unpacking a tarball.
This means you either must use your distro package manager's version, or compile from scratch. If you want to control the exact version that's being used across your team (via `asdf` or similar), this practically means you'll end up compiling the BEAM over and over...
This is very neat, thanks for the link!
Digging into it, you can get "universal" BEAM tarballs from here[1]. It links against muslibc and appears to bring it's own openssl. Very cool.
[1] https://beammachine.cloud/
Digging into it, you can get "universal" BEAM tarballs from here[1]. It links against muslibc and appears to bring it's own openssl. Very cool.
[1] https://beammachine.cloud/
Burrito works very well in my experience. I've used it for distributing an implementation of breakout in Elixir with OpenGL and Metal rendering backends as a binary. Pretty neat!
> The BEAM is fascinating for many reasons, including being register-based.
> I really just wish the BEAM was portable in the way the JVM is.
Inferno is both register based and highly portable using the same tool chain as Plan 9 which runs seamlessly across multiple architectures. This eventually evolved into the Go tooling as Rob Pike came up with the Plan 9 design and worked on it with Ken Thompson. https://seh.dev/go-legacy/
Unfortunately Inferno was never fully completed and bit-rotted a bit but it still builds on systems with 32 bit support. There are various forks and even an attempted 64 bit version. To me its a great design as it not only runs on bare metal but also has a hosted option so it runs under Plan 9, Windows, MacOS/Unix/BSD/Linux. Talk about a portable OS...
Inferno is both register based and highly portable using the same tool chain as Plan 9 which runs seamlessly across multiple architectures. This eventually evolved into the Go tooling as Rob Pike came up with the Plan 9 design and worked on it with Ken Thompson. https://seh.dev/go-legacy/
Unfortunately Inferno was never fully completed and bit-rotted a bit but it still builds on systems with 32 bit support. There are various forks and even an attempted 64 bit version. To me its a great design as it not only runs on bare metal but also has a hosted option so it runs under Plan 9, Windows, MacOS/Unix/BSD/Linux. Talk about a portable OS...
To be clear, Inferno is the operating system. The virtual machine is Dis. There exist a few examples[1] of 64-bit Dis, but usually the problem people run into is making the Limbo[2] compiler spit out 64-bit code.
It's rather unfortunate, Inferno is a really nice system with a lot of interesting reference stuff implemented like a typed shell[3]. It's also Plan 9 in a box with all the accoutrements, and knowers will have jumped out of their seat at the implication. I can't understate how cool the Inferno system is.
[1] - https://github.com/Plan9-Archive/9ferno
[2] - https://en.wikipedia.org/wiki/Limbo_(programming_language)
[3] - https://inferno-os.org/inferno/man/2/alphabet-intro.html
It's rather unfortunate, Inferno is a really nice system with a lot of interesting reference stuff implemented like a typed shell[3]. It's also Plan 9 in a box with all the accoutrements, and knowers will have jumped out of their seat at the implication. I can't understate how cool the Inferno system is.
[1] - https://github.com/Plan9-Archive/9ferno
[2] - https://en.wikipedia.org/wiki/Limbo_(programming_language)
[3] - https://inferno-os.org/inferno/man/2/alphabet-intro.html
I dunno what happened to the 64bit port, the author just up and deleted it with no reason given AFIK. Thankfully someone had a recent clone and uploaded it to the archive.
Limbo is incomplete, forget what exactly but a few people on the 9fans discord recently ran into issues. Always remember, Inferno was forced onto the Labs people by Lucent in an attempt to compete with Sun's Java and built in a year. Bell Labs did not do marketing and Lucent fumbled the whole thing.
Plan 9 in a box is only useful if you MUST use a foreign host like Linux. Otherwise I would much prefer the machine actually run Plan 9.
Limbo is incomplete, forget what exactly but a few people on the 9fans discord recently ran into issues. Always remember, Inferno was forced onto the Labs people by Lucent in an attempt to compete with Sun's Java and built in a year. Bell Labs did not do marketing and Lucent fumbled the whole thing.
Plan 9 in a box is only useful if you MUST use a foreign host like Linux. Otherwise I would much prefer the machine actually run Plan 9.
> The BEAM hooks into so many system libraries, you must compile it on every flavor of linux instead of just unpacking a tarball.
This isn't true, one can statically compile the BEAM for Linux exactly the same as the JVM. Here's an example: https://github.com/yoshi-monster/static_erlang/
This isn't true, one can statically compile the BEAM for Linux exactly the same as the JVM. Here's an example: https://github.com/yoshi-monster/static_erlang/
Perhaps you should give Nix a try :)
Correct me if I'm wrong, but you'd still have to compile it from source on nix, no?
On my relatively powerful workstation, Erlang/BEAM takes about 7 minutes to compile.
We're working around this currently by having a fat devcontainer image, pre-built with Erlang inside (from source) by our CI. It chews through CI minutes unavoidably due to how docker layer caching works.
It would be awesome to just download and unpack a tarball, regardless of which distro you're using.
On my relatively powerful workstation, Erlang/BEAM takes about 7 minutes to compile.
We're working around this currently by having a fat devcontainer image, pre-built with Erlang inside (from source) by our CI. It chews through CI minutes unavoidably due to how docker layer caching works.
It would be awesome to just download and unpack a tarball, regardless of which distro you're using.
Nix is centered around the local Nix store and binary caching.
As long as the specific version of Erlang you’re using is present in either your Nix store or the global cache for your OS and arch (at cache.nixos.org), you should not need to compile anything.
And if you rely on custom builds, you can just setup your own binary cache. This is similar to remote caching in Bazel.
Some more details on Nix caching here: https://zero-to-nix.com/concepts/caching/
As long as the specific version of Erlang you’re using is present in either your Nix store or the global cache for your OS and arch (at cache.nixos.org), you should not need to compile anything.
And if you rely on custom builds, you can just setup your own binary cache. This is similar to remote caching in Bazel.
Some more details on Nix caching here: https://zero-to-nix.com/concepts/caching/
We do exactly this at my dayjob - we have (multiple) very specific combinations of (erlang, elixir, hex, rebar3) that we use which are pinned to exactly the versions we need. We have a private Nix cache so we only have to build them once.
https://www.cachix.org/ simplifies running a custom cache. Well worth the money, IMO.
That said, learning nix and setting up a nix cache is still a lot of work. Docker buildx might offer you some more knobs to cache portions of your build in a finer-grained manner without having to take the nix plunge.
https://www.cachix.org/ simplifies running a custom cache. Well worth the money, IMO.
That said, learning nix and setting up a nix cache is still a lot of work. Docker buildx might offer you some more knobs to cache portions of your build in a finer-grained manner without having to take the nix plunge.
Nix is enormously complicated, kind of unstable and not well documented.
I get that if you've gone through the pain of learning it you get a system with some very nice properties. But casually suggesting "maybe try nix" is a bit like telling someone who wants to listen to Mozart "maybe try playing a piano".
I get that if you've gone through the pain of learning it you get a system with some very nice properties. But casually suggesting "maybe try nix" is a bit like telling someone who wants to listen to Mozart "maybe try playing a piano".
How is it unstable? It's one of the most reliable pieces of software I have used honestly.
In any case 'nix-shell -p erlang|elixir' does not seem too complicated to me.
In any case 'nix-shell -p erlang|elixir' does not seem too complicated to me.
Flakes is still experimental despite being very widely used.
Flakes is also more stable than many "stable" projects. Flakes is more stable than Python 3, for example.
OP is already trying to do something pretty un-casual:
> If you want to control the exact version that's being used across your team (via `asdf` or similar), this practically means you'll end up compiling the BEAM over and over...
So I think it is perfectly appropriate to suggest a sharp tool.
> If you want to control the exact version that's being used across your team (via `asdf` or similar), this practically means you'll end up compiling the BEAM over and over...
So I think it is perfectly appropriate to suggest a sharp tool.
Nice. Have to spend some time reading it but i really like the minimalistic and clean design of your site content. No unnecessary colors/asides/verbiage/etc. nonsense but THE content and only the content presented directly.
A suggestion: Please add a "Overview" section in the beginning to provide the big-picture architecture since without that it would be hard to understand your code.
PS: In case you didn't see it, my comment here mentions some documents that you might find useful for your implementation - https://news.ycombinator.com/item?id=45883694
A suggestion: Please add a "Overview" section in the beginning to provide the big-picture architecture since without that it would be hard to understand your code.
PS: In case you didn't see it, my comment here mentions some documents that you might find useful for your implementation - https://news.ycombinator.com/item?id=45883694
Did not know about it much before but apparently was doing something similar (in spirit) when experimenting with simple task execution queues in go. This is quite interesting and given that hot code reloading is a thing in beam it might be something to explore further.
Now I'm curious whether Joe Armstrong's original Prolog implementation of the VM is available anywhere, but I doubt it.
The next best thing : Implementing a Functional Language for Highly Parallel Real Time Applications by J.L.Armstrong et al. (pdf) - https://www.cs.tufts.edu/~nr/cs257/archive/joe-armstrong/erl... Seems like this is the design behind the original JAM VM. The "References" section cites more early work.
See also The Erlang BEAM Virtual Machine Specification by the OG Bogumil Hausman himself ! Note: This document describes BEAM as it was in 1997. BEAM has grown and changed significantly between then and the time this note was added (2012). This information is mainly for historical interest. - https://www.cs-lab.org/historical_beam_instruction_set.html
Together, they should provide a lot of insights into ERTS/BEAM.
See also The Erlang BEAM Virtual Machine Specification by the OG Bogumil Hausman himself ! Note: This document describes BEAM as it was in 1997. BEAM has grown and changed significantly between then and the time this note was added (2012). This information is mainly for historical interest. - https://www.cs-lab.org/historical_beam_instruction_set.html
Together, they should provide a lot of insights into ERTS/BEAM.
Joe used to have all that code in his world-readable NFS-mounted home directory. He would just create a new directory for every idea or project. Take it with him from one computer to the next.
I hope that's preserved and one day published as e.g. the old MIT AI lab file system snapshots were.
(Robert Virding or Bjärne Däcker might well have a copy of the Prolog code to share if asked nicely.)
I hope that's preserved and one day published as e.g. the old MIT AI lab file system snapshots were.
(Robert Virding or Bjärne Däcker might well have a copy of the Prolog code to share if asked nicely.)
Clicked on the link because I don’t know what a “BEAM” is, was not disappointed
efitz(6)
Beautiful! :clap:
Wikipedia says "Originally BEAM was short for Bogdan's Erlang Abstract Machine, named after Bogumil "Bogdan" Hausman, who wrote the original version, but the name may also be referred to as Björn's Erlang Abstract Machine, after Björn Gustavsson, who wrote and maintains the current version."
Whether the B is for Bogdan or Bjorn, there's something really fun and Space Quest-y about it.