Yes! This is exciting to see. Erin Catto is such a cool hacker. Thank you, Erin, for sharing your code with the open source community.
There wasn't anything about determinism in the announcement, but I'd really love to see some more about that, too. Trying to use Unity's built-in physics to make a networked billiards game is quite troubling, when none of the clients can happily agree on what happened.
I am researching for a talk on the philosophy of code, the similarities of engineering and art, and why we enjoy reading old code. This amazing work you folks have done may be an interesting tangent.
The biggest question I have for you is why you imagine we are so interested in reading these old scrolls. Surely some of it is to see whether or not, technically, we can. Surely some of it is to get a glimpse into the human expression inscribed on them. Are we looking to learn anything, or just to connect with our ancestors? I'd like to hear your take on it, both for why you think it's important and, if you know, why your colleagues feel similarly.
There are three aspects of this which concern me the most: personal, open source, and business.
On the personal side, code is art, not a means to an end. Many of us love coding. For me, it's my favorite thing in the world to do and it has been that way for 20 years. That's why we enjoy going back over old game code, to see the clever tricks that people came up with. That's why we preserve code in archives, to last thousands of years. Sure, we can generate classic art, too, and it can be fun. But generating Rambrandt paintings is nothing like actually seeing the originals. And it's nothing like creating your own.
Code being art is exactly why coders are so opinionated about their tools. Just like artists are. "I only use vim and functional programming and Linux" says the coder. "I only use these paint brushes, and this type of paint, on this particular type of canvas" says the painter. People don't consider this enough, for coding, so they just say "Use the best tool for the job." But any paint brush will do, for most jobs. Any language will do, for most jobs. Why it all matters is because we care. We care because art is a form of personal expression.
On the open source side, maintainers now just get huge PRs full of slop changes. The submitters tend to feel like they're helping, but they're just wasting everyone's time. The trust landscape of contributing has been eroded. The security concerns of large changes are now more pressing than ever. The gross disregard for licensing spits in the face of both the nature of open source and the actual legal ground on which LLMs try to stand. And then so many projects trying to grow end up getting vibe-coded weekend projects as competitors and people actually think they're comparable. For the non-trivial cases, they're not, because of the business side.
On the business side, let's take a programming language for example, like Zig. A business will be hesitant to adopt a particular technology if it's just one guy working on it. But if the technology has a community, a foundation, and a handful of devs working on it full-time, it is a safer investment. A big reason here is that the bus factor will be higher than one. However, LLM-generated code is a black box. It has a bus factor of zero. Nobody understands all of the code. Probably, nobody ever will. If there is a critical bug at a critical time, there is nobody to call. We would have to rely on LLMs to find and fix the bug, but how much faith can we actually put into this? That covers third party technology, but the same applies to first party technology.
If a small business decides to use Claude to create their mobile app, rather than hire an experienced dev, they now suffer the same black box and bus factor problem. Even if they hire a dev, but they expect the dev to finish at vibe-coding speeds, we end up with the same problem.
---
In short, vibe coding removes everything I love about my favorite thing to do. It also creates heaps of blackbox code with no ownership, beholden to proprietary tech which is only getting more expensive. Why would I like that?
Babashka's interop is with Java, since Babashka uses a Graal-compiled version of the JVM. It's still the JVM, just baked down.
This is different from interop with the native world. It's different from the host runtime actually being native, rather than a baked down version of a whole VM.
Graal's native images blur the line between the JVM and native, I would not say Babashka has a native runtime. Perhaps borkdude would disagree. Might be an interesting discussion.
Once you learn Clojure's syntax and semantics, you're no longer bound to the JVM. There's ClojureScript (JS), ClojureCLR, ClojureDart, jank (C++), Basilisp (Python), babashka (SCI), and many others. This means that, if you don't know Java or don't like the JVM, you can likely use Clojure wherever you already feel most comfortable.
For the most part, any Clojure code which doesn't use host interop will work on all dialects. Clojure also has support for conditional code, depending on the current dialect.
jank's custom IR is completely separate and unrelated to LLVM IR, aside from both of them being SSA-based IRs. We go from jank's AST into jank's IR into C++, which we then give to Clang compile into the LLVM JIT runtime. So LLVM IR is used in the pipeline, but we don't touch it directly. More info on that, and a diagram, is here: https://book.jank-lang.org/dev/ir.html (which I linked in the post)
As another said, jank is not replacing LLVM or LLVM IR. We still use LLVM IR! There is a diagram here which shows the pipeline: https://book.jank-lang.org/dev/ir.html
The main thing is that we just use our own IR first, to perform optimizations with contextual data which is gone by the time we get to LLVM IR. That's also why these optimizations are not practical to write in LLVM, since by the time we get to LLVM IR, we're too far separated from jank's AST with the high level semantics of Clojure.
So we just add an intermediate step. Once we have jank's AST, turn it into our own IR, do some optimizations on it for things that LLVM won't be able to see, and then hand it off to LLVM to do the rest.
The first three paragraphs here are on point! jank's IR passes will not worry much about things like load/store optimization, register allocation, inlining C++ functions, etc. These are in LLVM's domain. We just worry about the Clojure side of things. Polymorphic math is intense, but we do our best to avoid the extra work by unboxing whenever possible.
> A future optimisation might be to specialise for unboxed types: far more potential speed improvement over pointer tagging, and IMO quite amenable to analysis with the Jank IR
All of these math functions are templates with four specific categories:
1. Object and object
2. Primitive and primitive
3. Primitive and object
4. Object and primitive
We handle the difference between typed objects (like integer_ref) and type-erased objects (object_ref) as well. This template then gets inlined, which is exactly what the last step of the benchmark optimizations (adding annotations) ensured. The return type of these functions will prefer primitive types, rather than automatically boxing. jank's analyzer tracks all types used, at compile-time, and supports automatic boxing. This means that we're already using the most optimal primitive math whenever we can and that it will indeed inline to just an operator call when working on two primitives, or two typed objects, or a combination thereof.
I spoke with a couple Clang and LLVM devs about MLIR when I was doing the original design for jank's IR. The general consensus was that MLIR added a great deal of complexity on top of designing/implementing an IR and nobody was confident it was actually worth the effort. Since I knew exactly what I wanted, I just built that.
Hey lemming! You're right, which is why it should be used sparingly. Since clojure.core is compiled (on the JVM) with direct linking, reacting to var changes isn't an intended concern, since they're not going to work properly throughout any clojure.core code using that var. This makes it a good candidate ns for inlining things. But users shouldn't just be doing this for their normal application vars without giving it due consideration.
It's a native Clojure dialect which is also a C++ dialect, including a JIT compiler and nREPL server. I'm currently building out a custom IR so I can do optimization passes at the level of Clojure semantics, since LLVM will not be able to do them at the LLVM IR level.
jank is Clojure and will track upstream Clojure development. I'm working closely with the Clojure team and other dialect devs to ensure this remains the case. I am leading a cross-dialect clojure-test-suite to help ensure parity across all dialects: https://github.com/jank-lang/clojure-test-suite We have support or work ongoing for Clojure JVM, ClojureScript, Clojure CLR, babashka, Basilisp, and jank.
With that said, jank will do some trail blazing down other paths (see my other comments here about Carp), but they will be optional modes which people can enable which are specific to jank. Clojure compatibility will remain constant.
JIT compiling C++ is definitely the slowest thing we do. However, we're working on two different codegen modes.
1. C++
2. LLVM IR
The IR is much faster to compile, but its perf isn't nearly as good. This is meant to be a nice trade off, during iteration, so that you can use C++ codegen for your release artifact, but stick with IR when you're REPLing. The IR gen is still unstable right now, for the alpha, but we'll have both solidified this year.
Carp is great and I would love to include a mode of jank which is very much Carp-esque. If you're interested in working together on this, please let me know.
No full-lang static type system. jank has seamless C++ interop and all interop is statically typed, but as soon as things get back into Clojure land, it's all dynamically typed and hella polymorphic.
I will be exploring optional static typing modes for jank in the future, a la Carp. That will not be this year, though.
Hi! I'm happy to accept grammatical PRs to the book. You could also report issues via Slack or Github issue. I will not be accepting larger PRs to the book, to maintain a consistent voice.
There wasn't anything about determinism in the announcement, but I'd really love to see some more about that, too. Trying to use Unity's built-in physics to make a networked billiards game is quite troubling, when none of the clients can happily agree on what happened.