Leveraging Rust in our Java database(questdb.io)
questdb.io
Leveraging Rust in our Java database
https://questdb.io/blog/leveraging-rust-in-our-high-performance-java-database/
89 comments
Do you have any background reading for high performance java or how it's used in the finance world? I had no idea it was used in this niche. As a crufty Java dev who types `new` everywhere and never gives thought to GC, squeaking out high performance sounds like an interesting side of the language.
Maybe we should write a blog post, though there's gotta be one out there for this already.
The short of it is: Learn C. Learn your system calls. Learn JNI. Learn about com.sun.misc.Unsafe. Learn about the disruptor pattern. Learn how to pool objects. The long type can pack a lot of data. Go from there!
The short of it is: Learn C. Learn your system calls. Learn JNI. Learn about com.sun.misc.Unsafe. Learn about the disruptor pattern. Learn how to pool objects. The long type can pack a lot of data. Go from there!
Check out Peter Lawrey's blog [1] has lots of excellent content coming from high performance trading background. Also some older (probably dated) content on Martin Thompson's Mechanical Sympathy [2] blog.
[1] http://blog.vanillajava.blog/
[2] https://mechanical-sympathy.blogspot.com/
[1] http://blog.vanillajava.blog/
[2] https://mechanical-sympathy.blogspot.com/
So Rust combines the expressiveness of C++ with the ease of development of Java?
I'd say expressiveness of C++ with productivity of Java.
Rust is indeed not easy to learn.
A little example. Yesterday I was making changes to our C++ client library and I wanted to improve the example in our documentation.
We use a dedicated protocol called ILP for streaming data ingestion and each of the inserted rows has a designated timestamp.
In the Rust example, I using added support for chrono::DateTime and it was trivially easy for me to add a timestamp for a specific example date and time: Utc.with_ymd_and_hms(1997, 7, 4, 4, 56, 55).
Our C++ library instead takes an std::chrono::time_point. I wanted to use the same datetime. As far as I can tell it requires first going through the old C "struct tm" type (which is local and not UTC), then converting to "time_t" then converting to utc via gmtime and then constructing a time_point from that. After 10 minutes the code got too long and complicated so I just substituted it a timestamp specified as an int64_t in nanoseconds.
Don't get me wrong, the C++ time_point is a work of art in how flexible it is, but unnecessarily complicated in most cases.
I should add that I also spent 45 minutes yesterday debugging a CMake issue.
Rust is not easy to learn, but it's just more modern and productive.
C++ is still great if you've got a massive team, but at our scale I don't think it makes any sense.
A little example. Yesterday I was making changes to our C++ client library and I wanted to improve the example in our documentation.
We use a dedicated protocol called ILP for streaming data ingestion and each of the inserted rows has a designated timestamp.
In the Rust example, I using added support for chrono::DateTime and it was trivially easy for me to add a timestamp for a specific example date and time: Utc.with_ymd_and_hms(1997, 7, 4, 4, 56, 55).
Our C++ library instead takes an std::chrono::time_point. I wanted to use the same datetime. As far as I can tell it requires first going through the old C "struct tm" type (which is local and not UTC), then converting to "time_t" then converting to utc via gmtime and then constructing a time_point from that. After 10 minutes the code got too long and complicated so I just substituted it a timestamp specified as an int64_t in nanoseconds.
Don't get me wrong, the C++ time_point is a work of art in how flexible it is, but unnecessarily complicated in most cases.
I should add that I also spent 45 minutes yesterday debugging a CMake issue.
Rust is not easy to learn, but it's just more modern and productive.
C++ is still great if you've got a massive team, but at our scale I don't think it makes any sense.
Rust development is Java-esque in some ways, I think that is a fair characterization, but the Rust language is noticeably less expressive than C++. The relative lack of expressiveness has been a stumbling block for use in some domains, because modern C++ implementations require a fraction of the code to do the same thing. This disparity doesn't show up for all types of code, so it is not uncommon to see both Rust and C++ used in the same org depending on what the code is trying to do. They have different strengths.
As someone who's familiar with Rust, but not at all with C++, can you elaborate on what situations Rust is less expressive than C++ in?
My naïve guess is high performance data structures with reference cycles?
My naïve guess is high performance data structures with reference cycles?
I'd say both Rust and C++ trade blows when it comes to expressiveness. You know Rust already, so I'm not going to try sell you how powerful macros can be (see SQLx's ability to compile-time check SQL queries).
And indeed C++ templates are a lot more like Rust macros than Rust generics: They're turing-complete.
Joint with some interesting language choices like SFINAE (substitution failure is not an error), you end up with the ability to specialize functions, methods and whole classes in C++.
You can also have functions that return different types.
C++ templates work like duck-typing within a static language: In Rust you need to say what traits your generics need to support. In C++ it will try and substitute and if it fails (say, because T doesn't support the required methods) it will try another substitution until none are left.
If none of the substitutions work, you will be shown ALL of them in error reporting: This is what leads to pages and pages of compile errors of single-character typos in C++.
Templates are really cool, but also pretty confusing when reading code since you're in a guessing game of what types will fit the constraints imposed by the _implementation_ of the function.
From C++20 there's concepts to make templates work a little smoother.
There's been whole books written about how to abuse templates: these are pre-requisite knowledge when working in large codebases.
And indeed C++ templates are a lot more like Rust macros than Rust generics: They're turing-complete.
Joint with some interesting language choices like SFINAE (substitution failure is not an error), you end up with the ability to specialize functions, methods and whole classes in C++.
You can also have functions that return different types.
C++ templates work like duck-typing within a static language: In Rust you need to say what traits your generics need to support. In C++ it will try and substitute and if it fails (say, because T doesn't support the required methods) it will try another substitution until none are left.
If none of the substitutions work, you will be shown ALL of them in error reporting: This is what leads to pages and pages of compile errors of single-character typos in C++.
Templates are really cool, but also pretty confusing when reading code since you're in a guessing game of what types will fit the constraints imposed by the _implementation_ of the function.
From C++20 there's concepts to make templates work a little smoother.
There's been whole books written about how to abuse templates: these are pre-requisite knowledge when working in large codebases.
The big one is metaprogramming. Most people that have never really used it grok how powerful (and clean and maintainable) it has become in recent versions of C++. I work on a few different C++20 code bases and the amount of code that is no longer written because it is generated at compile-time with rigorous type safety is brilliant. It goes well beyond vanilla templating, you can essentially build a DSL for the application domain.
Another one, with a more limited audience, is data models where object ownership and lifetimes are inherently indeterminate at compile-time. Since C++ allows you to design your own safety models, since they are opt-in and not built into the compiler, you can provide traditional ownership semantics (e.g. the equivalent of std::unique_ptr) without exposing the mechanics of how ownership or lifetimes are resolved at runtime. Metaprogramming plays a significant role in making this transparent.
Those are the two the matter the most for my purposes. They save an enormous amount of code and bugs. Rust has a litany of other gaps (lack of proper thread local, placement new, et al) but I don't run into those cases routinely.
The data structure thing you mention would be annoying but to be honest I rarely design data structures like this. For performance, most data structures tend to rely on clever abuse of arrays.
Another one, with a more limited audience, is data models where object ownership and lifetimes are inherently indeterminate at compile-time. Since C++ allows you to design your own safety models, since they are opt-in and not built into the compiler, you can provide traditional ownership semantics (e.g. the equivalent of std::unique_ptr) without exposing the mechanics of how ownership or lifetimes are resolved at runtime. Metaprogramming plays a significant role in making this transparent.
Those are the two the matter the most for my purposes. They save an enormous amount of code and bugs. Rust has a litany of other gaps (lack of proper thread local, placement new, et al) but I don't run into those cases routinely.
The data structure thing you mention would be annoying but to be honest I rarely design data structures like this. For performance, most data structures tend to rely on clever abuse of arrays.
What makes you say that development in Rust is Java-esque? Genuinely curious!
Java and Rust have similar goals, you can see it in the design of the language and the ecosystem. I lived in the early Java ecosystem and the Rust ecosystem has a similar vibe. They were both attacking the same problem but were are products of their respective times. The key difference is that Rust was able to learn from Java's mistakes and make ambitious technical bets that would not have been feasible at the time Java was designed. Java was invented when most serious applications were written in C and sometimes early versions of C++. It eliminated much of the conceptual complexity that made it difficult for all but the best developers to be productive in C or C++. In this Java was a massive success, it was easy to scale development even if you didn't have the world's best engineers.
Java's mistake is that they went much too far when they nerfed the language. For highly skilled developers that could write robust C or C++, the poor expressiveness of Java made many easy things difficult or impossible. It was clearly a language designed with business logic in mind, any systems-y software was an afterthought. The release of C++11 ushered in the era of "modern" C++, killing Java's momentum in the systems-y software space.
Rust, in my view, attempts to solve the same abstract problem as Java -- we will never produce enough developers that are competent at writing C or C++. Rust talks about "safety by design" in the same way Java did when it was first released. However, it does so without being so limited that highly skilled software engineers will find the language unusable or giving up so much runtime performance that the operational economics are poor. In my mental taxonomy, Rust is pretty close to what Java intended to be but then never quite delivered on.
Java's mistake is that they went much too far when they nerfed the language. For highly skilled developers that could write robust C or C++, the poor expressiveness of Java made many easy things difficult or impossible. It was clearly a language designed with business logic in mind, any systems-y software was an afterthought. The release of C++11 ushered in the era of "modern" C++, killing Java's momentum in the systems-y software space.
Rust, in my view, attempts to solve the same abstract problem as Java -- we will never produce enough developers that are competent at writing C or C++. Rust talks about "safety by design" in the same way Java did when it was first released. However, it does so without being so limited that highly skilled software engineers will find the language unusable or giving up so much runtime performance that the operational economics are poor. In my mental taxonomy, Rust is pretty close to what Java intended to be but then never quite delivered on.
I wouldn’t call Rust easy to develop in.
I would as well. I've also placed in the top 100 of advent of code before... using Rust :)
I think once you get familiar with it, it is just slightly slower to write than python.
I think once you get familiar with it, it is just slightly slower to write than python.
I agree with this statement if my data structures map cleanly to Rust's preferred single-ownership, which most problems in my domain do.
Sometimes I do run across problems that are difficult to express in Rust without resorting to interior mutability, and it can slow me down to figure out the best way to model my data.
Sometimes I do run across problems that are difficult to express in Rust without resorting to interior mutability, and it can slow me down to figure out the best way to model my data.
Definitely agree - when I say slightly slower I'm mostly referring to the happy-path/basic uses (reading files, using hashmaps, web servers, etc).
There are definitely aspects of Rust that are much more complex (typically with a tradeoff of more expressiveness, but not always), but at least in my experience, these are usually areas where you can't easily express the same thing in Python. I think many times people forget that you can frequently `clone` your way out of many issues if you are trying to move fast.
And of course areas where I still only use python: manipulating tabular data, making graphs, quick scripts for interacting with APIs etc.
There are definitely aspects of Rust that are much more complex (typically with a tradeoff of more expressiveness, but not always), but at least in my experience, these are usually areas where you can't easily express the same thing in Python. I think many times people forget that you can frequently `clone` your way out of many issues if you are trying to move fast.
And of course areas where I still only use python: manipulating tabular data, making graphs, quick scripts for interacting with APIs etc.
There are idioms you can use where instead of references you use indexes into a Vec or other container. This is normal for folks coming from a gamedev background, but non-obvious to everyone else. Once you get the hang of these idioms, the productivity difference between "object soup" Python and Rust gets smaller, and the resulting code is also closer to what a "production" app would need to look like. This is an extra learning curve for Rust, though, on top of the already famously steep learning curve for the basics.
I really don’t like this approach - that’s just pointers without memory safety issues, but you get all the other problems, e.g. use-after-free, without any of the tooling to catch it for you like valgrind.
I use it professionally and can't disagree more with this statement.
I also use it professionally, for both web apps and HPC algorithms.
Are there some things faster to write in python? Sure. But I find the mental overhead is significantly less (for me) in Rust, and overall dev time is about equal, since I typically hit far fewer bugs and spend less time reading docs in Rust. I can't remember the last time I hit a footgun in Rust. Seems I hit one every week in python.
We recently migrated a ~10k SLOC Django JSON API server to axum/sqlx (Rust). I couldn't be happier - faster to ship new features, faster to refactor, fewer bugs, and response times got about 10x quicker.
Are there some things faster to write in python? Sure. But I find the mental overhead is significantly less (for me) in Rust, and overall dev time is about equal, since I typically hit far fewer bugs and spend less time reading docs in Rust. I can't remember the last time I hit a footgun in Rust. Seems I hit one every week in python.
We recently migrated a ~10k SLOC Django JSON API server to axum/sqlx (Rust). I couldn't be happier - faster to ship new features, faster to refactor, fewer bugs, and response times got about 10x quicker.
I work a lot on async code with data structures that need interior mutability and it's kind of a pathological case for borrow checking. Everything is effectively wrapped in Arc<RwLock<_>> which adds a bunch of noise to method implementations.
It's like skiing (or maybe riding a bike for the first time.) Steep learning curve then becomes somewhat instinctual and fairly routine and trivial and fun. Until you get into the tricky terrain, and then it will put up resistance. But usually for your own good.
Rolling with the analogy, I think learning rust a lot more like snowboarding, and C/C++ is more like learning to ski.
I fell a lot more learning on snowboard than on skis.
Skis had a much quicker early learning curve, and made me feel over-confident. Several times I found myself on trails too steep for my skills, and the skis made me have to work hard to recover. Most of the techniques I learned as a beginner didn't work beyond green trails, and blues, blacks all required new, harder, techniques.
With snowboarding, because I fell a lot more early, my confidence slowly grew. Meanwhile the techniques I was learning on the greens, and the tool in general, that were HARD to learn on the greens were actually EASIER on the blues, and continued to work on the black-diamonds. Granted I also had to learn new techniques on the harder trails, but the beginner techniques and the slower development made me MUCH more confortable across the whole mountain much faster than skis.
Double-blacks are more like unsafe rust. :-D
I fell a lot more learning on snowboard than on skis.
Skis had a much quicker early learning curve, and made me feel over-confident. Several times I found myself on trails too steep for my skills, and the skis made me have to work hard to recover. Most of the techniques I learned as a beginner didn't work beyond green trails, and blues, blacks all required new, harder, techniques.
With snowboarding, because I fell a lot more early, my confidence slowly grew. Meanwhile the techniques I was learning on the greens, and the tool in general, that were HARD to learn on the greens were actually EASIER on the blues, and continued to work on the black-diamonds. Granted I also had to learn new techniques on the harder trails, but the beginner techniques and the slower development made me MUCH more confortable across the whole mountain much faster than skis.
Double-blacks are more like unsafe rust. :-D
Yeah, fair, I often make the same point about snowboarding vs skiing. Skiing is "easy" to pick up but difficult to master. Most people on the hill are just backseating it down in bad form, like they learned in their first week. Getting good form is ... a lifelong effort.
Snowboarding is brutally hard to pick up at first, unless you hate your tailbone. I have never really tried, I'm too old for that kind of pain.
But I'm a telemark skier, so the worst of both worlds :-) Maybe telemark is like advanced C++. But probably more like Haskell.
Snowboarding is brutally hard to pick up at first, unless you hate your tailbone. I have never really tried, I'm too old for that kind of pain.
But I'm a telemark skier, so the worst of both worlds :-) Maybe telemark is like advanced C++. But probably more like Haskell.
Isn't that the same for most languages once you move beyond vanilla Java/C#/Go etc.?
Rust has "opposite" call semantics from what most people are used to, trained since they were new programmers. Sort of. It takes a while to think like the borrow checker and get used to the way arguments get passed around. It's like using C++ and doing std::move for every non-reference argument.
Sounds like the transition from OOP to functional. It feels opposite before you become comfortable.
I would
Thanks for the interesting post. Do you plan to maybe use in the future JEP 442? https://openjdk.org/jeps/442
When the time is right. There's finally new APIs coming in the Java space that will make native-code interop easier and more reliable.
Our open source database edition can also be used embedded though, so we can only upgrade at the pace of our customers and because of that we still are compatible all the way down to Java 8.
Were it not for this detail, we'd probably consider it a lot sooner.
Our open source database edition can also be used embedded though, so we can only upgrade at the pace of our customers and because of that we still are compatible all the way down to Java 8.
Were it not for this detail, we'd probably consider it a lot sooner.
Why did you not consider leveraging Java's recent Foreign Function and Memory API ?
FYI, even in the most recent Java (21) LTS release this is flagged as an "early access" feature so you're unlikely to see production applications using it yet.
One of our distribution channels is Maven Central where we ship Java 11 compatible library. Embedded users preclude us from leveraging latest Java features.
Do you know ravendb? Document db almost entirely written in c#. It's incredible how fast it can get, but as you said it does not look like typical lob code at all
> We seldom use the new keyword and objects are designed to be pooled and reused.
Just note that depending on the selection of the GC, this kind of usage may make the GC work more than when allocating new objects, not less. In particular, with the newer GCs -- G1 and ZGC -- mutating existing objects may be more costly than allocating new ones depending on circumstances. In general, the new GCs are optimised to work the least and give the best performance when the allocation rate is neither too high nor too low; the new GCs also reuse memory better than object pools. Reusing objects also precludes scalarization optimisations, i.e. not every `new Foo` actually results in a heap allocation, and can be optimised to work directly in registers.
So while on very old JVMs (such as Java 8) a "zero allocation" strategy may result in better performance and in "zero GC", on newer JVMs it may result in worse performance and more GC work (in fact, it will almost surely not yield zero GC). While it depends on many variables, I would advise against a zero allocation strategy on newer JVMs as the default path toward better performance or even better latency. It's an approach that seems to be very strongly coupled to the way the JVM was designed over a decade ago, but a lot has changed since then.
Additionally, Java now offers manual memory management and efficient FFI that are significantly better than what JNI offered: https://openjdk.org/jeps/442
Just note that depending on the selection of the GC, this kind of usage may make the GC work more than when allocating new objects, not less. In particular, with the newer GCs -- G1 and ZGC -- mutating existing objects may be more costly than allocating new ones depending on circumstances. In general, the new GCs are optimised to work the least and give the best performance when the allocation rate is neither too high nor too low; the new GCs also reuse memory better than object pools. Reusing objects also precludes scalarization optimisations, i.e. not every `new Foo` actually results in a heap allocation, and can be optimised to work directly in registers.
So while on very old JVMs (such as Java 8) a "zero allocation" strategy may result in better performance and in "zero GC", on newer JVMs it may result in worse performance and more GC work (in fact, it will almost surely not yield zero GC). While it depends on many variables, I would advise against a zero allocation strategy on newer JVMs as the default path toward better performance or even better latency. It's an approach that seems to be very strongly coupled to the way the JVM was designed over a decade ago, but a lot has changed since then.
Additionally, Java now offers manual memory management and efficient FFI that are significantly better than what JNI offered: https://openjdk.org/jeps/442
Finance and video games both have really stubborn engineering traditions.
> https://openjdk.org/jeps/442
This succinctly is the answer.
> not every `new Foo` actually results in a heap allocation, and can be optimised to work directly in registers.
While I know in my heart of hearts that you are right, this category of advice has so many caveats. It's like when Graal markets itself as a potential substitute for NodeJs, where its performance is abjectly terrible.
I have never personally succeeded in convincing anyone with these deep programming traditions to try something new that may be Better in Every Way. It always has to be this long, trickle down journey of adoption. People still talk about John Carmack and video game engineering, decades after any of his innovations have been verbatim relevant and decades after an individual could possibly have a hope and prayer of authoring a first person shooter with an audience from scratch. But see, I need to know a lot of stuff to understand that.
> https://openjdk.org/jeps/442
This succinctly is the answer.
> not every `new Foo` actually results in a heap allocation, and can be optimised to work directly in registers.
While I know in my heart of hearts that you are right, this category of advice has so many caveats. It's like when Graal markets itself as a potential substitute for NodeJs, where its performance is abjectly terrible.
I have never personally succeeded in convincing anyone with these deep programming traditions to try something new that may be Better in Every Way. It always has to be this long, trickle down journey of adoption. People still talk about John Carmack and video game engineering, decades after any of his innovations have been verbatim relevant and decades after an individual could possibly have a hope and prayer of authoring a first person shooter with an audience from scratch. But see, I need to know a lot of stuff to understand that.
Yeah scalar replacement is very conservative and things that a human can easily tell don’t escape the jit cannot.
This is 2ish years old but has a lot of detail on what escape analysis can and cannot see:
https://gist.github.com/JohnTortugo/c2607821202634a6509ec3c3...
This is 2ish years old but has a lot of detail on what escape analysis can and cannot see:
https://gist.github.com/JohnTortugo/c2607821202634a6509ec3c3...
Yeah, it kinda messes with the "most objects die young" philosophy behind generational collectors.
Here's another (related) caveat with taking the "minimal object creation", pooled memory, and/or using lots of non-GC native heap memory path: now you've got blocks of memory sitting there in Process RSS that the GC either can't do anything about, or (worse) knows nothing about.
These types of runtimes are on the whole designed with the philosophy that they Own All The Things, so it's like an invasive body driving the "immune system" nuts...
Never had this problem in Java (haven't worked in it for 10 years) but at a previous job (in Julia) I worked on a database buffer pool / pager where the memory was explicitly managed/allocated (through syscalls to anonymous mmap) for performance reasons. Those pages belong to the same PID as the broader Julia process, but were not managed by Julia runtime, and so the runtime can run into all sorts of issues with OOM kills or huge pauses as the system allocates aggressively without collecting often, thinking it has plenty of headroom... but doesn't.
I know for a fact the JVM GC is smarter than this, and can be tuned more expertly to manage these type of situations, but it's still a big giant caveat...
Here's another (related) caveat with taking the "minimal object creation", pooled memory, and/or using lots of non-GC native heap memory path: now you've got blocks of memory sitting there in Process RSS that the GC either can't do anything about, or (worse) knows nothing about.
These types of runtimes are on the whole designed with the philosophy that they Own All The Things, so it's like an invasive body driving the "immune system" nuts...
Never had this problem in Java (haven't worked in it for 10 years) but at a previous job (in Julia) I worked on a database buffer pool / pager where the memory was explicitly managed/allocated (through syscalls to anonymous mmap) for performance reasons. Those pages belong to the same PID as the broader Julia process, but were not managed by Julia runtime, and so the runtime can run into all sorts of issues with OOM kills or huge pauses as the system allocates aggressively without collecting often, thinking it has plenty of headroom... but doesn't.
I know for a fact the JVM GC is smarter than this, and can be tuned more expertly to manage these type of situations, but it's still a big giant caveat...
> In general, the new GCs are optimised to work the least and give the best performance when the allocation rate is neither too high nor too low; the new GCs also reuse memory better than object pools.
This a little bit surprising to me that low object creation can degrade GC performance; what's the failure mode for G1/ZGC in this scenario?
This a little bit surprising to me that low object creation can degrade GC performance; what's the failure mode for G1/ZGC in this scenario?
In many cases, because it defeats generational collection. It pushes everything into longer generations because they hang around longer.
Doing more young generation collection is sometimes cheaper in aggregate (more frequent but far smaller and usually much more efficient) than adding more data to the older generations (less frequent and more costly, longer pauses, for object pools it happens on all of it even when none of it is currently used, etc).
But as doctorpangloss said: so many caveats. There's ample evidence that it is both better and worse, it depends on lots of details.
The main thing you can confidently claim is that it is not the majority of code, so most language optimizations will choose to improve straightforward and common stuff at the cost of this niche. Not always, but there is definitely more energy in improving the 90%+ cases and that adds up over time. Squeezing out the last bits of performance requires constant upkeep.
Doing more young generation collection is sometimes cheaper in aggregate (more frequent but far smaller and usually much more efficient) than adding more data to the older generations (less frequent and more costly, longer pauses, for object pools it happens on all of it even when none of it is currently used, etc).
But as doctorpangloss said: so many caveats. There's ample evidence that it is both better and worse, it depends on lots of details.
The main thing you can confidently claim is that it is not the majority of code, so most language optimizations will choose to improve straightforward and common stuff at the cost of this niche. Not always, but there is definitely more energy in improving the 90%+ cases and that adds up over time. Squeezing out the last bits of performance requires constant upkeep.
> what's the failure mode for G1/ZGC in this scenario?
GC barriers (special code that gets triggered on some operations by some GCs, such as when mutating a reference field during a GC cycle -- that's a "write barrier").
Concurrent GCs have special rules for newly allocated objects (which are really just a pointer bump) because no one else has seen them yet; young objects require no GC barriers. But once an object is old, a concurrent GC needs to do some work to learn about references in the object changing. So while allocating a new object is usually a pointer bump (and sometimes not even that when the object is scalarized), mutating an old object triggers a GC slow-path that has to mark some data in a shared data structure (so we're talking memory ordering fences) to make sure that the mutated pointer is not overlooked by the GC.
OpenJDK's new GCs are really, very, very good. The new generational ZGC in JDK 21 (with sub-millisecond worst case pause) is just amazing. But these GCs are optimised for "reasonable" Java code and against "unreasonable" object pooling. Things are different from where they were a decade ago in Java 8.
GC barriers (special code that gets triggered on some operations by some GCs, such as when mutating a reference field during a GC cycle -- that's a "write barrier").
Concurrent GCs have special rules for newly allocated objects (which are really just a pointer bump) because no one else has seen them yet; young objects require no GC barriers. But once an object is old, a concurrent GC needs to do some work to learn about references in the object changing. So while allocating a new object is usually a pointer bump (and sometimes not even that when the object is scalarized), mutating an old object triggers a GC slow-path that has to mark some data in a shared data structure (so we're talking memory ordering fences) to make sure that the mutated pointer is not overlooked by the GC.
OpenJDK's new GCs are really, very, very good. The new generational ZGC in JDK 21 (with sub-millisecond worst case pause) is just amazing. But these GCs are optimised for "reasonable" Java code and against "unreasonable" object pooling. Things are different from where they were a decade ago in Java 8.
I mean no disrespect to the authors, but this seems like an extremely painful way to write an application. How common is it to write Java apps this way? It seems like there must have been a better language choice than trying to work against the GC and rewriting parts of the standard lib? And now adding in Rust via JNI? It just feels very painful to me.
QuestDB engineer here:
It's true that our non-idiomatic Java usage denies us some of the benefits typically associated with Java programming. Automatic memory management and the old "Write Once, Run Anywhere" paradigm are difficult to maintain due to our reliance on native libraries and manual memory management.
I see two classes of reasons for choosing Java:
1. Historical: The QuestDB codebase predates Rust. According to Wikipedia, the initial Rust release was in 2015. The oldest commit in the QuestDB repo is from 2014: https://github.com/questdb/questdb/commit/95b8095427c4e2c781... What were the options back in 2014? C++? Too complicated. C? Too low-level. Pretty much anything else? Either too slow or too exotic.
2. Technical: Java, even without GC or WORA, still offers some advantage. 2a: The tooling is robust, especially when compared to C++. This starts with build systems (don't get me started on CMake!), and extends to aspects like observability. Stacktraces in Java are taken for granted. What's the state of stacktrace walking/printing in C++? I think it boils down to either Boost, C++23, or some other form of black magic. (I might be wrong here tho) 2b: It's a simpler language, especially when compared to C++ or even Rust. This makes it easier to hire people and also attracts external contributors: https://github.com/questdb/questdb/graphs/contributors 2c: The HotSpot JIT still provides us with solid peak performance without having to mess with PGO, etc. 2d: Concurrency is easier with Java's managed memory, eliminating the need for hazard pointers and the like.
It's true that our non-idiomatic Java usage denies us some of the benefits typically associated with Java programming. Automatic memory management and the old "Write Once, Run Anywhere" paradigm are difficult to maintain due to our reliance on native libraries and manual memory management.
I see two classes of reasons for choosing Java:
1. Historical: The QuestDB codebase predates Rust. According to Wikipedia, the initial Rust release was in 2015. The oldest commit in the QuestDB repo is from 2014: https://github.com/questdb/questdb/commit/95b8095427c4e2c781... What were the options back in 2014? C++? Too complicated. C? Too low-level. Pretty much anything else? Either too slow or too exotic.
2. Technical: Java, even without GC or WORA, still offers some advantage. 2a: The tooling is robust, especially when compared to C++. This starts with build systems (don't get me started on CMake!), and extends to aspects like observability. Stacktraces in Java are taken for granted. What's the state of stacktrace walking/printing in C++? I think it boils down to either Boost, C++23, or some other form of black magic. (I might be wrong here tho) 2b: It's a simpler language, especially when compared to C++ or even Rust. This makes it easier to hire people and also attracts external contributors: https://github.com/questdb/questdb/graphs/contributors 2c: The HotSpot JIT still provides us with solid peak performance without having to mess with PGO, etc. 2d: Concurrency is easier with Java's managed memory, eliminating the need for hazard pointers and the like.
Looks like they have an interesting range of customers[0] so my take on "why even add the Rust" is because their customers are already using Java, a total rewrite might be considered irresponsible as it would be incompatible with their existing customer base. I do have to wonder though, if some serious Java refactoring in any way, would have helped at all. How many code smells do they have going on in their codebase? Or is it to the best of their knowledge a really clean Java codebase?
Note Discord themselves has used Rust for bottlenecks with Erlang/Elixir/Beam[1].
[0]: https://questdb.io/customers/
[1]: https://discord.com/blog/using-rust-to-scale-elixir-for-11-m...
Note Discord themselves has used Rust for bottlenecks with Erlang/Elixir/Beam[1].
[0]: https://questdb.io/customers/
[1]: https://discord.com/blog/using-rust-to-scale-elixir-for-11-m...
I'm not suggesting they rewrite. I'm questioning if Java was ever the best choice for this application in the first place. They are using Java as if it were C++. Perhaps it would have been better to write it in C++ in that case? I'm not drawing conclusion, as I'm not in their domain and have not given this a lot of thought, but it just strikes me as a particularly shaky foundation.
Java brings some good things over C++. For example memory-safe VM, easier language with way less gotchas, better tooling, awesome IDEs.
I'd say, if C++ performance is not essential, Java might actually be a good choice. It's very fast and GC issues could be worked around.
I'd say, if C++ performance is not essential, Java might actually be a good choice. It's very fast and GC issues could be worked around.
How is it memory safe if they are using off heap memory with malloc/free? I think it would only qualify as memory safe if they were using the GC which they purposefully avoid.
I will agree on the other points, however, but I wonder how much of a gain that is against the pain of working against the language. Hard to say.
I will agree on the other points, however, but I wonder how much of a gain that is against the pain of working against the language. Hard to say.
> How is it memory safe if they are using off heap memory with malloc/free? I think it would only qualify as memory safe if they were using the GC which they purposefully avoid.
You can still write a safe/minimal wrapper around that with just the actual API you need. (Instead of allowing everything to just peek/poke around in arbitrary off-heap memory.)
You can still write a safe/minimal wrapper around that with just the actual API you need. (Instead of allowing everything to just peek/poke around in arbitrary off-heap memory.)
I think this is an argument for correct program architecture/design, not one for language. I suspect one could do the same thing in any memory unsafe language. Granted, Java might make this somewhat easier by not having the concept of modifiable pointers, but discipline and static analysis could likely achieve the same.
> I suspect one could do the same thing in any memory unsafe language. Granted, Java might make this somewhat easier by not having the concept of modifiable pointers, but discipline and static analysis could likely achieve the same.
You can theoretically, but nobody has shown persuasively how to do it. The problem is when anything can be unsafe, everything can. Small safe abstractions using unsafe primitives "under the hood" in a safe language are king.
You can theoretically, but nobody has shown persuasively how to do it. The problem is when anything can be unsafe, everything can. Small safe abstractions using unsafe primitives "under the hood" in a safe language are king.
how do you prevent use-after-free with a wrapper?
They're not GCing/freeing memory at all, but if they were, you free memory by trashing all your references that could use-after-free so it's not an issue.
Picking Java feels interesting given that it’s a database. I would love to know their justification for the choice.
Although - I have known a trading firm that wrote their platform in Java, offloading any disk and network access to some limited C++ and JNI. In their case I believe they simply allocated large buffers that C++ pushed and pulled data from. The benefit of this strategy was their Quants could build memory-safe strategies and in a much friendlier language. For them if eliminated the risk of bad code and lost time to debugging awful C++ errors.
To my knowledge it all worked for them quite well, they made good money and any minor differences of speed in Java were negligible to them.
Although - I have known a trading firm that wrote their platform in Java, offloading any disk and network access to some limited C++ and JNI. In their case I believe they simply allocated large buffers that C++ pushed and pulled data from. The benefit of this strategy was their Quants could build memory-safe strategies and in a much friendlier language. For them if eliminated the risk of bad code and lost time to debugging awful C++ errors.
To my knowledge it all worked for them quite well, they made good money and any minor differences of speed in Java were negligible to them.
Java tooling was excellent back when QuestDB was started and still is excellent today compared to C++.
While they do present a reasonable explanation for needing to use a different language I do agree it seem painful and the type of decision that will lead to a follow-up article in a few year: "Why we're ripping out Rust".
In some sense it reads a lot like they just needed an excuse to use Rust. Not that Rust is a bad choice for the areas they list as possible candidates for non-java code, it's just a really odd choice for Java shop, but then again, so is fighting the GC.
It is incredible fascinating work though.
In some sense it reads a lot like they just needed an excuse to use Rust. Not that Rust is a bad choice for the areas they list as possible candidates for non-java code, it's just a really odd choice for Java shop, but then again, so is fighting the GC.
It is incredible fascinating work though.
Very interesting to see how the Rust-JNI interface gets used in a production environment (e.g., the topic of unifying logging typically doesn't come up in "your first Rust-JNI app" tutorials, for instance)
I do have one question around the assignment to `static mut CALL_STATE`. Don't you need some form of synchronization/memory fence/memory barrier to make sure that other threads see that assignment?
On x86/x64 it probably doesn't matter (total store order), but other architectures are less lenient.
I do have one question around the assignment to `static mut CALL_STATE`. Don't you need some form of synchronization/memory fence/memory barrier to make sure that other threads see that assignment?
On x86/x64 it probably doesn't matter (total store order), but other architectures are less lenient.
We have an initialisation step as soon as we load the jni lib that takes care of this. Given that this gets done before any other threads are started, I don't think there'd be an issue. Good point :-)
I'm not sure I understand why they're using java if they avoid GC? Doesn't sound like the best fit, especially the foreign memory in java isn't too pleasant to work with.
"Hard to write but easy for customers to deploy" is my guess. There are a bunch of very high-performance computing use-cases in finance (quant, HFT) that Java gets used for pretty routinely. That's a very attractive market to build primitives like databases for, they have very deep pockets, but you need to play in their ecosystem.
I've seen this "GC-less" Java in those use-cases quite a bit. From a conceptual design POV it's likely not the best approach, but there's a lot of sunk cost in that eco-system and a lot of trust and expertise where "Choosing a better language" is often several orders of magnitude more expensive.
I've seen this "GC-less" Java in those use-cases quite a bit. From a conceptual design POV it's likely not the best approach, but there's a lot of sunk cost in that eco-system and a lot of trust and expertise where "Choosing a better language" is often several orders of magnitude more expensive.
They're not the only ones to have done this in this space. VoltDB (Michael Stonebreaker of Postgres [among other things] fame) did this -- low or no-GC style Java, effectively non-idiomatic Java, but taking advantage of the Java runtime in other ways.
Others have done the same. And as others have pointed out, there's things outside the DB domain in high frequency trading and the like that have done this as well.
There are advantages to Java: mature runtime, large talent pool out there, good tooling (still haven't seen anything as good as JMX for any other runtime). And if there's any language whose GC could be tuned to be "responsible", it'd be the JVM; there's been more GC R&D in the JVM than in any other runtime.
I worked at RelationalAI (another DB vendor) for a bit, and their DB is all written in Julia, another garbage collected language... and the GC in Julia is what I'd characterize as ... immature... for that kind of application. I would have loved to have access to the JVM's GC there.
Also this looks to be more of an analytical, column oriented, database. So I can imagine they're optimizing more for throughput than transactional latency. (I could be wrong, correct me, Quest folks...)
And choice of Java likely has to do with when they began working on the project and what was out there at the time. It's the real world of software eng. We work with the tools and people we have because shipping a product on time and bringing in $$ is more important than anything else. I don't know when they got started, but Rust has only matured to "mainstream" stability/acceptance in the last 2-3 years.
Finally, DBs often have a very layered architecture and theyt could easily compartmentalize pieces such that latency sensitive bits could be done in native Rust. They're not apparently doing this, but I could see them doing things like moving the page buffer or column indices or storage engine over to Rust over time for performance benefits.
All power to them, it's great to see them working with Rust. (aside: my email history looks like I spoke to a recruiter there at some point, maybe, but didn't interview? I think if I'd known they were playing with Rust I would have given that more attention...)
Others have done the same. And as others have pointed out, there's things outside the DB domain in high frequency trading and the like that have done this as well.
There are advantages to Java: mature runtime, large talent pool out there, good tooling (still haven't seen anything as good as JMX for any other runtime). And if there's any language whose GC could be tuned to be "responsible", it'd be the JVM; there's been more GC R&D in the JVM than in any other runtime.
I worked at RelationalAI (another DB vendor) for a bit, and their DB is all written in Julia, another garbage collected language... and the GC in Julia is what I'd characterize as ... immature... for that kind of application. I would have loved to have access to the JVM's GC there.
Also this looks to be more of an analytical, column oriented, database. So I can imagine they're optimizing more for throughput than transactional latency. (I could be wrong, correct me, Quest folks...)
And choice of Java likely has to do with when they began working on the project and what was out there at the time. It's the real world of software eng. We work with the tools and people we have because shipping a product on time and bringing in $$ is more important than anything else. I don't know when they got started, but Rust has only matured to "mainstream" stability/acceptance in the last 2-3 years.
Finally, DBs often have a very layered architecture and theyt could easily compartmentalize pieces such that latency sensitive bits could be done in native Rust. They're not apparently doing this, but I could see them doing things like moving the page buffer or column indices or storage engine over to Rust over time for performance benefits.
All power to them, it's great to see them working with Rust. (aside: my email history looks like I spoke to a recruiter there at some point, maybe, but didn't interview? I think if I'd known they were playing with Rust I would have given that more attention...)
Also this looks to be more of an analytical, column oriented, database. So I can imagine they're optimizing more for throughput than transactional latency.
Yes that is the case
Yes that is the case
Seamless integration with parts that don't need to be GC-free comes to mind: they are not building an application, they are building a building block. And that building block can be used both in applications that do require the latency guarantees of GC-free as well as in applications that don't. Another class of applications would be ones that alternate between phases of unpredictable latency (like bootup or reconfiguration) and low-latency operation.
I remember reading that the founder was working in low-latency Java development with London investment banks for years. I guess it's what he knew.
Also, Rust is a hard language to start a company with so I wouldn't be surprised if this is more of a product maturity thing.
Also, Rust is a hard language to start a company with so I wouldn't be surprised if this is more of a product maturity thing.
Surely it is at least as hard to find people who know how to write Java without GC?
Presumably you can't use Hotspot so you have to write your own VM too?
Presumably you can't use Hotspot so you have to write your own VM too?
Folks with a background in electronic trading (FX, hedge funds, trading firms etc) are familiar with zero-gc Java. London / NY / HK are a good pool of talent in that respect
Yep, I ended up checking their code base. I know a little bit of Java but didn't realise that sun.misc.Unsafe existed, so it does actually look fairly straightforward to create code outside of GC (for anyone reading questdb/std/Unsafe.java seems to be where some allocations are handled). A pain I am sure, but way more manageable than I thought.
It's not necessarily outside of GC, it's that they make great efforts to avoid GC. Such as instantiating all objects at startup and holding references to avoid GC, never using new keyword, avoiding objects in favor of primitives, avoid exceptions, etc.
These systems often restart or do a full "stop the world" GC once per day.
The system is quite different than what most are used to, especially during a trend towards increasingly Functional styles with immutability as a default, etc.
Peter Lawrey[1] has some great posts/talks about his experiences in HFT.
[1] https://github.com/peter-lawrey
These systems often restart or do a full "stop the world" GC once per day.
The system is quite different than what most are used to, especially during a trend towards increasingly Functional styles with immutability as a default, etc.
Peter Lawrey[1] has some great posts/talks about his experiences in HFT.
[1] https://github.com/peter-lawrey
It's not that uncommon of move, to choose jumping through hoops with Java over writing C.
Might become less common now Rust is teaching the level it is.
Might become less common now Rust is teaching the level it is.
Having worked on writing DB internals in both Rust and in other languages, I can say that there's huge time-saving advantages to having something higher-level & garbage collected at the layer of the query parser/analyzer/compiler. The borrowing/ownership semantics can get really snaky when dealing with complicated expression trees, iteration patterns, etc.
It's fairly hard to write ergonomic interfaces for more complicated iteration patterns in Rust while still respecting safety. That's actually fine and by design, and it's possible with a lot of effort and thought but this is not as much of a concern in e.g. Java. E.g. skim the discussion on this proposed "cursor" API for Rust's stdlib BTree: https://github.com/rust-lang/rust/issues/107540
(And while Rust's enum-based algebraic types & pattern matching are nice, they're actually fairly limited when compared to what you can find in e.g. Scala or F#, Haskell, etc.)
But I think there's also huge win in doing something like the pager/buffer pool/storage/data structure/indexes layer in Rust. For safety and efficiency reasons.
It's fairly hard to write ergonomic interfaces for more complicated iteration patterns in Rust while still respecting safety. That's actually fine and by design, and it's possible with a lot of effort and thought but this is not as much of a concern in e.g. Java. E.g. skim the discussion on this proposed "cursor" API for Rust's stdlib BTree: https://github.com/rust-lang/rust/issues/107540
(And while Rust's enum-based algebraic types & pattern matching are nice, they're actually fairly limited when compared to what you can find in e.g. Scala or F#, Haskell, etc.)
But I think there's also huge win in doing something like the pager/buffer pool/storage/data structure/indexes layer in Rust. For safety and efficiency reasons.
Yeah, sounds like a lot of effort.
The only thing they say that explains it is they end up with a single jar file, whose only dependency is the JRE.
So I guess they get platform independence and easy installation.
The only thing they say that explains it is they end up with a single jar file, whose only dependency is the JRE.
So I guess they get platform independence and easy installation.
QuestDB engineer here: We use jlink to create images for selected platforms. This means not even JRE is a dependency: You unpack a tarball and you are good to go. See: https://questdb.io/docs/get-started/binaries/
Well, technically it is a dependency in that you can't target platforms there isn't a JRE for. But I take your point about simplifying installation.
Keep in mind the JNI libs are platform specific. That means available platforms are a function of what the JRE runs on AND they have built the shared lib for (and bundled in to the jar)
you know what fatjars are?
I've never heard of QuestDB until this post, but I very much like what you guys are doing.
InfluxDB 1.x had a chance to be great: the wrote a time series database that used a SQL-like dialect, they had a really nice alerting platform, they had a really nice visiualization tool. Then they abandoned all of that to jump on the hype train of "Write a new programming language!" which was the hot thing like 5 or 6 hype cycles ago. We've _never_ upgraded to their 2.x product because it literally threw away our investment.
I think if you guys get pick up where they departed you'll be tremendously successful.
InfluxDB 1.x had a chance to be great: the wrote a time series database that used a SQL-like dialect, they had a really nice alerting platform, they had a really nice visiualization tool. Then they abandoned all of that to jump on the hype train of "Write a new programming language!" which was the hot thing like 5 or 6 hype cycles ago. We've _never_ upgraded to their 2.x product because it literally threw away our investment.
I think if you guys get pick up where they departed you'll be tremendously successful.
thanks for the kind words! We want to stick with SQL - having done a few extensions to make it easier to work with time series data such as SAMPLE BY, LATEST ON, etc. Window functions that the product has been lacking for some time are next to bridge the gap vs other more mature platforms while offering something very new and unique on the performance side, especially ingestion related.
How are you feeling about the upcoming InfluxDB 3? They moved to Rust and support InfluxQL again.
I'll have to take a look. If Kapacitor is back, then I'm in.
> JNI
Have you guys benchmarked FFI in Java 21 (preview, now release) yet? :) Yes I know it's super new, but I'm curious if there is a benefit in terms of:
1. performance
2. ease of maintenance
3. ease of finding production problems
Have you guys benchmarked FFI in Java 21 (preview, now release) yet? :) Yes I know it's super new, but I'm curious if there is a benefit in terms of:
1. performance
2. ease of maintenance
3. ease of finding production problems
I needed to call Rust from Java a few years ago. The approach was build a straightforward .so and call into it via JNA. It seemed way less complicated
The rust-maven-plugin we wrote indeed also supports JNA for these simple use cases.
https://github.com/questdb/rust-maven-plugin
Compared with JNA, JNI is indeed more complex, but it's faster and has more features. It also solves the problem of calling Java from Rust.
https://github.com/questdb/rust-maven-plugin
Compared with JNA, JNI is indeed more complex, but it's faster and has more features. It also solves the problem of calling Java from Rust.
Nowadays there's a much better technology than JNI for doing this sort of thing in Java.
https://docs.oracle.com/en/graalvm/jdk/21/docs/reference-man...
https://docs.oracle.com/en/graalvm/jdk/21/docs/reference-man...
Native image is unrelated to this topic here (hence the downvotes, I guess).
Still GraalVM could be an interesting solution as Rust seems to be supported: https://www.graalvm.org/latest/reference-manual/llvm/Compili...
Still GraalVM could be an interesting solution as Rust seems to be supported: https://www.graalvm.org/latest/reference-manual/llvm/Compili...
https://github.com/oracle/graal/blob/master/substratevm/src/...
https://github.com/oracle/graal/blob/master/substratevm/src/...
And in any case:
JEP draft: Prepare to Restrict The Use of JNI https://openjdk.org/jeps/8307341
JEP 442: Foreign Function & Memory API (Third Preview) https://openjdk.org/jeps/442
https://github.com/oracle/graal/blob/master/substratevm/src/...
And in any case:
JEP draft: Prepare to Restrict The Use of JNI https://openjdk.org/jeps/8307341
JEP 442: Foreign Function & Memory API (Third Preview) https://openjdk.org/jeps/442
The focus of the article really is about JNI in Rust.
I see most questions are about "Why did you not use X language instead?", so let me try and address this.
To answer the "Why not just Rust", I should first mention that Rust was still in its early days (before 1.0), and it was a risky bet to choose an emerging language.
The project was started by Vlad (our CEO) who had a background writing high performance Java in the trading space. The Zero-GC techniques - whilst uncommon in open source software - are mature and a staple of writing high performance code in the financial industry. The product evolved organically, feature after feature.
I personally joined the team from a C and C++ background, having previously moved from a project that suffered from minute-long compile times from single .cpp files due to template overuse. Whilst I do miss how expressive high-level C++ can be, Java has really good tooling support.
When writing systems-style software most of what matters in terms of performance is how we call system calls, manage memory and debug and profile. This is an area where Java really shines. Don't get me wrong: In the absolute sense I think C++ tools tend to be better (Linux Perf is awesome!), but Java tooling is _there_. IntelliJ makes it trivially easy to run a test under the debugger reliably and consistently. It's equally easy to run a profiler and to get code coverage. The same tools work across all platforms too, might I add. It's not necessarily better, but it's easier. Turns out that while a little quaint, using Java turned out to be a pretty good choice in my opinion in practice.
Times have moved on. The Rust community really cares about tooling, and it's one of the reasons why we've picked it over expanding our existing C++ codebase: We just want to get stuff done and have enough time left in our dev cycle to properly debug and profile our code.