Instrumenting would have been 100% just as feasible. Ironically it would have been more work.
For context, our DB is highly optimised for ingestion (millions of rows / second) and adding any high resolution metrics there would impact performance, so would have to be either ripped out afterwards or engineered very carefully (read "not cheap") as to not impact performance.
This stuff took an afternoon, is reusable, and frankly, was more fun to implement :-).
I suspect there are tools out there that that do this stuff, of course. The matter is still finding and learning how to use them compared to writing a few hundred lines of code.
Maybe we should write a blog post, though there's gotta be one out there for this already.
The short of it is: Learn C. Learn your system calls. Learn JNI. Learn about com.sun.misc.Unsafe. Learn about the disruptor pattern. Learn how to pool objects. The long type can pack a lot of data. Go from there!
I'd say both Rust and C++ trade blows when it comes to expressiveness. You know Rust already, so I'm not going to try sell you how powerful macros can be (see SQLx's ability to compile-time check SQL queries).
And indeed C++ templates are a lot more like Rust macros than Rust generics: They're turing-complete.
Joint with some interesting language choices like SFINAE (substitution failure is not an error), you end up with the ability to specialize functions, methods and whole classes in C++.
You can also have functions that return different types.
C++ templates work like duck-typing within a static language: In Rust you need to say what traits your generics need to support. In C++ it will try and substitute and if it fails (say, because T doesn't support the required methods) it will try another substitution until none are left.
If none of the substitutions work, you will be shown ALL of them in error reporting: This is what leads to pages and pages of compile errors of single-character typos in C++.
Templates are really cool, but also pretty confusing when reading code since you're in a guessing game of what types will fit the constraints imposed by the _implementation_ of the function.
From C++20 there's concepts to make templates work a little smoother.
There's been whole books written about how to abuse templates: these are pre-requisite knowledge when working in large codebases.
We have an initialisation step as soon as we load the jni lib that takes care of this. Given that this gets done before any other threads are started, I don't think there'd be an issue. Good point :-)
I'd say expressiveness of C++ with productivity of Java.
Rust is indeed not easy to learn.
A little example. Yesterday I was making changes to our C++ client library and I wanted to improve the example in our documentation.
We use a dedicated protocol called ILP for streaming data ingestion and each of the inserted rows has a designated timestamp.
In the Rust example, I using added support for chrono::DateTime and it was trivially easy for me to add a timestamp for a specific example date and time: Utc.with_ymd_and_hms(1997, 7, 4, 4, 56, 55).
Our C++ library instead takes an std::chrono::time_point. I wanted to use the same datetime. As far as I can tell it requires first going through the old C "struct tm" type (which is local and not UTC), then converting to "time_t" then converting to utc via gmtime and then constructing a time_point from that.
After 10 minutes the code got too long and complicated so I just substituted it a timestamp specified as an int64_t in nanoseconds.
Don't get me wrong, the C++ time_point is a work of art in how flexible it is, but unnecessarily complicated in most cases.
I should add that I also spent 45 minutes yesterday debugging a CMake issue.
Rust is not easy to learn, but it's just more modern and productive.
C++ is still great if you've got a massive team, but at our scale I don't think it makes any sense.
When the time is right. There's finally new APIs coming in the Java space that will make native-code interop easier and more reliable.
Our open source database edition can also be used embedded though, so we can only upgrade at the pace of our customers and because of that we still are compatible all the way down to Java 8.
Were it not for this detail, we'd probably consider it a lot sooner.
The focus of the article really is about JNI in Rust.
I see most questions are about "Why did you not use X language instead?", so let me try and address this.
To answer the "Why not just Rust", I should first mention that Rust was still in its early days (before 1.0), and it was a risky bet to choose an emerging language.
The project was started by Vlad (our CEO) who had a background writing high performance Java in the trading space. The Zero-GC techniques - whilst uncommon in open source software - are mature and a staple of writing high performance code in the financial industry. The product evolved organically, feature after feature.
I personally joined the team from a C and C++ background, having previously moved from a project that suffered from minute-long compile times from single .cpp files due to template overuse. Whilst I do miss how expressive high-level C++ can be, Java has really good tooling support.
When writing systems-style software most of what matters in terms of performance is how we call system calls, manage memory and debug and profile. This is an area where Java really shines. Don't get me wrong: In the absolute sense I think C++ tools tend to be better (Linux Perf is awesome!), but Java tooling is _there_. IntelliJ makes it trivially easy to run a test under the debugger reliably and consistently. It's equally easy to run a profiler and to get code coverage. The same tools work across all platforms too, might I add. It's not necessarily better, but it's easier. Turns out that while a little quaint, using Java turned out to be a pretty good choice in my opinion in practice.
Times have moved on. The Rust community really cares about tooling, and it's one of the reasons why we've picked it over expanding our existing C++ codebase: We just want to get stuff done and have enough time left in our dev cycle to properly debug and profile our code.
Instrumenting would have been 100% just as feasible. Ironically it would have been more work.
For context, our DB is highly optimised for ingestion (millions of rows / second) and adding any high resolution metrics there would impact performance, so would have to be either ripped out afterwards or engineered very carefully (read "not cheap") as to not impact performance.
This stuff took an afternoon, is reusable, and frankly, was more fun to implement :-).
I suspect there are tools out there that that do this stuff, of course. The matter is still finding and learning how to use them compared to writing a few hundred lines of code.