We originally introduced Rust (back in 2018) as a small experiment alongside existing systems, mostly to validate safety and performance assumptions under real production load.
Over time, the reduction in memory-related incidents and clearer failure modes led us to expand its use into increasingly critical paths. This post focuses less on “Rust is great” and more on the tradeoffs, mistakes, and organizational changes required to make that transition work in practice.
Happy to answer questions about what did not work, where Rust was a poor fit, or how we handled interop with existing systems.
If by excessive typing you mean complex trait bounds and the like, it's really something isolated to the most generic libraries, and reasonably rare. Most application code is simply typed - take a struct or an enum, return another. So it's not been a problem.
We don't use Copilot or any tool that sends our source code because of our security posture, defense in depth, etc. If and when there will be an offering that lets us run the server component and provides significant productivity improvements, we will subscribe.
With hindsight, considering the cards we were dealt, there's not much I would have done differently. If I had known better before, I would have ensured stronger buy-in because after a while our internal stakeholders were often pushing back on the effort, and that led to concessions where throw away code in the legacy systems was built even for weak business outcomes.
Overall I share your concerns. Having the right reasons to rewrite is key. I believe this blog[1] about software as theory building does a great job at describing the challenge with software gardening, and the times where a rewrite is the solution are few. Even then, it's critical to handle the rewrite in ways that can work - in our case, we chose to progressively eat the legacy software without making major changes when we could avoid them. The legacy software we had was mostly the results of one man heroics and traded off performance and availability for correctness and security. It also was designed to be maintained by a small group. Solid choices if you are early Kraken - but as many successful startups, we were victim of our success and we needed it all.
When it became clear that we had to rewrite the stack (the 2017 3-days shutdown happened just before that realization), those in charge at the time decided to experiment with Rust. It was a crazy bet in early 2018, it was still Rust 2015, no NLL, no async, far rougher ecosystem. The fact that it became successful enough to warrant pursuing a full rewrite is to credit on some lucky hires who made it a success.
In that regard, Rust was a very strong talent magnet. In my experience, having hired 200+ Rust engineers over the last 5 years, there are a few kind of engineers attracted to Rust: (1) some just like shiny things/hype, (2) some are perfectionists and never complete a project, (3) some just are doers who have found that Rust is a particularly effective language.
Overall, Rust has been a great to hire for. Many engineers out there want to use Rust, even if it's their 1st Rust professional experience. We were also known in the Rust community for hiring for full time Rust, probably also the place currently with the highest density of Rust talent (there are massive companies with more Rust devs, but smaller % overall). Budget wise, our Rust engineers are not paid particularly better than other engineers in the company, but the compensations at Kraken are generally in the higher tier.
At the risk of sounding boastful, in my experience Rust is reasonably easy to learn for experienced/strong developers (we have some very young outstanding Rust devs as well, most of the time they learned before joining). Average developers struggle and may never become productive. Again, we have an engineering excellence culture so it is okay for us, YMMV.
Re scope, yes we use Rust for everything in the backend, including CRUD type of work like Web APIs. We've found we're at least as productive than other languages (Go / Java+Spring / Ruby / PHP) while having far fewer incidents, and easier maintenance / cheaper KTLO. Rust's ability for reuse is excellent which means that there are very strong network effects when having more services in Rust, including the Web layer.
A nice "side effect" of a full Rust stack is that our p99.9 latency internally is usually stable around 3-4ms for most operations, even though multiple services are involved. That's coming from a much higher baseline with much more deviation across operations providing the same functionality (60-100ms).
Regarding your own rewrite discussions, you're not going to be convinced by a post on HN, I'll just say that I am very reluctant to even think working at a workplace that doesn't predominantly uses Rust. I've been in the industry for 20+ years, across many stacks and there's a before and an after Rust for me. It has been a super power and made our life easier. It makes it easier to model business problems thanks to algebraic data types and their usage for error handling (versus inheritance), traits allow to abstract behavior better than OOP-style interfaces, the absence of data races is a game changer for multi-threaded code, dependency management is trivial, the ecosystem is rich and things work well. A lot of these are properties found in other languages but no other has the same full package and is on its way to become mainstream.
Completed a full rewrite of many components of the Kraken.com backend in about 4 years.
The new system is around 1.5M loc of Rust. There was no serious alternative to rewriting, sometimes you find yourself in a corner and need to fix issues, and pay the price.
Everything in that blog post still rings true and hindsight is that it were were right. But it was a massive grind and required extreme dedication to get it done, for a variety of reasons that work was very taxing.
We also didn't stop feature development and kept the two systems running concurrently (which explains why it took so long, also growing and training a new team 10x the size took time, so there are many factors).
I'm also against rewrites if I can help it, but reality is complex and sometimes we can't help it. Now however, since we removed the last pieces of legacy that were preventing larger DB schema changes (or required massive, unreasonable changes to the legacy systems), we've been shipping faster and easier than ever and caught-up on a lot of the accumulated backlog, including some of the more ambitious projects that were unthinkable in the legacy systems due to limitations.
Kraken | Full remote, globally distributed | Full-time
DESCRIPTION: Kraken is one of the top cryptocurrency exchanges worldwide (and becoming more than that!). We build a lot of our modern backend code in Rust in various teams, and the Frontend in React (and React Native) and Angular. There are many more roles as we're hoping to hire 100-200 engineers (developers, SREs/DevOps, data...) in 2021. Make sure to check our careers page.
Rust roles (backend or desktop): Core Backend (the core services of the exchange), Bank (the neo-bank we are building), Futures (the Futures/derivatives trading platform), Trading Technology (trading related services). We have a trading desktop app written in Rust with the Iced framework (https://cryptowat.ch/apps/desktop). We hire backend, test and site reliability engineers starting at 2 years of experience. For dev roles, Rust experience is mandatory, but significant pet projects counts. I wrote a blog to provide more context on what the Core Backend team does and the profiles we are hiring: https://blog.kraken.com/post/7964/oxidizing-kraken-improving.... Please read the blog before applying!
LOCATION: We are fully remote+global and have been for years. We also have some offices in around the world for those who like these.
ESTIMATED COMPENSATION: Depends on the role - we base our compensation on the top percentiles of US compensations and apply the same ranges wherever you live. All our offers have option packages. For an idea of the compensation, head to levels.fyi and check Google, Amazon or Apple comps per levels.
REMOTE: Some teams have timezone restrictions: Futures are centered around UTC, other teams are mostly Eastern Europe to US West Coast.
The Core Backend team is planning to expand its presence in APAC and is hiring outstanding engineers who are senior + self-driven + advanced in Rust, who agree to adapt their working hours to maximize overlap with US or EU at least until the group gets big enough to become fully autonomous.
VISA: No visa sponsorship
CONTACT: You can apply on our career page at https://jobs.lever.co/kraken or you can DM me on Twitter (@simach) though I may take a few days to reply.
Kraken was founded in 2011, being a pioneer exchange in the crypto market. Kraken is in rapid expansion of its engineering teams. At Kraken we have a remote global workforce, allowing our employees to have more freedom in choosing how to organize their routine and balance work and personal life.
We're also hiring software engineers with the following skillsets: Backend (Rust/Go/Typescript/PHP), Frontend (React/Flux/Typescript/Sass/Less/NextJS) and many more.
We're using thiserror, and haven't explored macros that generate an enum from the Result type. I remember there are crates exploring this, but this is something that would really benefit from compiler support. Ocaml's polymorphic variants appear to provide an interesting solution here.
We originally introduced Rust (back in 2018) as a small experiment alongside existing systems, mostly to validate safety and performance assumptions under real production load.
Over time, the reduction in memory-related incidents and clearer failure modes led us to expand its use into increasingly critical paths. This post focuses less on “Rust is great” and more on the tradeoffs, mistakes, and organizational changes required to make that transition work in practice.
Happy to answer questions about what did not work, where Rust was a poor fit, or how we handled interop with existing systems.