Announcing Zstandard in Rust(trifectatech.org)
trifectatech.org
Announcing Zstandard in Rust
https://trifectatech.org/blog/announcing-zstandard-in-rust/
9 comments
The "Why" section exclusively opens with a reference to zstd-rs:
> Why though?
> Using zstd in Rust is already supported via the zstd crate, so why bother with a whole new implementation?
Separately, I'd say their end goal seems to be replacing the C library dependency in Rust's zstd-rs:
> We have our own fork of the zstd that uses libzstd-rs-sys instead of the C library. We'd like to upstream this at some point.
> Why though?
> Using zstd in Rust is already supported via the zstd crate, so why bother with a whole new implementation?
Separately, I'd say their end goal seems to be replacing the C library dependency in Rust's zstd-rs:
> We have our own fork of the zstd that uses libzstd-rs-sys instead of the C library. We'd like to upstream this at some point.
They seems to be referering to https://github.com/gyscos/zstd-rs which looks like a wrapper around the C library.
https://github.com/KillingSpark/zstd-rs. is a pure rust implementation
That is an absolutely massive amount of unsafe, is the plan to reduce it over time? The benefits of the Rust build system are reason alone to use this (as a direct competitor to the C version), but claims about memory safety are suspect no matter how confident the authors are about correctness.
they're using c2rust. It transpiles c -> rust "directly", e.g. ints map to libc::c_int, pointers map to pointers, array access map to dereferencing pointers at an offset, etc.
Typically the suggested way to use it is to obtain "bad" rust directly from it (but which still works etc), and then make that "bad" rust more idiomatic. I don't remember any publicly visible success stories for this though. The current Bun rewrite is arguably of this form, but not exactly a "success story" in producing idiomatic, safe rust. Other large rewrites that were well-publicized/docoument, e.g. the fish shell from c++ -> Rust, took a different approach (factor into modules/rewrite each module by hand).
https://fishshell.com/blog/rustport/
Typically the suggested way to use it is to obtain "bad" rust directly from it (but which still works etc), and then make that "bad" rust more idiomatic. I don't remember any publicly visible success stories for this though. The current Bun rewrite is arguably of this form, but not exactly a "success story" in producing idiomatic, safe rust. Other large rewrites that were well-publicized/docoument, e.g. the fish shell from c++ -> Rust, took a different approach (factor into modules/rewrite each module by hand).
https://fishshell.com/blog/rustport/
I was using the original zstd rust crate for my web app, but I couldn't get my module any smaller than 150kb, but when I switched to C for web app I was able to do a whole lot of optimizing and culling until I got it down to 39kb.
I wanted to like Rust, but I was using unsafe code for performance and such anyway that it made more sense to switch to C.
I wanted to like Rust, but I was using unsafe code for performance and such anyway that it made more sense to switch to C.
> I got it down to 39kb.
If you're counting KBs and you have to include zstd yourself (a reverse proxy isn't handling it for you) does that mean you're targeting an embedded device? Are you supporting TLS at that size?
If you're counting KBs and you have to include zstd yourself (a reverse proxy isn't handling it for you) does that mean you're targeting an embedded device? Are you supporting TLS at that size?
39kb includes the zstd decoder which is statically compiled into it. I'm not targeting microcontrollers, but I am targeting smartphones in areas with bad internet connections, so every kilobyte saved is nice. I also just like to optimize things and make them minimal while still being powerful. I don't handle tls, GitHub pages handles all that, I'm just using it to host a static pwa.
https://github.com/KillingSpark/zstd-rs
And better yet if they compared it explicitly in their “why” section. As it is, I’m left guessing.