Fairly straightforward overall using https://crates.io/crates/kafka. The client doesn't automatically handle Kafka node failures, so that's the responsibility of your application code.
I've been using Rust to process fairly large amounts of streaming financial data. I am using Protocol Buffers (and evaluating Cap'n Proto) for network serialization and Kafka for buffering/queuing, and overall the library support is quite good.
Previously I was using JVM languages for this purpose, but grew weary of the resource footprint, and especially the unpredictable GC pauses. I am aware of the Azul JVM which removes GC pauses and of various Java techniques to avoid GC altogether, but switching to Rust provided a GC-less model from the ground-up, a powerful type system, and familiar functional programming facilities at no cost.
> Overall though, Go is probably still the more practical choice between the two languages (due to Rust's incredibly high barrier to entry).
Does Rust really have an "incredibly high barrier to entry?"
I've been using Rust for a few months, and just deployed my first high-throughput application a month ago, and my experience has been the opposite. Yes, the first couple of weeks were a bit rough while I was getting used to the ownership system, but since then I have been progressing at a relatively quick pace. The package and dependency management facilities are incredibly good, and I've found high-quality libraries for nearly all my initial needs.
Compilation times could be faster, but the error messages provided by the compiler are so useful that I have come to depend on compilation errors for refactoring. The gains in predictable performance and resource utilization have far outweighed any initial cognitive overhead in the learning process. The community and the resources they provide are fantastic.
Coming from a mixed dynamic language and functional programming background, I could see room for improving certain FP aspects of the language, but am impressed with the pervasive pattern matching and collection handling.
Not a knock on Go, but rather an endorsement of Rust and its future.
Similar story here. Been using Rust in production for only a couple of weeks for a financial analytics pipeline processing billions of events per day and TBs of data, but absolutely no issues so far. The predictable performance of Rust has been wonderful to observe, compared to GC'ed languages this iteration of the pipeline has replaced (JVM, Go). The type system makes otherwise daring refactorings a joy.
I haven't had the chance to try https://github.com/fede1024/rust-rdkafka yet, but it looks promising and partially wraps the C/C++ library https://github.com/edenhill/librdkafka.