Messaging in Rust — Kafka, AMQP & NATS

Overview

Rust's messaging map has one de-facto crate per broker, like Go's, with a wrinkle worth knowing: the Kafka default (rdkafka) is bindings over the C library librdkafka rather than a native implementation — battle-tested behaviour shared with every other librdkafka binding, at the price of a C build dependency in an otherwise pure-Rust graph. lapin (AMQP 0.9.1) and async-nats (NATS's official client) are native async Rust. All three expose the broker's real semantics directly; there is no Spring-Kafka-style framework layer, and payloads are serde at the boundary.

Key points

• rdkafka: StreamConsumer (a Stream of messages) and FutureProducer (send(...).await resolves on ack) — full librdkafka semantics: consumer groups, rebalance callbacks, transactions, exactly-once via idempotent producers. Configure with the same key/value options every librdkafka client uses.
• Offset discipline is yours: auto-commit intervals vs commit_message after processing — the same at-least-once bookkeeping as Java and Go; spawn per partition + select! on shutdown is the worker shape.
• rskafka is the pure-Rust alternative (InfluxData) — simpler feature set, no C dependency; chosen when build hygiene beats feature completeness.
• lapin models AMQP faithfully: channels, exchanges, queues, publisher confirms, consumer acks (basic_ack/basic_nack with requeue) — the delivery-vs-ack patterns from message brokers & event streaming apply verbatim; deadpool-lapin pools connections.
• async-nats is the official, first-class client: core NATS pub/sub, JetStream (persistence, consumers, acks), KV and object store — and NATS's lightweight, all-in-one posture makes it the same natural fit in a Rust shop as in a Go one (the server is Go; several NATS ecosystem pieces are appearing in Rust).
• Payloads are &[u8] + serde: serde_json for readable topics, prost/Avro (apache-avro, schema_registry_converter for Confluent SR) where schemas govern — the schema-registry story is thinner than the JVM's; protobuf-on-Kafka with prost types is the common strongly-typed choice.
• No framework indirection: retries, DLQs, poison-message handling and tracing context propagation are explicit code (tower-like middleware doesn't reach here); the compensating benefit is that nothing hides the broker's actual behaviour.

Examples

let consumer: StreamConsumer = ClientConfig::new()
    .set("bootstrap.servers", brokers)
    .set("group.id", "billing")
    .set("enable.auto.commit", "false")
    .create()?;
consumer.subscribe(&["orders"])?;

let mut stream = consumer.stream();
while let Some(msg) = stream.next().await {
    let msg = msg?;
    let order: Order = serde_json::from_slice(msg.payload().unwrap_or_default())?;
    process(order).await?;
    consumer.commit_message(&msg, CommitMode::Async)?;   // after processing
}

Related

• Backend protocols in Rust — the map — parent map.
• gRPC in Rust — the synchronous alternative for service-to-service calls.
• Message brokers and event streaming — broker-side architecture and delivery guarantees, the protocol-agnostic theory.
• API documentation in Rust — documenting these channels: hand-authored AsyncAPI and schemars payload contracts.
• Messaging in Java and in Go — the counterparts; Java adds framework layers, Go adds native-client breadth, Rust keeps the brokers raw.