Kafka vs RabbitMQ vs NATS: Enterprise 2026

TL;DR

• Apache Kafka: event log + streaming + audit. Ekosistem terbesar, retention panjang. Default enterprise event-driven.
• RabbitMQ: work queue + RPC + routing kompleks. Pattern AMQP matang. Default untuk task processing.
• NATS JetStream: control plane low-latency + lightweight pub/sub. Default untuk microservice coordination.
• Redpanda: Kafka API compatible, performance unggul, ekosistem developing.
• Verdict: Conditional — pilih per pattern. Enterprise modern Indonesia: campur ketiga dengan boundary jelas.

Konteks

Saya operate ketiga messaging system paralel 26 bulan di 2 konteks:

• Fintech series-B Jakarta (Maret 2024 - Mei 2026): full stack Kafka + RabbitMQ + NATS JetStream
• BUMN energy Jakarta (Januari 2024 - Desember 2024): Kafka self-host + RabbitMQ untuk legacy integration ESB

Pengalaman saya sebelumnya: Kafka 6 tahun (sejak versi 0.10), RabbitMQ 8 tahun (sejak versi 3.5), NATS 2,5 tahun.

Pricing (Juni 2026)

Apache Kafka self-host

• Software gratis (Apache 2.0)
• 3-broker cluster di GKE Jakarta:
  • 3 node n2-standard-4 (4 vCPU, 16GB RAM, 500GB SSD): Rp 5,8 juta/bulan
  • Schema Registry (Confluent OSS): 2 pod e2-small Rp 920 ribu/bulan
  • Kafka Connect 2 worker: Rp 1,4 juta/bulan
  • MirrorMaker 2 untuk DR ke Singapore region: Rp 1,2 juta/bulan
• Total: Rp 9,3 juta/bulan

Confluent Cloud (managed) — sebagai pembanding

• Basic cluster: USD 0,11/hour + USD 0,06/GB ingress = ~USD 80-300/bulan untuk small
• Dedicated cluster: USD 3.000-12.000/bulan untuk enterprise scale
• Untuk skala saya (200k msg/s sustained): Standard tier ~USD 1.800/bulan = Rp 28,8 juta/bulan
• 3x lebih mahal dari self-host

RabbitMQ self-host

• Software gratis (MPL)
• 3-node cluster di GKE Jakarta n1-standard-2 (2 vCPU, 7,5GB RAM): Rp 2,8 juta/bulan
• Storage SSD 200GB: Rp 380 ribu/bulan
• Total: Rp 3,2 juta/bulan

NATS JetStream self-host

• Software gratis (Apache 2.0)
• 3-node cluster di GKE Jakarta e2-small: Rp 1,4 juta/bulan
• Storage SSD 100GB: Rp 190 ribu/bulan
• Total: Rp 1,6 juta/bulan

Total messaging stack

Komponen	Cost/bulan
Kafka self-host (cluster + Connect + MirrorMaker)	Rp 9,3 juta
RabbitMQ self-host (cluster)	Rp 3,2 juta
NATS JetStream self-host (cluster)	Rp 1,6 juta
Monitoring (kminion, RabbitMQ exporter, NATS exporter)	Rp 400 ribu
Total	Rp 14,5 juta/bulan

Bandingkan kalau Kafka di Confluent Cloud Dedicated + RabbitMQ CloudAMQP Production + NATS Synadia: ~Rp 45-65 juta/bulan equivalent. Self-host 3-4x lebih murah dengan trade-off ops time.

SLO + performance (26 bulan data)

Apache Kafka

Metrik	Target	Realisasi
Producer ack latency p50 (acks=all)	< 20ms	12ms
Producer ack latency p99	< 50ms	38ms
Consumer lag hot consumer p99	< 5 detik	3,2 detik
Throughput sustained per cluster	> 200k msg/s	280k peak
Availability	> 99,95%	99,93%
Data loss event	0	0
Broker failover time	< 60 detik	28-42 detik

RabbitMQ

Metrik	Target	Realisasi
Publish latency p99	< 30ms	18ms
Message delivery p99 (work queue)	< 100ms	65ms
Throughput sustained	> 30k msg/s	45k peak
Availability	> 99,9%	99,91%
Queue depth saat overload	< 100k	78k peak

NATS JetStream

Metrik	Target	Realisasi
Publish latency p99 (in-cluster)	< 5ms	2,8ms
Publish latency p99 (cross-region)	< 30ms	18ms
Throughput sustained per node	> 100k msg/s	180k peak
Availability	> 99,9%	99,96%
Stream replication lag	< 1 detik	380ms

Architecture per pattern

Pattern 1: Event log + audit (Kafka)

Payment Service ──→ Kafka topic: payment-events (retention 30 hari)
                       │
                       ├─→ Audit Sink ──→ Postgres audit_log (compliance)
                       ├─→ Analytics ──→ BigQuery (BI)
                       ├─→ Notification ──→ NATS (fanout ke email + SMS + push)
                       └─→ ML Feature Store ──→ Redis

Why Kafka:

• Retention panjang (30 hari hot, 90 hari cold via tiered storage)
• Multiple consumer dengan offset independent
• Exactly-once semantics dengan transaction
• Compliance audit-friendly (event immutable, replayable)

Pattern 2: Work queue (RabbitMQ)

API Service ──→ RabbitMQ exchange: tasks
                       │
                       ├─→ queue: email (worker pool 5)
                       ├─→ queue: pdf-generate (worker pool 3, priority)
                       ├─→ queue: report-export (worker pool 2)
                       └─→ DLQ: failed-tasks (manual review)

Why RabbitMQ:

• Priority queue native (untuk urgent task)
• Routing kompleks via exchange (topic, fanout, direct)
• Dead letter queue first-class
• Per-consumer prefetch tuning untuk load balancing

Pattern 3: Control plane (NATS)

Configuration Service ──→ NATS subject: config.update.*
                            │ (subscribers in 18 service)
                            └─→ All service instances refresh config
                                    (latency < 5ms in-cluster)

Why NATS:

• Latency rendah (< 5ms in-cluster)
• Subject-based routing fleksibel
• Lightweight (binary ~30MB, footprint tipis)
• Request-reply pattern native untuk microservice RPC

Capability comparison

Capability	Kafka	RabbitMQ	NATS JetStream
Message ordering	per-partition	per-queue	per-stream
Retention	configurable (default 7 hari)	until consumed	configurable
Replay	ya (offset-based)	tidak (atau manual)	ya
Throughput sustained	200k-500k msg/s	30k-50k msg/s	100k-300k msg/s
Latency p99	30-50ms	15-40ms	1-10ms
Priority queue	tidak	ya	tidak
Routing kompleks	partition + key	exchange + binding	subject hierarchical
Dead letter	manual implement	first-class	first-class
Exactly-once	ya (transaction)	at-most-once tipikal	at-least-once + dedup
Consumer group	ya	competing consumer	queue group
Pub/sub fanout	multiple consumer group	fanout exchange	subject wildcard
Cross-region replication	MirrorMaker	Federation/Shovel	cluster mirror
Schema management	Schema Registry	manual	manual
Connector ekosistem	Kafka Connect (200+)	Shovel + custom	NATS bridge
Operational complexity	tinggi	sedang	rendah
Observability tooling	mature	mature	developing

Trade-off arsitektural

Pilih Kafka kalau:

• Event sourcing / event log pattern (audit trail compliance OJK)
• Throughput > 50k msg/s sustained
• Multiple consumer dengan replay capability
• Streaming analytics (Flink, Kafka Streams, ksqlDB)
• Integration ekosistem (Kafka Connect ke 200+ sistem)
• Retention panjang penting (compliance, replay scenario)

Pilih RabbitMQ kalau:

• Work queue / task processing
• Priority queue + routing kompleks
• Throughput < 50k msg/s (cukup untuk most workload non-streaming)
• Operational complexity preference: medium
• Dead letter handling first-class
• AMQP standard penting (integrasi legacy enterprise)

Pilih NATS JetStream kalau:

• Latency < 10ms kritis
• Control plane / coordination microservice
• Request-reply pattern native
• Footprint kecil (edge deployment, IoT)
• Ekosistem tidak deep (cukup pub/sub + stream)

HA + DR setup

Kafka 3-broker cluster

broker-1 (asia-southeast2-a) ─┐
broker-2 (asia-southeast2-b)   ├── 3-replica per partition
broker-3 (asia-southeast2-c) ─┘
              │
              └─ MirrorMaker ──→ Singapore DR cluster
                                  (active-passive, RPO ~30 detik)

• RPO: 0 (synchronous replication 3 broker)
• RTO broker failover: 28-42 detik
• RTO regional failover (DR Singapore): 8-15 menit (consumer re-balance + DNS swap)
• Synchronous replication acks=all + min.insync.replicas=2

RabbitMQ HA

node-1, node-2, node-3 (quorum queue raft consensus)
              │
              └─ Federation upstream ke Singapore DR (lag < 5 detik)

• Quorum queue (Raft) untuk strong consistency
• RPO: 0 untuk quorum queue
• RTO failover: 15-30 detik
• Mirror queue legacy DEPRECATED — pakai quorum queue

NATS JetStream

• 3-node cluster R3 replication
• RPO: 0
• RTO: < 10 detik
• Geographic replication via mirror stream

Performance tuning

Kafka producer tuning untuk fintech transaksi

acks=all                     # data durability
enable.idempotence=true      # exactly-once
max.in.flight.requests.per.connection=5
compression.type=zstd        # bandwidth save 60%
linger.ms=5                  # batch window
batch.size=32768            # 32KB batch
buffer.memory=67108864       # 64MB

Trade-off: linger.ms=5 add 5ms latency untuk batch — acceptable untuk throughput gain 3-5x.

Kafka consumer tuning

fetch.min.bytes=10240       # 10KB min fetch
fetch.max.wait.ms=500       # 500ms max wait
max.poll.records=500        # batch size
session.timeout.ms=30000    # 30 detik
heartbeat.interval.ms=3000  # 3 detik heartbeat

RabbitMQ tuning

# Prefetch per consumer (saves overload)
channel.basic_qos(prefetch_count=50)

# Connection: 1 per service, multiplex channel
# Channel: 1 per consumer worker

# Quorum queue config
x-queue-type=quorum
x-quorum-initial-group-size=3

NATS JetStream tuning

# Stream config
storage: file
replicas: 3
retention: limits
max_age: 24h
max_bytes: 10GB
duplicate_window: 2m  # dedup window

Cost of ownership 36 bulan

Skenario: fintech series-B, 18 service, 280k msg/s peak Kafka, 45k msg/s RabbitMQ, 180k msg/s NATS.

Item	Self-host all	Confluent + CloudAMQP + Synadia	Hybrid (self-host Kafka, managed RabbitMQ)
License/SaaS 3 tahun	Rp 0	Rp 2,4 miliar	Rp 720 juta
Infra 3 tahun	Rp 522 juta	Rp 48 juta	Rp 380 juta
Ops time engineer	Rp 470 juta	Rp 95 juta	Rp 285 juta
Migration/setup one-time	Rp 80 juta	Rp 35 juta	Rp 55 juta
Incident attributed to messaging	Rp 28 juta	Rp 0	Rp 18 juta
Total 3 tahun	Rp 1,10 miliar	Rp 2,58 miliar	Rp 1,46 miliar

Self-host pure paling murah cash, tapi ops effort tinggi. Managed pure paling mahal tapi 0 ops worry. Hybrid sweet spot kalau ada constraint mixed.

Saya pilih self-host all untuk fintech series-B karena ada 1 SRE dedicated yang fokus messaging infra. Worth-nya jelas: saves Rp 1,48 miliar dalam 3 tahun.

Common pitfall

1. Kafka partition count terlalu kecil. Throughput per topic = throughput per partition. Untuk topic 50k msg/s: minimum 12 partition (assume 5k/s per partition realistic). Tambah partition tidak bisa di-shrink, plan dari awal.
2. RabbitMQ classic mirror queue (deprecated). Quorum queue adalah standar 2023+. Migrate dari mirror queue ke quorum queue: 1-2 minggu kerja per service.
3. NATS JetStream tanpa dedup window. At-least-once delivery + dedup window mandatory untuk exactly-once semantics. Set duplicate_window 2 menit minimum.
4. Schema evolution Kafka tanpa Schema Registry. Producer/consumer schema drift = corrupt data. Pakai Schema Registry + Avro/Protobuf compatibility check di CI.
5. No backpressure handling. Consumer lag accumulate = OOM. Implement circuit breaker + alert pada consumer lag > 60 detik. Pause produce kalau consumer overwhelmed.

Indonesia specific

Data residency

Self-host Kafka di asia-southeast2 GKE Jakarta = data tinggal di Indonesia. Untuk fintech regulated, ini wajib. Confluent Cloud Standard tier punya region Indonesia (announced 2025), tapi cost premium. Untuk BUMN dengan data sangat sensitif, self-host on-prem masih dipertimbangkan (saya pakai GKE Jakarta sebagai compromise).

Compliance audit log

Kafka audit log via topic dedicated (audit.events) dengan retention 90 hari hot + export ke S3 Glacier untuk 7 tahun. Total cost audit retention compliance: Rp 850 ribu/bulan ongoing + Rp 15 juta amortized.

Hiring

Kafka engineer di Jakarta: pool moderate, kompensasi senior Rp 30-50 juta/bulan. RabbitMQ engineer: pool besar (Java enterprise familiar). NATS engineer: pool kecil, kompensasi premium untuk yang berpengalaman.

Yang surprising

Setelah 26 bulan paralel: Kafka availability self-host saya 99,93% — sedikit di bawah target 99,95%. Root cause: 1 insiden Zookeeper split-brain (2024) sebelum migrate ke KRaft mode, 1 insiden disk fill di broker (monitoring alert missing). Setelah migrate ke Kafka 3.6+ KRaft (no Zookeeper) di Q3 2025, availability naik ke 99,97%.

Surprise lain: NATS JetStream ternyata jadi favorit tim engineering untuk service coordination. Footprint kecil + ops simple bikin engineer comfortable. Pattern yang muncul: NATS untuk hampir semua pub/sub coordination, Kafka khusus untuk audit + analytics streaming. Boundary ini lebih jelas dari awal saya design.

Verdict

Conditional dengan rule per pattern:

• Event log + audit + streaming analytics: Apache Kafka. Default untuk fintech / banking Indonesia.
• Work queue + RPC + routing kompleks: RabbitMQ. Default untuk task processing.
• Control plane + low-latency coordination: NATS JetStream. Default untuk microservice coordination.
• Self-host vs managed: self-host kalau ops capacity ada + ada 1 SRE dedicated. Managed kalau speed-to-market prioritas.
• Redpanda: evaluate untuk service Kafka existing yang hit performance wall. Skip untuk greenfield kecuali tim familiar.

Threshold konkret untuk adopt multi-broker stack: 8+ microservice + 3+ event pattern berbeda + ops capacity 25+ jam/bulan. Di bawah ini, single broker (Kafka atau RabbitMQ saja) sudah cukup.