HTTP vs. Kafka Load Testing Patterns¶
Modern distributed architectures combine synchronous request-response APIs (HTTP/gRPC) with asynchronous event-streaming backbones (Apache Kafka). Load testing both requires understanding how their latency boundaries, failure modes, and success criteria differ fundamentally.
Architectural Comparison¶
sequenceDiagram
participant C as Gatling Client
participant API as Order HTTP Service
participant K as Kafka Broker
participant W as Downstream Consumer (Inventory)
rect rgb(235, 245, 255)
Note over C,API: Pattern A — Synchronous HTTP (End-to-End Request/Response)
C->>API: POST /api/orders
API-->>C: 201 Created (Client measures full round-trip: 145ms)
end
rect rgb(245, 240, 235)
Note over C,W: Pattern B — Asynchronous Kafka (Decoupled Ack vs. Consumer Lag)
C->>K: Produce OrderCreated (acks=all)
K-->>C: RecordMetadata Ack (Producer measures broker ack: 8ms)
K->>W: Poll Batch
Note over W: Process & Commit Offset (Monitored via Consumer Lag!)
end
| Dimension | HTTP Simulations (Labs 1–5) | Kafka Simulations (Labs 6–7) |
|---|---|---|
| Communication Model | Synchronous Request-Response | Asynchronous Publish-Subscribe |
| Gatling Support | Native HttpDsl (http("...").get(...)) |
Custom exec(session -> ...) invoking KafkaProducer |
| What Gatling Latency Shows | Full client-to-server round-trip time | Producer-to-Broker acks latency (NOT downstream consumer processing!) |
| Primary Overload Symptom | Rising P95/P99 HTTP latency, 503/504 timeouts |
Growing Consumer Group Lag (records-lag-max), broker request queue saturation |
| Routing & Ordering | Stateless URL / Load-balancer round-robin | Message Key hashing (murmur2(key) % numPartitions) preserves per-entity ordering |
| Sync vs. Async Send | Non-blocking Netty I/O under the hood | Sync send().get() (Lab 6) vs. Async send(record, callback) batching (Lab 7) |
Sync vs. Async Kafka Producer Patterns¶
In Lab 6 (Sim06_BasicKafkaProducer) and Lab 7 (Sim07_KafkaWithFeeders), you will compare both producer patterns:
Waits for broker acknowledgment on every single message before advancing:
.exec(session -> {
try {
producer.send(new ProducerRecord<>(topic, key, payload)).get();
return session.markAsSucceeded();
} catch (Exception e) {
return session.markAsFailed();
}
})
linger.ms), capping throughput per thread.
Leverages Kafka's native batching (batch.size + linger.ms) for massive throughput:
Producer Green != System Healthy!
A Kafka broker can effortlessly ingest 50,000 messages/second in 5 ms while your downstream Spring Kafka consumer only processes 1,000 messages/second! If you only check Gatling's producer report, the test looks 100% green while consumer lag balloons by 49,000 messages every second. Always monitor kafka.consumer_group.lag alongside Kafka producer tests!
Knowledge Check¶
During a Kafka producer load test at 10,000 messages/sec, Gatling reports 0% errors and P99 send latency of 12 ms, yet customers complain order emails arrive 45 minutes late. Why?
Next Steps¶
- Read next: Scenarios & Feeders — Parameterize HTTP and Kafka user journeys with CSV, JSON, and custom feeders in Java DSL.