Workload Modeling & Coordinated Omission¶
Designing an accurate performance test requires more than picking a target RPS. If your workload model does not match how real traffic arrives at your system—or if your load generator suffers from Coordinated Omission—your test can report green SLAs while your production system would melt down under the exact same traffic.
1. Open vs. Closed Workload Models¶
In performance engineering theory (see Schroeder, Wierman & Harchol-Balter, Open Versus Closed: A Cautionary Tale), systems fall into three workload classes:
flowchart TB
subgraph Open["Open Workload Model (Arrival-Driven)"]
direction LR
Internet["Unbounded Population"] -->|"Fixed Arrival Rate λ (e.g. 200/s)"| ServerO["Server Under Test"]
ServerO --> Leave["Users Exit"]
end
subgraph Closed["Closed Workload Model (Concurrency-Driven)"]
direction LR
Pool["Fixed Pool: N Users"] -->|"Send Request"| ServerC["Server Under Test"]
ServerC -->|"Response"| Think["Think Time (Z)"]
Think -->|"Loop Back"| Pool
end
Detailed Comparison¶
| Dimension | Open Workload Model | Closed Workload Model |
|---|---|---|
| Control Variable | Arrival rate ($\lambda$) — new users/requests per second | Concurrency ($N$) — active users in the loop |
| Behavior When Server Slows Down | Arrivals keep coming at $\lambda$; concurrency $L$ spikes, queues fill, timeouts surface realistically | Existing $N$ users wait on slow responses; arrival rate $\lambda$ automatically drops, masking overload! |
| Real-World Match | Public websites, e-commerce checkouts, mobile APIs, webhooks, Kafka event streams | Call-center terminals, internal back-office apps with 50 seated employees, fixed worker thread pools, DB connection pools |
| Gatling DSL | .injectOpen(constantUsersPerSec(100).during(300)) |
.injectClosed(constantConcurrentUsers(100).during(300)) |
| Coordinated Omission Risk | Low — arrivals are scheduled independently of prior completions | High if misused to simulate public web traffic |
How Gatling Expresses Both Models¶
setUp(
checkoutScenario.injectOpen(
nothingFor(5), // (1)!
rampUsersPerSec(10).to(200).during(120), // (2)!
constantUsersPerSec(200).during(600) // (3)!
)
).protocols(httpProtocol);
- Initial quiet window before traffic starts.
- Linearly increase arrival rate from 10 to 200 new virtual users/sec regardless of server response time.
- Sustain 200 arrivals/sec for 10 minutes—if the server stalls, active concurrent users grow naturally per Little's Law.
setUp(
backOfficeScenario.injectClosed(
rampConcurrentUsers(0).to(50).during(60), // (1)!
constantConcurrentUsers(50).during(600) // (2)!
)
).protocols(httpProtocol);
- Ramp up to a capped pool of 50 concurrent operators.
- Whenever 1 user finishes their scenario, Gatling immediately spawns 1 replacement so active concurrency stays pinned at 50.
Never Use a Closed Model for Public Internet Traffic
If 5,000 shoppers click "Buy Now" during a flash sale and your database stalls for 5 seconds, new shoppers do not politely wait in their living rooms for previous shoppers' HTTP requests to finish! An Open Model reproduces the resulting queue explosion; a Closed Model throttles itself and hides the outage.
2. Coordinated Omission¶
Coined by Gil Tene (CTO of Azul Systems) in his famous talk "How NOT to Measure Latency", Coordinated Omission is one of the most pervasive measurement bugs in performance testing history.
How the Bug Happens¶
Suppose you configure a load generator thread (or closed user) to send 1 request every 100 ms (10 RPS). - Normally, the server responds in 5 ms—well within the 100 ms slot. - For 59 seconds, the thread sends $59 \times 10 = 590$ requests, each taking 5 ms. - At second 60, the server hits a 1-second Stop-The-World JVM GC pause (1,000 ms).
Watch what a synchronous/closed client does vs. what real-world arrivals experience:
sequenceDiagram
participant Gen as Naive Closed Generator (1 req / 100ms)
participant Srv as Server (1,000ms GC Freeze at t=0.1s)
Gen->>Srv: Req #1 at t=0.0s
Srv-->>Gen: 200 OK at t=0.005s (5ms)
Gen->>Srv: Req #2 at t=0.1s
Note over Srv: FULL GC PAUSE (1,000ms)<br/>Reqs at 0.2s, 0.3s ... 1.0s<br/>ARE NEVER SENT!
Srv-->>Gen: 200 OK for Req #2 at t=1.1s (1,000ms)
Gen->>Srv: Req #3 at t=1.2s
Srv-->>Gen: 200 OK at t=1.205s (5ms)
The Numerical Distortion¶
During that 60-second window:
| Reality (Un-omitted Open Arrivals) | Naive Closed / Synchronous Measurement |
|---|---|
| 600 requests should have arrived (10/s × 60s) | Only 591 requests were sent (9 were omitted!) |
| Request at $t=0.1\text{s}$ waited 1,000 ms | Request #2 recorded 1,000 ms |
| Request at $t=0.2\text{s}$ waited 900 ms | Omitted — never measured |
| Request at $t=0.3\text{s}$ waited 800 ms | Omitted — never measured |
| ... down to $t=1.0\text{s}$ waiting 100 ms | Omitted — never measured |
| 10 out of 600 requests (1.67%) suffered severe stalls | 1 out of 591 requests (0.17%) recorded a stall! |
| True P99 Latency = ~500 ms! | Reported P99 Latency = 5 ms! |
Reported P99 Was Off by 100x!
Because the load generator coordinated with the server's pause by backing off and omitting the 9 requests that should have queued up during the freeze, 99.83% of recorded samples were 5 ms—making P99 look like 5 ms instead of 500 ms! Worse, the longer the freeze (say, 10 seconds = 100 omitted requests), the smaller a single stalled sample looks in the percentage count!
How to Prevent Coordinated Omission in Gatling¶
- Use Open Injection (
injectOpen): Gatling's event-loop scheduler injects fresh virtual users on wall-clock schedule regardless of whether earlier users are blocked waiting on responses. - Avoid Looping Requests Inside a Fixed User Pool Without Pacing: If a single virtual user loops (
forever().on(...)), a slow iteration delays all subsequent iterations of that user. - Never Block Gatling's Netty Event Loop: In custom protocol actions (like Kafka), use async callbacks or bounded pools so generator threads never stall.
3. Think Time vs. Pacing¶
Even when modeling multi-step user journeys, how you space requests determines whether throughput stays predictable as server latency changes.
Pause / Think Time (pause)¶
pause(duration) sleeps after the previous response returns:
- If $R$ increases from
100 msto900 msandpauseis1,000 ms, cycle time grows from1.1 sto1.9 s—reducing per-user request rate by 42%.
Pacing (pace)¶
pace(duration) enforces a fixed start-to-start interval between iterations by dynamically subtracting response time from the wait:
ScenarioBuilder pacedJourney = scenario("Paced Polling Client")
.repeat(20).on(
pace(Duration.ofSeconds(5)) // (1)!
.exec(http("Poll Order Status").get("/orders/#{orderId}"))
);
- If
GET /orders/#{orderId}takes200 ms, Gatling waits4,800 ms. If it takes1,500 ms, Gatling waits3,500 ms—keeping iteration cadence locked at 1 request every 5 seconds until $R > 5\text{ s}$.
Realistic Pause Distributions¶
Real humans do not all wait exactly 5.000 seconds—synchronized pauses create artificial traffic waves (heartbeat synchronization). Always randomize pauses:
| Strategy | Gatling Java DSL | Characteristics |
|---|---|---|
| Uniform Range | .pause(Duration.ofSeconds(3), Duration.ofSeconds(8)) |
Evenly distributed between min and max |
| Exponential (Poisson) | .pause(Duration.ofSeconds(5), exponentialPauses) |
Memoryless inter-arrival times matching real random arrivals |
| Normal / Gaussian | .pause(Duration.ofSeconds(5), normalPausesWithStdDevDuration(Duration.ofSeconds(1))) |
Bell-curve human reading times |
4. Realistic User Mixes & Markov Chains¶
Production traffic is never 100% checkouts. Modeling realistic read/write ratios, cache-hit skew (Pareto / Zipfian 80/20 distribution), and abandonment funnels is critical because database locks, caches, and connection pools behave completely differently under mixed workloads.
Probabilistic Funnel with randomSwitch¶
stateDiagram-v2
[*] --> Browse: 100% Users
Browse --> Search: 70%
Browse --> Bounce: 30% (Exit)
Search --> AddToCart: 35%
Search --> ReadOnlyExit: 65% (Exit)
AddToCart --> Checkout: 50%
AddToCart --> CartAbandon: 50% (Exit)
Checkout --> [*]
In Gatling, you can model this either as weighted concurrent scenarios inside setUp() or via randomSwitch() inside a single journey:
ScenarioBuilder realisticShopper = scenario("Shopper Funnel")
.feed(productFeeder)
.exec(http("1. Home & Catalog").get("/products"))
.pause(2, 6)
.randomSwitch().on(
percent(70.0).then( // (1)!
exec(http("2. Product Detail").get("/products/#{productId}"))
.pause(3, 8)
.randomSwitch().on(
percent(30.0).then( // (2)!
exec(http("3. Add to Cart").post("/cart"))
.pause(2, 5)
.exec(http("4. Checkout").post("/checkout"))
)
)
)
);
- 70% of visitors view a product detail page; 30% bounce after the catalog.
- Of the 70% who view details, 30% ($0.70 \times 0.30 = 21\%$ overall) proceed to cart & checkout, producing an ~80/20 read-to-write ratio.
Knowledge Check¶
Why is using a Closed Workload Model dangerous when load testing a public e-commerce website?
What causes the "Coordinated Omission" measurement flaw?
Which Gatling construct dynamically adjusts wait time so each loop iteration starts on a fixed 10-second schedule (as long as response time is under 10 seconds)?
Next Steps¶
- Continue: Load Testing Methodology — Apply these laws and workload models in an end-to-end performance testing workflow.