Skip to content

Workload Modeling & Coordinated Omission

Designing an accurate performance test requires more than picking a target RPS. If your workload model does not match how real traffic arrives at your system—or if your load generator suffers from Coordinated Omission—your test can report green SLAs while your production system would melt down under the exact same traffic.


1. Open vs. Closed Workload Models

In performance engineering theory (see Schroeder, Wierman & Harchol-Balter, Open Versus Closed: A Cautionary Tale), systems fall into three workload classes:

flowchart TB
    subgraph Open["Open Workload Model (Arrival-Driven)"]
        direction LR
        Internet["Unbounded Population"] -->|"Fixed Arrival Rate λ (e.g. 200/s)"| ServerO["Server Under Test"]
        ServerO --> Leave["Users Exit"]
    end

    subgraph Closed["Closed Workload Model (Concurrency-Driven)"]
        direction LR
        Pool["Fixed Pool: N Users"] -->|"Send Request"| ServerC["Server Under Test"]
        ServerC -->|"Response"| Think["Think Time (Z)"]
        Think -->|"Loop Back"| Pool
    end

Detailed Comparison

Dimension Open Workload Model Closed Workload Model
Control Variable Arrival rate ($\lambda$) — new users/requests per second Concurrency ($N$) — active users in the loop
Behavior When Server Slows Down Arrivals keep coming at $\lambda$; concurrency $L$ spikes, queues fill, timeouts surface realistically Existing $N$ users wait on slow responses; arrival rate $\lambda$ automatically drops, masking overload!
Real-World Match Public websites, e-commerce checkouts, mobile APIs, webhooks, Kafka event streams Call-center terminals, internal back-office apps with 50 seated employees, fixed worker thread pools, DB connection pools
Gatling DSL .injectOpen(constantUsersPerSec(100).during(300)) .injectClosed(constantConcurrentUsers(100).during(300))
Coordinated Omission Risk Low — arrivals are scheduled independently of prior completions High if misused to simulate public web traffic

How Gatling Expresses Both Models

setUp(
    checkoutScenario.injectOpen(
        nothingFor(5),                             // (1)!
        rampUsersPerSec(10).to(200).during(120),   // (2)!
        constantUsersPerSec(200).during(600)       // (3)!
    )
).protocols(httpProtocol);
  1. Initial quiet window before traffic starts.
  2. Linearly increase arrival rate from 10 to 200 new virtual users/sec regardless of server response time.
  3. Sustain 200 arrivals/sec for 10 minutes—if the server stalls, active concurrent users grow naturally per Little's Law.
setUp(
    backOfficeScenario.injectClosed(
        rampConcurrentUsers(0).to(50).during(60),  // (1)!
        constantConcurrentUsers(50).during(600)    // (2)!
    )
).protocols(httpProtocol);
  1. Ramp up to a capped pool of 50 concurrent operators.
  2. Whenever 1 user finishes their scenario, Gatling immediately spawns 1 replacement so active concurrency stays pinned at 50.
setUp(
    apiScenario.injectOpen(
        constantUsersPerSec(500).during(300)
    )
).throttle(
    reachRps(300).in(60),
    holdFor(240)
).protocols(httpProtocol);

Never Use a Closed Model for Public Internet Traffic

If 5,000 shoppers click "Buy Now" during a flash sale and your database stalls for 5 seconds, new shoppers do not politely wait in their living rooms for previous shoppers' HTTP requests to finish! An Open Model reproduces the resulting queue explosion; a Closed Model throttles itself and hides the outage.


2. Coordinated Omission

Coined by Gil Tene (CTO of Azul Systems) in his famous talk "How NOT to Measure Latency", Coordinated Omission is one of the most pervasive measurement bugs in performance testing history.

How the Bug Happens

Suppose you configure a load generator thread (or closed user) to send 1 request every 100 ms (10 RPS). - Normally, the server responds in 5 ms—well within the 100 ms slot. - For 59 seconds, the thread sends $59 \times 10 = 590$ requests, each taking 5 ms. - At second 60, the server hits a 1-second Stop-The-World JVM GC pause (1,000 ms).

Watch what a synchronous/closed client does vs. what real-world arrivals experience:

sequenceDiagram
    participant Gen as Naive Closed Generator (1 req / 100ms)
    participant Srv as Server (1,000ms GC Freeze at t=0.1s)

    Gen->>Srv: Req #1 at t=0.0s
    Srv-->>Gen: 200 OK at t=0.005s (5ms)
    Gen->>Srv: Req #2 at t=0.1s
    Note over Srv: FULL GC PAUSE (1,000ms)<br/>Reqs at 0.2s, 0.3s ... 1.0s<br/>ARE NEVER SENT!
    Srv-->>Gen: 200 OK for Req #2 at t=1.1s (1,000ms)
    Gen->>Srv: Req #3 at t=1.2s
    Srv-->>Gen: 200 OK at t=1.205s (5ms)

The Numerical Distortion

During that 60-second window:

Reality (Un-omitted Open Arrivals) Naive Closed / Synchronous Measurement
600 requests should have arrived (10/s × 60s) Only 591 requests were sent (9 were omitted!)
Request at $t=0.1\text{s}$ waited 1,000 ms Request #2 recorded 1,000 ms
Request at $t=0.2\text{s}$ waited 900 ms Omitted — never measured
Request at $t=0.3\text{s}$ waited 800 ms Omitted — never measured
... down to $t=1.0\text{s}$ waiting 100 ms Omitted — never measured
10 out of 600 requests (1.67%) suffered severe stalls 1 out of 591 requests (0.17%) recorded a stall!
True P99 Latency = ~500 ms! Reported P99 Latency = 5 ms!

Reported P99 Was Off by 100x!

Because the load generator coordinated with the server's pause by backing off and omitting the 9 requests that should have queued up during the freeze, 99.83% of recorded samples were 5 ms—making P99 look like 5 ms instead of 500 ms! Worse, the longer the freeze (say, 10 seconds = 100 omitted requests), the smaller a single stalled sample looks in the percentage count!

How to Prevent Coordinated Omission in Gatling

  1. Use Open Injection (injectOpen): Gatling's event-loop scheduler injects fresh virtual users on wall-clock schedule regardless of whether earlier users are blocked waiting on responses.
  2. Avoid Looping Requests Inside a Fixed User Pool Without Pacing: If a single virtual user loops (forever().on(...)), a slow iteration delays all subsequent iterations of that user.
  3. Never Block Gatling's Netty Event Loop: In custom protocol actions (like Kafka), use async callbacks or bounded pools so generator threads never stall.

3. Think Time vs. Pacing

Even when modeling multi-step user journeys, how you space requests determines whether throughput stays predictable as server latency changes.

Pause / Think Time (pause)

pause(duration) sleeps after the previous response returns:

Cycle Time = Response Time (R) + Pause (Z)
  • If $R$ increases from 100 ms to 900 ms and pause is 1,000 ms, cycle time grows from 1.1 s to 1.9 s—reducing per-user request rate by 42%.

Pacing (pace)

pace(duration) enforces a fixed start-to-start interval between iterations by dynamically subtracting response time from the wait:

Dynamic Wait = max(0, Pace Interval - Response Time R)
ScenarioBuilder pacedJourney = scenario("Paced Polling Client")
    .repeat(20).on(
        pace(Duration.ofSeconds(5)) // (1)!
            .exec(http("Poll Order Status").get("/orders/#{orderId}"))
    );
  1. If GET /orders/#{orderId} takes 200 ms, Gatling waits 4,800 ms. If it takes 1,500 ms, Gatling waits 3,500 ms—keeping iteration cadence locked at 1 request every 5 seconds until $R > 5\text{ s}$.

Realistic Pause Distributions

Real humans do not all wait exactly 5.000 seconds—synchronized pauses create artificial traffic waves (heartbeat synchronization). Always randomize pauses:

Strategy Gatling Java DSL Characteristics
Uniform Range .pause(Duration.ofSeconds(3), Duration.ofSeconds(8)) Evenly distributed between min and max
Exponential (Poisson) .pause(Duration.ofSeconds(5), exponentialPauses) Memoryless inter-arrival times matching real random arrivals
Normal / Gaussian .pause(Duration.ofSeconds(5), normalPausesWithStdDevDuration(Duration.ofSeconds(1))) Bell-curve human reading times

4. Realistic User Mixes & Markov Chains

Production traffic is never 100% checkouts. Modeling realistic read/write ratios, cache-hit skew (Pareto / Zipfian 80/20 distribution), and abandonment funnels is critical because database locks, caches, and connection pools behave completely differently under mixed workloads.

Probabilistic Funnel with randomSwitch

stateDiagram-v2
    [*] --> Browse: 100% Users
    Browse --> Search: 70%
    Browse --> Bounce: 30% (Exit)
    Search --> AddToCart: 35%
    Search --> ReadOnlyExit: 65% (Exit)
    AddToCart --> Checkout: 50%
    AddToCart --> CartAbandon: 50% (Exit)
    Checkout --> [*]

In Gatling, you can model this either as weighted concurrent scenarios inside setUp() or via randomSwitch() inside a single journey:

ScenarioBuilder realisticShopper = scenario("Shopper Funnel")
    .feed(productFeeder)
    .exec(http("1. Home & Catalog").get("/products"))
    .pause(2, 6)
    .randomSwitch().on(
        percent(70.0).then( // (1)!
            exec(http("2. Product Detail").get("/products/#{productId}"))
                .pause(3, 8)
                .randomSwitch().on(
                    percent(30.0).then( // (2)!
                        exec(http("3. Add to Cart").post("/cart"))
                            .pause(2, 5)
                            .exec(http("4. Checkout").post("/checkout"))
                    )
                )
        )
    );
  1. 70% of visitors view a product detail page; 30% bounce after the catalog.
  2. Of the 70% who view details, 30% ($0.70 \times 0.30 = 21\%$ overall) proceed to cart & checkout, producing an ~80/20 read-to-write ratio.

Knowledge Check

#

Why is using a Closed Workload Model dangerous when load testing a public e-commerce website?

#

What causes the "Coordinated Omission" measurement flaw?

#

Which Gatling construct dynamically adjusts wait time so each loop iteration starts on a fixed 10-second schedule (as long as response time is under 10 seconds)?


Next Steps

  • Continue: Load Testing Methodology — Apply these laws and workload models in an end-to-end performance testing workflow.