Performance & Scalability — Microservices Interview

Target: Senior Engineer · Engineering Lead · Pre-Architect Focus: Bottleneck diagnosis, auto-scaling, latency optimization, caching

---

Q: Your system shows high latency only during peak hours. How do you identify the bottleneck?

Why interviewers ask this: Production latency issues are complex. Tests your ability to methodically diagnose across layers (network, service, database, JVM).

Answer

Diagnosis pyramid (test from top down):

Network latency (1-10ms)?
  ↓ DNS, TCP handshake, TLS
Service latency (10-100ms)?
  ↓ Handler logic, serialization
Database latency (50-500ms)?
  ↓ Query time, locks, I/O
JVM overhead (5-50ms)?
  ↓ GC pauses, thread contention

Tools & metrics:

Layer	Tool	Metric
End-to-end	Distributed tracing (Jaeger)	p50, p95, p99 latency per service
Database	Slow query log, EXPLAIN PLAN	Query time, lock waits
JVM	-XX:+PrintGCDetails	GC pause duration, frequency
System	top, iostat, netstat	CPU, memory, disk I/O, network
Thread pool	Spring Boot Actuator	Active threads, queue depth

Spring Boot diagnostic code:

@RestController
public class DiagnosticController {

    @GetMapping("/api/orders/{id}")
    public Order getOrder(@PathVariable String id) {
        long start = System.nanoTime();

        try {
            // Database call — measure separately
            long dbStart = System.nanoTime();
            Order order = orderRepository.findById(id).orElseThrow();
            long dbTime = System.nanoTime() - dbStart;

            log.info("Order lookup: {}ms", dbTime / 1_000_000);
            return order;
        } finally {
            long total = System.nanoTime() - start;
            log.info("Total latency: {}ms", total / 1_000_000);
        }
    }
}

Peak hours diagnosis checklist:

• [ ] Distributed trace shows which service is slow
• [ ] Database slow query log identifies problematic queries
• [ ] jstat -gc shows if GC pauses spike during load
• [ ] Thread pool metrics show saturation (queue_depth > 0)
• [ ] Network latency within expected range (< 50ms)

---

Q: You notice uneven load distribution across instances. What could be wrong?

Answer

Load balancer issues:

Problem	Sign	Fix
Sticky sessions misconfigured	Some instances get 80% traffic	Remove session affinity or use shared session store (Redis)
Health check failing	Healthy instance marked down	Verify /health endpoint is working
Round-robin only	No awareness of instance load	Switch to least-connections or weighted algorithm
DNS caching	Requests go to old instance	Reduce DNS TTL, use service discovery
Colocation	Instances on same physical host	Check infrastructure layout, spread replicas

Kubernetes load balancing example:

apiVersion: v1
kind: Service
metadata:
  name: order-service
spec:
  selector:
    app: order-service
  type: ClusterIP
  sessionAffinity: None  # Disable sticky sessions
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 10800
  ports:

    - port: 80
      targetPort: 8080
  loadBalancerAlgorithm: leastconn  # Use least-connections

Monitoring:

Per-instance metrics:

- Instance A: 50% CPU, 8K req/sec
- Instance B: 25% CPU, 4K req/sec ← Uneven!
- Instance C: 75% CPU, 12K req/sec

Action: Check if Instance B is slow, remove from pool, rebalance

---

Q: A database becomes the bottleneck. How do you optimize?

Answer

Optimization hierarchy:

1. Query optimization

   - Add indexes, use EXPLAIN
   - Avoid N+1 queries
   - Batch operations

2. Caching

   - Redis for hot data
   - Cache-aside pattern
   - Invalidation strategy

3. Read replicas

   - Offload reads to read-only followers
   - Trade consistency for throughput

4. Sharding

   - Partition by tenant or key
   - Requires app-level routing

Query optimization checklist:

-- BEFORE (slow):
SELECT o.* FROM orders o
WHERE o.customer_id = ?;
-- No index → table scan

-- AFTER (fast):
CREATE INDEX idx_orders_customer_id ON orders(customer_id);
-- Now uses index → O(log N)

-- EXPLAIN shows:
Index Seek (good) vs Table Scan (bad)

Caching pattern:

@Cacheable(value = "products", key = "#productId")
public Product getProduct(String productId) {
    // Only called if cache miss
    return productRepository.findById(productId).orElseThrow();
}

@CacheEvict(value = "products", key = "#productId")
public void updateProduct(String productId, Product update) {
    productRepository.save(update);
}

Read replica routing:

@Repository
public class OrderRepository {

    // Write to primary
    public Order save(Order order) {
        return primaryDataSource.save(order);
    }

    // Read from replica
    public Optional<Order> findById(String id) {
        return replicaDataSource.findById(id);
    }
}

---

Q: A sudden traffic spike crashes services. How do you scale and stabilize?

Answer

Multi-layer response:

Spike detected (CPU > 80%, errors rising)?
├─ Immediate (< 1 sec)
│  ├─ Rate limiting: reject new requests
│  ├─ Load shedding: drop low-priority traffic
│  └─ Circuit breaker: stop calling failing services
├─ Short-term (10-60 sec)
│  ├─ Auto-scaling: spin up new pods
│  ├─ Message queue: buffer requests
│  └─ Cache: serve stale data
└─ Long-term (> 1 min)
   ├─ Database optimization
   ├─ Code profiling & optimization
   └─ Infrastructure changes

Kubernetes auto-scaling:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: order-service-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: order-service
  minReplicas: 3
  maxReplicas: 20
  metrics:

    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30
      policies:

        - type: Percent
          value: 100  # Double replicas
          periodSeconds: 30
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:

        - type: Percent
          value: 50   # Reduce by 50%
          periodSeconds: 60

---

Diagram — Complete Scaling Architecture

graph LR
    Traffic["Traffic Spike\n100x normal"]
    RateLimit["Rate Limiter\n· Reject excess"]
    Queue["Message Queue\n· Buffer requests"]
    HPA["HPA\n· Scale 3→20 pods"]
    Cache["Cache\n· Serve stale data"]
    DB["Database\n· Read replicas"]

    Traffic -->|Phase 1: Block| RateLimit
    Traffic -->|Phase 2: Queue| Queue
    HPA -->|Phase 3: Scale| HPA
    Cache -->|Phase 4: Degrade| Cache
    DB -->|Phase 5: Distribute| DB

    style RateLimit fill:#ff6b6b
    style Queue fill:#ffe066
    style HPA fill:#51cf66
    style Cache fill:#4ecdc4

---

Caching

---

Q: What caching strategies exist for microservices, and how do you choose between them?

Why interviewers ask this: Caching is one of the most impactful performance levers, but the wrong strategy causes stale data bugs, cache stampedes, or wasted memory. Tests understanding of cache write patterns and their trade-offs.

Answer

Three write strategies:

Strategy	Write Flow	Read Flow	Consistency	Complexity
Cache-Aside (Lazy)	App writes DB, invalidates cache	Miss → load from DB, populate cache	Eventual	Low
Write-Through	App writes cache AND DB synchronously	Always cache hit (after first write)	Strong	Medium
Write-Behind (Write-Back)	App writes cache only; cache flushes to DB async	Cache hit	Eventual (risk of loss on crash)	High

Cache-Aside — most common pattern in microservices:

@Service
public class ProductService {

    @Autowired private ProductRepository repo;
    @Autowired private RedisTemplate<String, Product> redis;

    private static final Duration TTL = Duration.ofMinutes(10);

    public Product getProduct(String id) {
        String key = "product:" + id;

        // 1. Check cache
        Product cached = redis.opsForValue().get(key);
        if (cached != null) return cached;

        // 2. Cache miss — load from DB
        Product product = repo.findById(id).orElseThrow();

        // 3. Populate cache with TTL
        redis.opsForValue().set(key, product, TTL);
        return product;
    }

    public void updateProduct(String id, Product updated) {
        repo.save(updated);
        redis.delete("product:" + id);  // Invalidate — next read reloads from DB
    }
}

Spring @Cacheable (declarative cache-aside):

@Service
public class OrderService {

    @Cacheable(value = "orders", key = "#id",
               condition = "#id != null",
               unless = "#result == null")
    public Order getOrder(String id) {
        return orderRepository.findById(id).orElseThrow();
    }

    @CacheEvict(value = "orders", key = "#order.id")
    public Order updateOrder(Order order) {
        return orderRepository.save(order);
    }
}

Cache stampede (thundering herd) — a critical production problem:

Problem:
  Key expires. 500 concurrent threads all miss cache simultaneously.
  All 500 hit the database at once → DB overload → cascade failure.

Solutions:
  1. Probabilistic early expiration: refresh before TTL expires (with low probability)
  2. Mutex/lock: first thread acquires lock, loads from DB, others wait for cache
  3. Stale-while-revalidate: serve stale data, refresh in background

// Mutex pattern — only one thread reloads on miss
public Product getProductWithMutex(String id) {
    String key = "product:" + id;
    String lockKey = "lock:product:" + id;

    Product cached = redis.opsForValue().get(key);
    if (cached != null) return cached;

    // Acquire distributed lock (SET NX EX)
    Boolean locked = redis.opsForValue().setIfAbsent(lockKey, "1", Duration.ofSeconds(5));
    if (Boolean.TRUE.equals(locked)) {
        try {
            Product product = repo.findById(id).orElseThrow();
            redis.opsForValue().set(key, product, Duration.ofMinutes(10));
            return product;
        } finally {
            redis.delete(lockKey);
        }
    } else {
        // Another thread is loading — wait briefly and retry
        Thread.sleep(50);
        return getProductWithMutex(id);
    }
}

CDN and edge caching:

Cache Layer	Where	What to Cache	TTL
Browser cache	Client	Static assets (JS, CSS, images)	1 year (with content hash in URL)
CDN (CloudFront, Fastly)	Edge PoP	Public API responses, static assets	Minutes to hours
API Gateway cache	Gateway	Authenticated responses per user	Seconds to minutes
Application cache (Redis)	App tier	DB query results, computed aggregates	Minutes to hours
Database query cache	DB layer	Row data, often best left to app tier	Auto-managed

Cache eviction policies (Redis):

LRU (Least Recently Used)  — evict oldest-accessed items (general purpose)
LFU (Least Frequently Used) — evict least-used items (better for skewed access)
TTL / volatile-lru         — evict expired keys first, then LRU
allkeys-lru                — evict any key (when no TTL set)

graph LR
    Client["Client"]
    CDN["CDN · Edge Cache"]
    GW["API Gateway Cache"]
    Redis["Redis · App Cache"]
    DB["Database"]

    Client -->|1 · Request| CDN
    CDN -->|HIT: serve| Client
    CDN -->|MISS| GW
    GW -->|HIT: serve| CDN
    GW -->|MISS| Redis
    Redis -->|HIT: serve| GW
    Redis -->|MISS| DB
    DB -->|Load · populate cache| Redis

    style CDN fill:#4ecdc4
    style Redis fill:#ffe066
    style DB fill:#ff6b6b

Architect Insight

Cache invalidation is famously hard. Prefer short TTLs + cache-aside over explicit invalidation for most microservices. Only use write-through or write-behind when you can afford the complexity and truly need strong consistency between cache and DB. For public read-heavy APIs (product catalog, pricing), multi-layer caching (CDN + Redis) can reduce DB load by 95%+.

---

High-Performance Data Management

---

Q: What are the differences between horizontal and vertical scaling? When do you choose each?

Why interviewers ask this: Tests practical scaling judgment — not just knowing the definitions but knowing the limits and cost profiles of each approach.

Answer

	Vertical Scaling (Scale Up)	Horizontal Scaling (Scale Out)
Definition	Add more CPU/RAM/disk to one machine	Add more instances behind a load balancer
Limit	Hardware ceiling (largest instance type)	Virtually unlimited
Cost	Expensive at high end, no linear pricing	Roughly linear cost
Downtime	Often requires restart	Zero downtime (rolling)
Failure risk	Single point of failure	Distributed — partial failures tolerated
Stateful apps	Easier — no shared state issue	Harder — session/state must be externalised

Decision guide:

Vertical (scale up) when:
  - Single-threaded workloads that can't parallelize (legacy batch jobs)
  - Stateful databases that are hard to shard (start here first)
  - Quick short-term fix while designing horizontal scaling

Horizontal (scale out) when:
  - Stateless services (REST APIs, workers) — default choice
  - Handling burst traffic (scale to N, back to 1)
  - High availability requirement — no single point of failure
  - Cost efficiency at scale

graph LR
    VS["Vertical Scale\nBigger Machine"]
    HS["Horizontal Scale\nMore Machines"]
    LB["Load Balancer"]
    I1["Instance 1"]
    I2["Instance 2"]
    I3["Instance 3"]

    VS -->|Single node limit| VS
    HS --> LB
    LB --> I1
    LB --> I2
    LB --> I3

Architect Insight

For microservices, design all stateless services for horizontal scaling from day one — externalise all state (sessions to Redis, files to S3, config to Config Server). Vertical scaling is a database strategy until you're ready to shard.

---

Q: How do you design database partitioning and sharding for a high-traffic microservice?

Why interviewers ask this: A single database node is always the bottleneck at scale. Tests understanding of data distribution strategies, trade-offs, and consistency implications.

Answer

Partitioning splits a single database table into smaller pieces. Two types:

Type	How	Use Case
Horizontal (row-based)	Rows split across partitions by key range or hash	Time-series data, user data by ID range
Vertical (column-based)	Columns split — hot columns in one table, cold in another	Large BLOB columns, audit fields

Sharding takes horizontal partitioning further — data is distributed across separate database nodes:

Strategies:
1. Range-based: user IDs 1–1M → Shard A, 1M–2M → Shard B
   ✅ Simple queries on ranges   ❌ Hot shards if traffic skewed to recent data

2. Hash-based: shard = hash(userId) % N
   ✅ Even distribution          ❌ Range queries require all shards

3. Directory-based: lookup table maps key → shard
   ✅ Flexible rebalancing       ❌ Lookup table is a bottleneck/SPOF

Spring Boot with sharding (routing datasource):

public class ShardRoutingDataSource extends AbstractRoutingDataSource {

    @Override
    protected Object determineCurrentLookupKey() {
        String userId = ShardContext.getUserId();
        int shard = Math.abs(userId.hashCode()) % 4;
        return "shard-" + shard;
    }
}

// Usage
@Transactional
public Order findOrder(String userId, String orderId) {
    ShardContext.setUserId(userId);     // Route to correct shard
    return orderRepository.findById(orderId).orElseThrow();
}

Challenges to address at interview:

Challenge	Solution
Cross-shard joins	Denormalize, or use scatter-gather queries
Cross-shard transactions	Saga pattern — no distributed 2PC across shards
Shard rebalancing	Consistent hashing minimises data movement on resize
Hot shards	Add salt to key, or use virtual nodes (consistent hashing)

---

Q: How do you approach index optimization for slow database queries?

Why interviewers ask this: Query performance is the most common production bottleneck. Tests systematic diagnosis and knowledge of index internals.

Answer

Diagnosis first — always run EXPLAIN ANALYZE:

EXPLAIN ANALYZE
SELECT * FROM orders
WHERE customer_id = 'cust-123'
  AND status = 'PENDING'
ORDER BY created_at DESC;

-- Look for:
-- Seq Scan → no index being used
-- High rows_removed → high filter cost, composite index needed
-- Sort → missing index on ORDER BY column

Index selection rules:

Scenario	Index Type	Rationale
Single column equality filter	B-tree	Default, most queries
Multi-column filter (WHERE a AND b)	Composite index (a, b)	Put most selective column first
Range + equality (WHERE status=X AND date>Y)	Composite (status, date)	Equality columns first
Full-text search	Full-text / GIN index	LIKE '%term%' can't use B-tree
JSON fields (PostgreSQL)	GIN on JSONB	Fast containment queries
Low-cardinality columns alone	Avoid B-tree	status with 3 values — table scan faster

Index anti-patterns to call out:

-- ❌ Function on indexed column kills the index
SELECT * FROM orders WHERE LOWER(email) = 'user@example.com';

-- ✅ Fix: functional index
CREATE INDEX idx_orders_email_lower ON orders (LOWER(email));

-- ❌ SELECT * forces full table read even with index
SELECT * FROM orders WHERE customer_id = 'cust-123';

-- ✅ Fix: covering index includes all needed columns
CREATE INDEX idx_orders_customer_covering
  ON orders (customer_id) INCLUDE (status, created_at, amount);

Architect Insight

Indexes speed up reads at the cost of write overhead and storage. For write-heavy tables, benchmark before adding indexes. In microservices, favour denormalisation and read models (CQRS) over complex multi-join queries with many indexes.

---

Q: How do you model data in NoSQL? What are the key differences from relational modeling?

Why interviewers ask this: NoSQL modeling is fundamentally different — most candidates transpose relational thinking and get poor performance. Tests access-pattern-first design.

Answer

Core principle: In SQL you model data, then write queries. In NoSQL you model access patterns, then design the schema around them.

Relational vs NoSQL comparison:

	Relational (SQL)	NoSQL (DynamoDB/Cassandra)
Model around	Data relationships	Access patterns
Joins	✅ Server-side joins	❌ Must pre-join (denormalise)
Normalization	3NF — eliminate redundancy	Deliberate duplication for query speed
Schema	Fixed schema	Flexible / schema-per-item
Query flexibility	Any ad-hoc query	Only pre-designed access paths

DynamoDB single-table design example (orders + items):

Access patterns:
  1. Get order by orderId
  2. Get all items for an order
  3. Get all orders for a customer

Table design (single-table):
PK              | SK                    | Attributes
CUSTOMER#cust1  | ORDER#ord123          | {status, total, createdAt}
ORDER#ord123    | ITEM#item001          | {productId, qty, price}
ORDER#ord123    | ITEM#item002          | {productId, qty, price}

GSI: SK = ORDER#ord123 → fetch entire order with all items in one query

// Spring Data DynamoDB — query by access pattern
@DynamoDBTable(tableName = "orders")
public class OrderItem {
    @DynamoDBHashKey  String pk;   // "ORDER#ord123"
    @DynamoDBRangeKey String sk;   // "ITEM#item001"
    String productId;
    int quantity;
}

// Get all items for order in one call
List<OrderItem> items = dynamoDBMapper.query(OrderItem.class,
    new DynamoDBQueryExpression<OrderItem>()
        .withHashKeyValues(new OrderItem("ORDER#ord123", null))
        .withKeyConditionExpression("pk = :pk")
        .withExpressionAttributeValues(Map.of(":pk", new AttributeValue("ORDER#ord123")))
);

Common Mistake

Don't model NoSQL tables the way you'd model relational tables. A common mistake is creating separate tables for each entity and then "joining" in application code. This results in N+1 query problems and defeats NoSQL's purpose. Design for your top 5 access patterns first.

---