Microservices Performance: Patterns, Anti-Patterns, and Distributed Debugging

When Microservices Collide: The Cascade Effect

Scenario: User Service calls Order Service calls Payment Service. Payment Service is slow (5s timeout). User Service has no timeout.

The cascade:

User Service starts request → calls Order Service
Order Service starts request → calls Payment Service  
Payment Service: 5s delay
Order Service waits 5s → no response from User Service
User Service waits 5s → no response from Order Service

1 slow user = 1 blocked thread
100 slow users = 100 blocked threads
1000 slow users = Thread pool exhausted

Result: System collapse

Cost: $500k+ in downtime, reputation damage, customer churn.

---

Pattern 1: Timeout Everywhere

What It Is

@RestClient
public interface OrderServiceClient {
    @Timeout("5s")  // Max wait time
    @Retries(3)      // Retry 3 times before giving up
    Order getOrder(@PathParam Long id);
}

Why It Matters

Without timeout:

• Slow service → all threads blocked → cascading failure
• Response time: 5000ms (unacceptable)
• Thread pool exhaustion

With timeout:

• Slow service → fail fast → 5000ms request fails after 5s
• Circuit breaker opens → return cached data
• Response time: degraded but stable
• System stays alive

Configuration

restclient:
  timeout: 5000          # 5 second timeout
  connection-timeout: 2000
  read-timeout: 5000
  write-timeout: 5000
  retry-attempts: 3
  retry-delay: 100ms

---

Pattern 2: Circuit Breaker

States

CLOSED (normal)
  ↓ (3 failures in 10s)
OPEN (reject requests)
  ↓ (30s later)
HALF_OPEN (test 1 request)
  ↓ (success: back to normal)
CLOSED

Implementation (Resilience4j)

@Configuration
public class CircuitBreakerConfig {
    @Bean
    public CircuitBreaker orderServiceBreaker() {
        CircuitBreakerConfig config = CircuitBreakerConfig.custom()
            .failureRateThreshold(50)  // Open after 50% failures
            .slowCallRateThreshold(50)  // Open after 50% slow calls
            .waitDurationInOpenState(Duration.ofSeconds(30))
            .build();
        return CircuitBreaker.of("order-service", config);
    }
}

@Service
public class OrderService {
    @CircuitBreaker(name = "order-service")
    public Order getOrder(Long id) {
        // Calls external Order Service
        // If 50% fail → circuit opens
        // Return cached/fallback data
    }
}

Benefits

• Fast fail: Don't wait 5s, fail in 100ms
• System alive: Return degraded response (cached data)
• Self-healing: Automatically close after 30s
• No cascading: Prevent propagation to upstream services

---

Pattern 3: Bulkhead

What It Is

@Service
public class PaymentService {
    // One thread pool for payment processing (10 threads)
    private final ExecutorService paymentPool = 
        Executors.newFixedThreadPool(10);
    
    // Another pool for order queries (5 threads)
    private final ExecutorService queryPool = 
        Executors.newFixedThreadPool(5);
}

Why It Works

Without bulkhead:

Thread pool: 100 threads
Payment slow (50 threads blocked)
Order queries: Only 50 threads left
Order queries timeout too

With bulkhead:

Payment pool: 10 threads (max blocked = 10)
Order pool: 5 threads (unaffected by payment slowness)
Order queries work normally

Configuration

threadpool:
  payment:
    core-size: 10
    max-size: 20
    queue-capacity: 100
  queries:
    core-size: 5
    max-size: 10
    queue-capacity: 50

---

Anti-Pattern 1: Unbounded Thread Pools

The Problem

// ❌ Creates new thread per request
ExecutorService executor = Executors.newCachedThreadPool();
for (Request req : requests) {
    executor.submit(() -> handleRequest(req));
}

Result:

• 10k requests = 10k threads
• Thread memory: 10k × 1MB = 10GB
• OS can't handle 10k threads
• System crash

The Fix

// ✅ Bounded thread pool
ExecutorService executor = Executors.newFixedThreadPool(20);
for (Request req : requests) {
    executor.submit(() -> handleRequest(req));
}

---

Anti-Pattern 2: Synchronous Calls Chain

The Problem

// ❌ Blocks on each call
User user = userService.getUser(id);          // 200ms wait
List<Order> orders = orderService.getOrders(user.id);  // 300ms wait
Payment payment = paymentService.getPayment();  // 400ms wait

Total time: 900ms (serial)

The Fix (Async)

// ✅ Parallel requests
CompletableFuture<User> userFuture = 
    userService.getUserAsync(id);
CompletableFuture<List<Order>> ordersFuture = 
    orderService.getOrdersAsync(userId);
CompletableFuture<Payment> paymentFuture = 
    paymentService.getPaymentAsync();

CompletableFuture.allOf(userFuture, ordersFuture, paymentFuture)
    .join();

Total time: 400ms (parallel, limited by slowest)

---

Distributed Tracing: Find Root Causes

Without Tracing

API Gateway → Service A → Service B → Service C
              ↓                        ↓
            200ms                   5000ms (slow!)
            
Where is the problem?
ServiceA: looks fine
ServiceB: looks fine  
ServiceC: 5000ms

But why? Database? Network? Lock contention?

With Distributed Tracing (Jaeger/Zipkin)

Trace ID: 12345abc
  │
  ├─ API Gateway: 5200ms
  │   ├─ ServiceA: 200ms
  │   │   ├─ Parse request: 10ms
  │   │   ├─ Validate: 20ms
  │   │   └─ Call ServiceB: 170ms (waits)
  │   │
  │   └─ ServiceB: 5000ms
  │       ├─ Database query: 4900ms ← ROOT CAUSE
  │       ├─ Transform: 50ms
  │       └─ Call ServiceC: 50ms

Found it in 30 seconds!

---

Monitoring Checklist

✅ Latency

• P50, P95, P99
• Track per endpoint
• Alert if P99 > SLA

✅ Error Rates

• Track by service
• Alert if > 1%
• Correlate with deployments

✅ Throughput

• Requests/second
• Track per service pair
• Alert on unexpected drops

✅ Dependencies

• Circuit breaker status
• Thread pool utilization
• Queue depth

---

Real Numbers: Impact of Patterns

Before (No patterns)

One slow service → System collapse
Downtime/month: 8 hours
Customer impact: 100%
Cost: $500k/month

After (All patterns)

One slow service → Degraded but alive
Downtime/month: 0
Customer impact: 5% (stale data)
Cost: $0

---

Summary

Microservices performance comes from:

1. Timeouts everywhere (fail fast)
2. Circuit breakers (self-healing)
3. Bulkheads (isolation)
4. Distributed tracing (root cause analysis)
5. Monitoring (alert on problems)

Master these, and your microservices stay alive under load.

Get started: joptimize.io