Advanced Caching Patterns used by 2000 microservices

How do you handle 500 billion daily requests across 2000 microservices? Discover three advanced caching patterns that ensure availability and reduce latency at massive scale.

#1about 7 minutes

Why caching is critical for services at scale

Caching reduces latency, lowers infrastructure costs, and improves reliability by making services less dependent on databases or third-party services.

#2about 1 minute

Knowing when not to implement a cache

Avoid adding a cache prematurely for young products with low traffic, as it introduces unnecessary complexity, potential bugs, and additional failure points.

#3about 4 minutes

Caching critical configuration with an S3-backed cache

Use a read-through cache backed by S3 to store static, rarely updated configuration data, ensuring service startup reliability even when dependencies are down.

#4about 6 minutes

Building a dynamic LRU cache with DynamoDB and CDC

Implement a cache-aside pattern using an in-memory LRU cache backed by DynamoDB and populated via Kafka CDC streams to reduce database load for frequently accessed data.

#5about 5 minutes

Using Kafka compact topics for in-memory datasets

For smaller datasets, use Kafka's compact topics to maintain a complete, up-to-date copy of the data in-memory for each service instance.

#6about 6 minutes

Implementing an HTTP reverse proxy cache with Varnish

Use a reverse proxy like Varnish Cache with a robust invalidation strategy to dramatically reduce response times for services with expensive computations like server-side rendering.

#7about 4 minutes

A decision tree for choosing the right caching pattern

Follow a simple flowchart to select the appropriate caching strategy based on whether the data is for startup, dynamic retrieval, or stable HTTP responses.

#8about 12 minutes

Q&A on caching strategies and implementation details

The discussion covers HTTP header caching, custom invalidation logic, handling the "thundering herd" problem, and the choice of JVM for high-performance services.