Designing L1 and L2 Caches

When the database can't keep up, the first instinct is to add Redis. When Redis itself starts straining, it's time for multi-level caching. This post covers the most common pattern: the L1 + L2 cache design.

Why Multi-Level Caching

In internet-scale systems, the database tends to become the performance bottleneck as the business grows. If a flood of requests hits the database directly, you get high latency, an overloaded database, and eventually exhausted connections and slow queries. That's why virtually every high-concurrency system introduces a cache to boost performance.

The most common setup uses Redis as a unified caching layer. Client requests hit Redis first; on a cache hit, the data is returned directly, and on a miss, the system queries the database and writes the result back into the cache. This alone can dramatically reduce database load.

But as the system keeps scaling, relying on Redis alone runs into new problems. Under high QPS, a huge number of requests hitting Redis simultaneously generates network overhead and puts pressure on Redis CPU. Meanwhile, certain hot keys get read constantly, and paying a network round trip to Redis every single time adds unnecessary latency.

To solve these problems, many large systems adopt a multi-level cache architecture, and the most common form is the L1 + L2 cache design.

The Basic L1/L2 Cache Architecture

L1/L2 caching is essentially a tiered caching architecture, typically with two layers.

L1 Cache

The L1 cache lives inside the application process and is also known as a local cache. For example:

• Go: bigcache / ristretto
• Java: Caffeine / Guava Cache
• Node: LRU Cache

Its defining characteristics:

• Extremely fast access (in-memory)
• No network communication
• Each service instance has its own copy

L2 Cache

The L2 cache is typically a distributed caching system — most commonly Redis or Memcached — shared by all service instances.

Its defining characteristics:

• Shared across nodes
• Better data consistency
• Larger capacity
• Supports distributed deployment

The overall access flow: client request → service process → check L1 cache → on miss, check L2 cache → on miss again, query the database.

The L1/L2 Access Flow

In a real system, a typical request flows like this.

First, the request enters the application service, which checks the L1 cache. If the data is there, it returns immediately — the lowest-latency path, usually on the order of microseconds.

On an L1 miss, the system queries the L2 cache (Redis). If Redis has the data, it returns the result and also writes it into the L1 cache, so subsequent requests can hit locally.

If Redis doesn't have it either, the cache has fully missed, and the system goes to the database. Once the database returns, the result is written into both Redis and the L1 cache, establishing fresh cache entries.

The whole flow boils down to:

L1 Cache → L2 Cache → Database

This way, the vast majority of requests are intercepted at the cache layers and never touch the database.

Why Two Tiers

The two cache tiers each solve different problems, and combining them delivers a major performance boost.

Reducing Redis Load

In a high-concurrency system, if every request goes to Redis, Redis itself can become the new bottleneck. With an L1 cache, hot data is served straight from application memory, cutting the number of Redis requests.

Cutting Network Overhead

Talking to Redis means a network round trip. Even at 1ms of latency, that adds up under high QPS. The L1 cache is an in-process lookup — far faster than any network call.

Increasing Throughput

With an L1 cache, the system can handle more requests without adding load to Redis or the database.

Faster Access to Hot Data

In many business scenarios, a small set of hot data gets read constantly — popular products, user profiles, configuration data, recommendation results. The L1 cache serves these locally.

Pitfalls of L1 Cache Design

The L1 cache performs beautifully, but it introduces problems of its own.

1) Data Consistency

Since each service instance holds its own L1 cache, the copies across instances can drift apart. When data is updated, without a proper invalidation mechanism you may serve stale reads. The usual fix is a cache invalidation strategy — on update, notify all service instances to evict their local copies via a message queue or a pub/sub mechanism.

2) Capacity Control

The L1 cache lives in application memory; if it grows unbounded, it can eat significant memory and hurt system stability. Eviction policies like LRU or LFU are typically used to cap its size.

3) Cache Penetration and Breakdown

When a flood of requests targets nonexistent data, it can slam the database directly. Bloom filters or mutex-based locking are the usual defenses.

Cache Update Strategies

In a multi-level cache system, the update strategy is critical. Common approaches:

1) Cache Aside

The application checks the cache first; on a miss it queries the database and then populates the cache. This is the most common pattern.

2) Write Through

The application updates the cache at the same time it writes to the database, keeping the two in sync.

3) Write Back

The application writes only to the cache, and the cache system asynchronously flushes to the database. This offers higher performance but is more complex to implement.

Most internet-facing workloads go with Cache Aside, because it's simple to implement and reliable.

When L1/L2 Caching Fits

Multi-level caching is a good fit for scenarios like:

• Read-heavy, high-concurrency systems: product detail pages, user profile lookups, config reads.
• Frequently accessed hot data: leaderboards, recommendation lists.
• Heavy database load: the cache layers can drastically cut database queries.

For systems with strict real-time consistency requirements — financial trading systems, for example — multi-level caching is usually avoided in favor of going straight to the database.

Wrapping Up

The core idea of L1/L2 caching is layered interception: the local cache absorbs the hottest traffic, Redis holds the data shared across instances, and the database only handles true misses. This architecture significantly reduces latency and backend load — at the cost of multi-copy consistency, which must be designed alongside invalidation notifications, capacity control, and penetration defenses. Whether multi-level caching is worth it depends on your read/write ratio and consistency requirements — for consistency-sensitive scenarios, the simpler design is often the safer one.