Rate Limiting in Istio
After one of our production endpoints got hammered, I moved rate limiting from the application layer to the gateway layer. These notes cover the complete setup for global rate limiting on the Istio IngressGateway using Envoy RateLimit + Redis, plus a few matching details that are easy to trip over.
Background and Problem
In a microservice architecture, the system typically exposes a unified entry point through the Istio IngressGateway. When traffic spikes (a promotion, crawlers, endpoint abuse) and there's no rate limiting in place, the backend can run into problems like:
- Endpoints hit at high frequency, overloading service CPU or the database
- Traffic bursts causing system meltdown or cascading failures
- Core APIs abused by malicious callers, crowding out legitimate users
Implementing rate limiting inside each service works, but the rules end up scattered and inconsistent—and by the time a request is rejected, it has already entered the business process and consumed resources. The better approach is unified rate limiting at the gateway layer, so traffic is governed before it ever reaches the backend.
Istio's data plane is Envoy Proxy, which has this capability built in: with the Envoy RateLimit filter + a standalone ratelimit service + Redis, you get global rate limiting shared across gateway replicas, with per-path access frequency control.
How It Works
There are three actors on the path. Once you understand how they relate, the YAML below is easy to read:
-
The Envoy RateLimit filter. It sits on the IngressGateway's HTTP filter chain. For every request passing through, it builds a set of descriptors from the
actionsconfigured on the route, then asks the ratelimit service over gRPC—along with thedomain—whether the request should be allowed. -
The ratelimit service (envoyproxy/ratelimit). A standalone gRPC service that loads quota rules from a ConfigMap. When it receives descriptors, it looks up the rules, decrements the quota, and returns OK or OVER_LIMIT.
-
Redis. The storage for quota counters. Because the counters live in Redis, all gateway replicas share the same counts—that's what makes this "global" rate limiting, as opposed to local rate limiting, where each Envoy instance counts on its own.
The key to matching: the descriptor combination generated by the route's actions must correspond exactly to the key/value hierarchy of descriptors in the ConfigMap, and the domain on both sides must match. If any link is off, rate limiting fails silently and every request passes through.
When You'd Use This
The typical candidates are low-traffic but sensitive endpoints where abuse is expensive:
/api/login
/api/send-code
/api/payment
A hammered login endpoint triggers a flood of password checks and risk-control computation; an abused verification-code endpoint burns SMS fees directly; the payment endpoint touches downstream resources. These endpoints have low QPS by nature, so a very small quota blocks the vast majority of malicious traffic with minimal collateral damage.
Full Configuration
Here's a configuration you can apply directly, in four parts: a Secret for the Redis password, a ConfigMap defining quotas, the ratelimit service itself, and two EnvoyFilters—one to mount the filter, one to attach descriptors.
# --------------------------------------------
# 0) Redis password in a Secret (more secure)
# Use a Secret for sensitive values (like the Redis password) so they never
# appear in plaintext in images or Pod env var history
# --------------------------------------------
apiVersion: v1
kind: Secret
metadata:
name: ratelimit-redis-secret # Secret name, referenced by the Deployment via secretKeyRef
namespace: istio-system # Same namespace as the ratelimit service, for easy reference and management
type: Opaque
stringData:
REDIS_AUTH: "password" # K8s handles base64 encoding of the plaintext
---
# --------------------------------------------
# 1) Ratelimit quota config (runtime config for the Envoy Ratelimit Server)
# - This ConfigMap is mounted read-only into the container at /data/ratelimit/config
# - domain must exactly match the domain in the Envoy HTTP filter, or nothing matches
# - descriptors define the "dimension combinations" for limiting (formed by combined actions)
# --------------------------------------------
apiVersion: v1
kind: ConfigMap
metadata:
name: ratelimit-config
namespace: istio-system
data:
config.yaml: |
domain: ingress-ratelimit # Must match the domain in the RateLimit filter
descriptors:
- key: header_match # Top-level key: matches header_value_match in HTTP_ROUTE (action 1/2)
value: path-api # Top-level value: from descriptor_value in the route's actions
descriptors: # Second-level descriptors: further refinement (combined into a unique limit key)
- key: generic_key # Matches generic_key in HTTP_ROUTE (action 2/2)
value: istio-limit-v1 # Matches the descriptor_value set on the route
rate_limit:
unit: second # Time unit: second / minute / hour / day
requests_per_unit: 5 # Quota: requests allowed per unit of time
---
# --------------------------------------------
# 2) Ratelimit Service (connects to your existing standalone Redis)
# - Quota service implementing global sliding-window/token-bucket limits via envoyproxy/ratelimit
# - replicas=1: single replica for testing; in production run at least 2-3 replicas with HA Redis
# - readiness/liveness: health checks on port 6070
# - Key env vars: REDIS_URL / REDIS_AUTH / RUNTIME_ROOT / RUNTIME_SUBDIRECTORY
# --------------------------------------------
apiVersion: v1
kind: Service
metadata:
name: ratelimit # Resolved by Envoy via cluster name (outbound|8081||ratelimit.istio-system.svc.cluster.local)
namespace: istio-system
spec:
selector: { app: ratelimit } # Matches the Deployment's labels
ports:
- name: grpc
port: 8081 # gRPC service port (the Envoy RateLimit filter connects here)
targetPort: 8081
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ratelimit
namespace: istio-system
spec:
replicas: 1 # 1 for testing; in production use >=3 plus HPA/PodDisruptionBudget
selector:
matchLabels: { app: ratelimit }
template:
metadata:
labels: { app: ratelimit }
spec:
containers:
- name: ratelimit
image: envoyproxy/ratelimit:875d418c # Pin a commit/tag for reproducibility; consider upgrading to a newer official tag
imagePullPolicy: IfNotPresent
command: ["/bin/ratelimit"] # Entry point
ports:
- containerPort: 8080 # HTTP Admin (optional: metrics/debugging)
- containerPort: 8081 # gRPC main service port
- containerPort: 6070 # Health check port (/healthcheck)
env:
- name: REDIS_SOCKET_TYPE
value: tcp # Connect to Redis over TCP
- name: REDIS_URL
value: "redis://1.1.1.1:6379"# Your Redis address (in production, use an internal DNS name + sentinel or cluster)
- name: REDIS_POOL_SIZE
value: "20" # Connection pool size; tune for concurrency/replica count
- name: REDIS_AUTH
valueFrom:
secretKeyRef:
name: ratelimit-redis-secret # Inject the password from the Secret, no plaintext
key: REDIS_AUTH
- name: USE_STATSD
value: "false" # Enable if you have a statsd/Prometheus sidecar
- name: RUNTIME_ROOT
value: /data # Root of the mount point
- name: RUNTIME_SUBDIRECTORY
value: ratelimit # Subdirectory; final config path is /data/ratelimit/config
- name: RUNTIME_IGNOREDOTFILES
value: "true" # Ignore dotfiles
volumeMounts:
- name: config
mountPath: /data/ratelimit/config # Must match the runtime path above so config lands in the config subdirectory
readinessProbe:
httpGet:
path: /healthcheck
port: 6070
initialDelaySeconds: 2 # Initial delay to avoid false negatives during cold start
periodSeconds: 5 # Probe interval
livenessProbe:
httpGet:
path: /healthcheck
port: 6070
initialDelaySeconds: 10
periodSeconds: 10
volumes:
- name: config
configMap:
name: ratelimit-config # Mounts the ConfigMap above
---
# --------------------------------------------
# 3) Inject the global RateLimit filter into the IngressGateway + point it at the ratelimit CLUSTER
# - Uses an EnvoyFilter to insert the global ratelimit HTTP filter before the HTTP Router
# - workloadSelector: limits scope to the gateway workload (label: istio=ingressgateway)
# - rate_limit_service: connects to the ratelimit gRPC service defined above (via cluster name/authority)
# - Also appends a Lua response filter: rewrites backend 429s into a unified JSON body
# --------------------------------------------
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: gateway-global-limit-filter
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway # Only applies to IngressGateway instances
configPatches:
# 3.1 Insert the ratelimit HTTP filter before the router (applies globally)
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager # HTTP connection manager
subFilter:
name: envoy.filters.http.router # Insert before the Router
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.http.ratelimit
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
domain: ingress-ratelimit # Must match the domain in the ConfigMap
failure_mode_deny: false # Whether to reject requests when the ratelimit service is down; false=allow (recommended)
timeout: 10s # Timeout for calls to the ratelimit service
rate_limit_service:
grpc_service:
envoy_grpc:
cluster_name: outbound|8081||ratelimit.istio-system.svc.cluster.local # Cluster name auto-generated by Istio
authority: ratelimit.istio-system.svc.cluster.local # HTTP/2 Host (SNI), optional
transport_api_version: V3 # Use the v3 API (recommended)
# 3.2 Insert the Lua response filter on the port 1035 listener (optional: applies only to that listener)
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
portNumber: 1035 # Only applies to the listener on port 1035 (your gateway exposes this port)
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.http.lua
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
inlineCode: |
function envoy_on_response(handle)
local s = tonumber(handle:headers():get(":status") or "0")
if s == 429 then
local body = '{"code":429,"message":"Too many requests. Please try again later."}'
handle:headers():replace("content-type", "application/json")
handle:headers():replace("content-length", tostring(#body))
handle:body(true):setBytes(body)
end
end
# Notes:
# - If your gateway doesn't listen on 1035 (or uses multiple ports), consider widening this filter to all ports (drop the portNumber condition)
# - Alternatively, local rate limiting can generate the 429 locally on rejection, and this Lua rewrites it uniformly
---
# --------------------------------------------
# 4) Inject per-route rate limit actions into the VirtualHost
# - Configure actions on a specific Route (or the whole vhost) to generate descriptors that match the ConfigMap
# - Two actions are used here:
# a) header_value_match: when :path matches the /api prefix, emit descriptor_value=path-api
# b) generic_key: additionally emit descriptor_value=istio-limit-v1
# - Combined => (key=header_match,value=path-api) + (key=generic_key,value=istio-limit-v1)
# which matches the ConfigMap exactly, activating the quota
# --------------------------------------------
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: gateway-limit-descriptor-svc
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway # Scope limited to the gateway
configPatches:
- applyTo: HTTP_ROUTE
match:
context: GATEWAY
routeConfiguration:
vhost:
name: host:port # Watch this vhost name: usually "<host>:<port>"
patch:
operation: MERGE
value:
route:
rate_limits:
- actions:
- header_value_match:
headers:
- name: ":path" # Match the :path pseudo-header, prefix-based
prefix_match: "/api"
expect_match: true # Only add the descriptor when the match succeeds
descriptor_value: "path-api" # Matches the ConfigMap's top-level key/value
- generic_key:
descriptor_value: "istio-limit-v1" # Matches the ConfigMap's second-level key/value
A few points worth unpacking:
-
Both EnvoyFilters are required. The first one only mounts the ratelimit filter onto the filter chain and tells it where to find the ratelimit service. What actually decides "which requests carry which descriptors to the limiter" is the second one—without
rate_limits.actionson the route, the filter does nothing at all for a request. -
failure_mode_deny: false. When the ratelimit service or Redis goes down, allow requests through instead of rejecting them. Rate limiting is a protective measure; it shouldn't become a new single point of failure. Unless an endpoint is so sensitive that you'd rather have it unavailable than abused, keep this false. -
The Lua filter is a UX refinement. When Envoy's rate limit trips, it returns a bare 429 by default, which frontends can't handle uniformly. This Lua snippet rewrites the 429 body into a consistent JSON structure at the response stage—much friendlier to clients.
Pitfalls and Caveats
- A mismatched domain makes rate limiting fail silently. The
domainin the filter and thedomainin the ConfigMap must match character for character. When they don't, there's no error—the ratelimit service simply finds no rules and returns OK for everything. - Descriptor hierarchy must correspond exactly. The two-level structure in the ConfigMap—
header_match/path-apinestinggeneric_key/istio-limit-v1—maps to the order of the two actions on the route. One missing action, a wrong value, or a flipped hierarchy, and nothing matches. - The vhost name isn't arbitrary.
routeConfiguration.vhost.nameis usually of the form"<host>:<port>"and must match the vhost name in the route config Envoy actually generates. Confirm it from the gateway's config_dump; if it's wrong, the MERGE simply won't take effect. - Mind the port your EnvoyFilter matches. The Lua filter above is scoped to
portNumber: 1035; if your gateway listens on a different port or multiple ports, adjust or drop that condition, or the 429 rewrite will only apply to part of your traffic. - Availability of the ratelimit service itself. A single replica is fine for testing; run multiple replicas in production and plan for Redis high availability too. Tune
REDIS_POOL_SIZEfor your concurrency and replica count. - EnvoyFilter is a fairly low-level extension mechanism. It patches Envoy config directly and is version-coupled to Envoy's internal APIs. Before a major Istio upgrade, verify in a test cluster that both filters still take effect.
When debugging rate limits that don't fire, walk the chain in order: does the ratelimit service log show incoming requests → does the domain match → does the descriptor combination line up with the ConfigMap → is the vhost name correct. The vast majority of problems are one of the last three mismatches.
Wrap-Up
Global rate limiting at the gateway boils down to three things: the actions on the route tag requests with descriptors, the ratelimit service checks quotas against the rules in the ConfigMap, and Redis lets all gateway replicas share the counts. The configuration itself isn't complicated—the difficulty is concentrated in domain and descriptor matching, where mistakes produce no errors, just silent no-ops. Once the matching chain is clear in your head, the rest is just tuning quotas to your business needs.
COMMENTS