API Gateway vs Service Mesh

API gateways and service meshes manage traffic at different layers of the architecture. An API gateway handles north-south traffic (external clients entering the system). A service mesh handles east-west traffic (services communicating with each other internally).

API Gateway

An API gateway sits at the edge of your system. It handles authentication, rate limiting, request routing, protocol translation, and API versioning for external clients. Tools like Kong, AWS API Gateway, and Traefik serve this role.

Service Mesh

A service mesh deploys a sidecar proxy (typically Envoy) alongside every service instance. These proxies handle mTLS encryption, load balancing, retries, circuit breaking, and observability for all internal traffic. Istio, Linkerd, and Consul Connect are popular implementations.

They Solve Different Problems

The API gateway protects your system from the outside world. The service mesh protects services from each other. In a mature microservices architecture, you need both.

When a Service Mesh Is Overkill

Service meshes add significant complexity and resource overhead. Every service gets a sidecar proxy consuming CPU and memory. For architectures with fewer than 10-20 services, the operational cost often outweighs the benefits. Simple retry logic and circuit breakers in application code may suffice.

When It Is Essential

At scale (50+ services), implementing retries, mTLS, circuit breaking, and observability in every service's application code is unsustainable. The service mesh provides these capabilities uniformly without changing application code.

Learn more in our system design interview guide and microservices interview questions. See pricing for access.

The Bottom Line

Use an API gateway for external traffic management. Add a service mesh when internal service communication complexity justifies the operational investment — typically at 20+ services with strong observability and security requirements.