Service meshes like Istio, Linkerd, and Consul Connect promise transparent mutual TLS between all services in your Kubernetes cluster. Every service gets a cryptographic identity, every connection is encrypted and authenticated, and you never have to modify application code.
The promise is real, but the default configurations that ship with most service meshes prioritize compatibility over security. Getting from "installed" to "securely configured" requires understanding the mTLS modes, certificate lifecycle, and failure modes that the getting-started guides gloss over.
How Service Mesh mTLS Works
Identity Through Certificates
Each service in the mesh receives an X.509 certificate that encodes its identity. In Istio, these certificates contain a SPIFFE ID (Secure Production Identity Framework for Everyone) that identifies the service account and namespace:
spiffe://cluster.local/ns/production/sa/api-server
The mesh control plane acts as a certificate authority, issuing certificates to the sidecar proxies that handle service traffic.
Transparent Encryption
The sidecar proxy intercepts all network traffic to and from the application container. Outbound connections are upgraded to mTLS. Inbound connections are verified against the certificate presented by the calling service. The application sees plain HTTP; the network sees encrypted mTLS.
Policy Enforcement
mTLS provides the identity foundation for authorization policies. Once every service has a verifiable identity, you can write policies like "only the frontend service can call the API service" and the mesh enforces them at the network level.
mTLS Modes
Permissive Mode
Permissive mode (sometimes called "opportunistic mTLS") accepts both mTLS and plaintext connections. If the client presents a certificate, the connection uses mTLS. If not, it falls back to plaintext.
This mode is intended for migration: you can roll out the service mesh gradually while services that are not yet in the mesh can still communicate. But many organizations never move past permissive mode because it does not break anything.
The security problem is obvious: an attacker who gains access to the pod network can communicate with any service over plaintext, bypassing the entire mTLS identity model.
Strict Mode
Strict mode requires mTLS for all connections. Services without valid certificates cannot communicate. This is the mode that provides actual security guarantees.
In Istio, enable strict mTLS with a PeerAuthentication resource:
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: production
spec:
mtls:
mode: STRICT
Apply this per namespace or cluster-wide. Namespace-level strict mode allows you to enforce mTLS in production while keeping permissive mode in development namespaces.
Disable Mode
Disable mode turns off mTLS entirely. This should never be used in production. It exists for debugging and migration scenarios.
Certificate Management
Automatic Rotation
Service mesh control planes automatically rotate certificates. Istio's default certificate lifetime is 24 hours, with rotation before expiration. This limits the window during which a stolen certificate can be used.
Verify that automatic rotation is working. Monitor certificate expiration times across your mesh. A certificate that is not being rotated indicates a control plane issue that will eventually cause service outages.
Root Certificate Authority
The mesh's root CA signs all service certificates. If the root CA is compromised, every certificate in the mesh is untrustworthy.
For production deployments, use an external CA instead of the mesh's built-in CA. Istio supports plugging in external CAs, including HashiCorp Vault, cert-manager, and cloud KMS services. This provides hardware-backed key storage and integration with your organization's PKI.
Intermediate Certificates
Use an intermediate CA between the root CA and service certificates. This limits the exposure if the signing key is compromised: you can revoke the intermediate CA without replacing the root.
Certificate Revocation
Standard mTLS in service meshes does not support certificate revocation lists (CRLs) or OCSP. A compromised certificate remains valid until it expires. This is a known limitation.
Mitigate this by using short certificate lifetimes (24 hours or less) and by rotating the signing CA if a broad compromise is suspected.
Common Pitfalls
Not Verifying Strict Mode Is Active
After applying a strict mTLS policy, verify it is enforced. Test by attempting a plaintext connection from inside the mesh. If it succeeds, strict mode is not active.
Excluding Services From mTLS
Some services, particularly third-party tools and legacy applications, may not work with mTLS sidecars. Teams create exceptions for these services, punching holes in the mTLS coverage.
Document every exception. Monitor excepted services for abuse. Plan migration paths to bring excepted services into the mesh.
Ignoring External Traffic
mTLS within the mesh does not protect traffic entering or leaving the mesh. Ingress traffic from external clients and egress traffic to external services are handled separately. Ensure your Ingress controller enforces TLS for external traffic and that egress policies require TLS for outbound connections.
Control Plane Compromise
The service mesh control plane issues certificates and distributes configuration. If the control plane is compromised, the attacker can issue certificates for any identity, intercept any connection, and modify any policy.
Secure the control plane with the same rigor as any critical infrastructure: restricted network access, strong authentication, audit logging, and regular security updates.
Monitoring mTLS Health
Track mTLS Coverage
Monitor the percentage of connections using mTLS versus plaintext. Any plaintext connections in a namespace with strict mode indicate a configuration problem.
Monitor Certificate Errors
Watch for TLS handshake errors, certificate verification failures, and expired certificate alerts. These indicate either misconfiguration or potential attack activity.
Audit Authorization Policies
Regularly audit your authorization policies to ensure they align with your intended service communication patterns. Overly permissive policies undermine the security benefits of mTLS.
How Safeguard.sh Helps
Safeguard.sh complements service mesh security by providing supply chain analysis for the services running within your mesh. While mTLS ensures secure communication between services, Safeguard.sh ensures the services themselves are free of known vulnerabilities and supply chain risks. It generates SBOMs for every container in your mesh, identifies vulnerable components, and provides the visibility needed to assess risk across your entire service mesh deployment.