WTF is the difference between mTLS and TLS?

A screenshot from an episode of The Simpsons in which Mr. Burns is shopping for groceries and holds up two red condiment bottles; one labeled "ketchup" and the other labeled "catsup", but the bottels have the letters TLS and mTLS superimposed on them. — Figure 1: Ketchup or catsup?

tl;dr - Istio uses mTLS for in-mesh pod to pod traffic and TLS for traffic entering the mesh from outside the cluster.

The Protocols Themselves

Certificates

TLS and mTLS both rely on certificates which contain some identity information and a key used to encrypt data in transit. The validity of a certificate can be verified by checking if it has been signed by a trusted key. Anyone can sign a certificate, but most clients will not trush a certificate which has been signed by just anyone. Browsers trust certificates which have been signed by members of the certificate authority browser forum and Istio sidecars trust certificates which have been signed by istiod ¹.

When a new pod in the mesh starts up, the sidecar container calls home to istiod and identifies itself using its Kubernetes service account token. istiod validates the token, and extracts the name of the service account and the namespace it exists in from the token metadata and then encodes it into the Subject Alternative Name field of a brand new certificate, signs it, and returns it to the sidecar.

TLS

TLS provides two things: it encrypts data in transit between a client and a server, and it proves the identity of the server to the client.

mTLS

Mutual TLS also encrypts data in transit, and it also proves the identity of the server to the client, but it then goes on to prove the identiy of the client to the server.

How Istio Uses Them

Pod to Pod Traffic

When traffic is sent between two pods which are part of the mesh and mutual TLS is enabled (which is the default), both the downstream and the upstream pods will exchange certificates in order to verify each others identity and encrypt traffic in transit.

Authorization

Users can optionally enforce policies on pod-to-pod traffic such as blocking all traffic from a particular namespace or service account, by using the information which istiod added into the certificates SAN field.

Getting Traffic Into Your Cluster

When traffic from the internet arrives at an Istio cluster it is not protected by mTLS. Random web browsers can't talk to istiod to request a client certificate, and they don't trust certs signed by istiod, so they can't participate in mesh traffic. They need something to turn their TLS traffic into mesh traffic protected by mTLS and that something is called an ingress gateway.

Fortunately, Istio gateways can accept plain old non-mutual TLS for incoming traffic, which is convenient because web browsers do prefer TLS everywhere. For plain old non-mutual TLS traffic, Istio (like every other web server out there) requires some other authority to mint the certificates it will use. This could be from a commercial vendor like Digicert or a non-profit like LetsEncrypt. The bottom line is that for external clients to communicate securely with an Istio ingress gateway, someone needs to give the gateway a cert to use. Istio cannot mint these itself.

This process could involve just getting a .pem file from a provider and manually feeding it to kubectl create secret ... or it could leverage a tool like Cert Manager to automatically provision certs from providers who support such tooling.

In this configuration traffic is protected by vanilla TLS on the way from the client to the ingress gateway, and then mTLS on the way from the ingress gateway to the upstream pod.

flowchart RL
  client(Web Browser)-- TLS -->ingressgw(Ingress Gateway)
  ingressgw-- mTLS -->pod(Pod)

linkStyle 0 stroke:lightgreen
linkStyle 1 stroke:lightgreen

Figure 2: ¿Por qué no los dos?

Summary

mTLS is used for in cluster pod-to-pod traffic, protected by self signed cert.
TLS is used for incoming traffic, protected by a publicly trusted cert.
They can be used together but they don't have to be.

Footnotes

By default Istio mints its own mTLS client certs and distributes them to each pod in the mesh, but it can also integrate with other projects capable of minting certificates like SPIRE and cert-manager.↩︎