Email encryption in transit: STARTTLS, MTA-STS, DANE

SMTP was designed in an era when the network between two mail servers was assumed to be friendly. As RFC 3207 puts it, “SMTP servers and clients normally communicate in the clear over the Internet,” often through routers that neither side controls. The TLS we bolt onto SMTP today closes the eavesdropping gap, but it does not, by itself, prove you are talking to the right server, and it can be silently switched off by anyone on the path. MTA-STS and DANE exist to fix exactly those two holes.

The 60-second version

• STARTTLS (RFC 3207) upgrades a plaintext SMTP connection to TLS in-band, after the EHLO handshake. It is opportunistic: a public MX on port 25 may never require it.
• By default that TLS is unauthenticated and downgradeable. An active attacker can strip the 250 STARTTLS line and the session falls back to cleartext, or redirect the connection to a server whose certificate nobody checks.
• MTA-STS (RFC 8461) lets a domain publish, over DNS plus HTTPS, a policy that says “my MX hosts support TLS with a valid certificate; refuse to deliver if that is not true.” It is anchored in the Web PKI and does not need DNSSEC.
• DANE (RFC 7672) does the same job by publishing a TLSA record in DNSSEC-signed DNS that names the exact certificate or key to expect. It is downgrade-resistant because DNSSEC can prove a record is really absent.
• TLS-RPT (RFC 8460) is the feedback channel: senders mail you a daily JSON report of how many TLS sessions succeeded or failed and why.

The rest of this page is the detail. The three sibling pages go deep on MTA-STS, TLS-RPT, and DANE.

Why SMTP transport security is hard

The root problem is structural, and RFC 7672 states it plainly: SMTP is “a single-hop protocol in a multi-hop store-and-forward email delivery process,” and the destination server is chosen indirectly through DNS MX records. Two consequences follow:

• The recipient address carries no security policy. With HTTPS you type https:// and the browser demands TLS. An email address has no equivalent. Nothing in user@example.com says “encryption required.”
• TLS only protects one hop. RFC 3207 is explicit: “SMTP is not an end-to-end mechanism.” Encrypting the link between two relays “does not mean that the entire SMTP chain has been made private.” A message can pass through several MTAs; each hop is its own decision.

Because no in-band signal exists, SMTP TLS is, in RFC 7672’s words, “by necessity, opportunistic.”

STARTTLS: how the upgrade works

STARTTLS is an ESMTP extension. The server advertises the keyword STARTTLS in its EHLO response; the client issues the STARTTLS verb (no parameters); the server replies 220 and both sides begin the TLS handshake.

S: 220 mail.example.com SMTP service ready
C: EHLO relay.sender.example
S: 250-mail.example.com
S: 250-8BITMIME
S: 250-STARTTLS
S: 250 DSN
C: STARTTLS
S: 220 Go ahead
C: <begins TLS negotiation>
C & S: <negotiate a TLS session, then check the result>
C: EHLO relay.sender.example
S: 250-mail.example.com
S: 250-8BITMIME
S: 250 DSN

The server can answer the STARTTLS command with one of three codes (RFC 3207):

Reply	Meaning
220 Ready to start TLS	Proceed with the handshake
501 Syntax error (no parameters allowed)	The command was malformed
454 TLS not available due to temporary reason	Try later or fall back per local policy

Two rules from RFC 3207 matter for everything that follows:

1. The session is reset after the handshake. “The SMTP protocol is reset to the initial state.” The client MUST discard the extension list it saw in cleartext and SHOULD send a fresh EHLO; the server MUST discard the earlier EHLO argument. Anything learned before TLS is untrusted.
2. A public MX may not demand TLS. “A publicly-referenced SMTP server MUST NOT require use of the STARTTLS extension in order to deliver mail locally.” This keeps the public mail backbone interoperable, and it is precisely why plain STARTTLS cannot guarantee encryption.

The two attacks STARTTLS cannot stop alone

1. The downgrade / stripping attack

STARTTLS is negotiated over a cleartext connection, so the offer itself is unprotected. RFC 3207 describes the attack directly: “A man-in-the-middle attack can be launched by deleting the ‘250 STARTTLS’ response from the server. This would cause the client not to try to start a TLS session.” The client, seeing no STARTTLS offer, sends the mail in the clear. RFC 7672 is blunt about the limit: “pre-DANE SMTP TLS security can be subverted by simply downgrading a connection to cleartext. No TLS security feature can prevent this. The attacker can simply disable TLS.”

More generally, RFC 3207 notes that “before the TLS handshake has begun, any protocol interactions are performed in the clear and may be modified by an active attacker.”

2. The unauthenticated-server problem

Even when TLS is negotiated, plain STARTTLS does not require that the certificate be valid or match anyone in particular. RFC 3207 allows both ends to “move ahead even if the TLS negotiation ended with no authentication and/or no privacy.” So an attacker who can spoof the MX record or redirect the TCP connection can present any certificate, and a default opportunistic sender will accept it. RFC 7672 explains why authentication is impossible to bootstrap from SMTP alone: server names come indirectly from MX records, and “without DNSSEC, the MX lookup is vulnerable to MITM and DNS cache poisoning attacks.”

RFC 3207 already pointed at the only real fix: “both clients and servers MUST be able to be configured to require successful TLS negotiation of an appropriate cipher suite for selected hosts before messages can be successfully transferred.” MTA-STS and DANE are the standardised, scalable ways to express that requirement.

Opportunistic vs enforced TLS

	Opportunistic TLS (plain STARTTLS)	Enforced TLS (MTA-STS / DANE)
Encryption	Used if offered, else cleartext	Required; delivery is withheld on failure
Certificate check	Typically none (any cert accepted)	Validated against a published policy / TLSA record
Downgrade resistance	None - the offer can be stripped	Yes - the requirement is published out of band
Failure mode	Silently sends in the clear	Treated as a transient error; mail is delayed, not exposed

The shift from the left column to the right is the whole point of MTA-STS and DANE.

How MTA-STS and DANE fix it

Both publish a sender-visible commitment that TLS is expected, so a stripped 250 STARTTLS becomes a detectable failure instead of a silent downgrade. They differ in where the trust comes from.

MTA-STS (RFC 8461) - Web PKI, no DNSSEC

The recipient publishes a tiny _mta-sts DNS TXT record pointing at a policy, and serves the policy itself over HTTPS at a fixed path. The policy lists the allowed MX hostnames and a mode (enforce, testing, or none). A sender in enforce mode “MUST NOT deliver the message to hosts that fail MX matching or certificate validation or that do not support STARTTLS.” The certificate is checked against the public CA system, exactly like a browser. MTA-STS deliberately “relies on certification authorities (CAs) and does not require DNSSEC, at a cost of risking malicious downgrades.”

_mta-sts.example.com.   IN TXT "v=STSv1; id=20260601T120000Z;"
; policy served at https://mta-sts.example.com/.well-known/mta-sts.txt

Full detail on the MTA-STS page.

DANE (RFC 7672) - DNSSEC-anchored

The recipient publishes a TLSA record under _25._tcp.<mx-hostname> in a DNSSEC-signed zone, naming the certificate or public key to expect. Because the zone is signed, the record cannot be forged and its absence can be proven, so an attacker cannot pretend the domain “has no TLS.” When a usable TLSA record exists, RFC 7672 requires that the “SMTP client MUST NOT deliver mail via the corresponding host unless a TLS session is negotiated via STARTTLS.”

_25._tcp.mx1.example.com. IN TLSA 3 1 1 (
    0c72ac70b745ac19998811b131d662c9ac69dbdbe7cb23e5b514b56664c5d3d6 )

Full detail on the DANE page.

Why DNSSEC is the dividing line

The core difference is what each can prove about a domain it has never contacted. MTA-STS learns about a policy by fetching a DNS TXT record and an HTTPS file; an attacker who can block that discovery on first contact can make the domain look like it has no policy. RFC 8461 acknowledges this and recommends long max_age caching to shrink the window. DANE does not have that gap: RFC 8461 itself notes that resistance to downgrade “due to the ability to authoritatively determine ‘lack of a record’ even for non-participating recipients — is a feature of DANE, due to its use of DNSSEC.”

DNSSEC (RFC 4033) is what makes that possible. It “provides origin authentication and integrity assurance services for DNS data, including mechanisms for authenticated denial of existence” - that last clause is the one DANE depends on. DNSSEC does not encrypt DNS (“DNSSEC is not designed to provide confidentiality”); it signs it, so a resolver can tell a real answer from a forged one.

You still need to see what happened: TLS-RPT

Enforcement without visibility is dangerous: a misconfiguration looks identical to an attack, and both just delay mail. TLS-RPT (RFC 8460) closes the loop. A domain publishes a _smtp._tls TXT record, and cooperating senders mail a daily JSON summary of TLS sessions - how many succeeded, how many failed, and a result-type for each failure (for example starttls-not-supported, certificate-expired, sts-webpki-invalid, or dnssec-invalid).

_smtp._tls.example.com. IN TXT "v=TLSRPTv1; rua=mailto:tlsrpt@example.com"

Deploy TLS-RPT before you turn MTA-STS or DANE to enforcing - it is how you confirm real-world senders can negotiate TLS with you. Full detail on the TLS-RPT page.

How the pieces fit together

                      STARTTLS (RFC 3207)
                 opportunistic, unauthenticated TLS
                              |
        +---------------------+---------------------+
        |                                           |
   MTA-STS (RFC 8461)                          DANE (RFC 7672)
   DNS TXT + HTTPS policy                  DNSSEC-signed TLSA record
   trust: Web PKI (CAs)                    trust: DNSSEC chain
   no DNSSEC required                      DNSSEC required
        |                                           |
        +---------------------+---------------------+
                              |
                       TLS-RPT (RFC 8460)
              daily JSON reports: success + failure types

Common confusion

• “We have STARTTLS, so our mail is encrypted.” Only opportunistically. Without MTA-STS or DANE, the 250 STARTTLS offer can be stripped and the mail sent in the clear, with nothing logged as wrong.
• “TLS proves I reached the right server.” Not by default. Plain STARTTLS accepts unauthenticated and even mismatched certificates; only MTA-STS certificate validation or a DANE TLSA match authenticates the peer.
• “MTA-STS and DANE are competitors; pick one.” They coexist. RFC 8461 requires that a failing DANE result is never overridden by MTA-STS. Many domains publish both.
• “This encrypts the message end to end.” No. These protocols secure the transport between two MTAs, hop by hop. End-to-end content protection is S/MIME or PGP, a different layer.

What this means for you, and what Egressif does

Transport security is a receiver-and-sender contract: the receiver publishes what it expects, and a well-behaved sender honors it. On the domains we operate, we publish and maintain MTA-STS and TLS-RPT, and DANE where the domain is DNSSEC-signed, so that mail to us is encrypted and authenticated rather than merely encrypted-if-convenient. We watch the TLS-RPT stream the same way we watch DMARC aggregate reports: a spike in certificate-expired or starttls-not-supported is a signal we act on before it becomes delayed mail. We do not promise that every hop beyond our control is protected - no one honestly can - but we make the hops we control verifiable, and we keep the records current as certificates and MX hosts change.