Greylisting, tarpitting, and rate controls

The cheapest spam to handle is the spam you never accept. Before a receiver spends CPU on Bayesian tokenization, fuzzy hashing, or rule evaluation, it can apply connection-level controls that work on the SMTP conversation itself - deferring, delaying, or rate-limiting a suspicious source. The best-known of these is greylisting, and it rests on a single asymmetry: legitimate mail servers retry after a temporary failure, and a large class of spam-sending software does not.

This page covers greylisting (which IETF standardized in RFC 6647), the SMTP retry obligation that makes it work (RFC 5321), and the related techniques of tarpitting and rate limiting. It is written for senders, because these controls are the ones most likely to delay your legitimate mail - and the fix is almost always on the sending side: retry correctly, send consistently, and stay recognizable.

The 60-second version

• Greylisting returns a temporary SMTP failure (a 4xx) to an unknown sender, then accepts the message when the sender retries.
• It works because “spamware does not perform retransmission attempts after receiving an SMTP temporary failure” - in spamming, “volume counts for far more than reliability” (RFC 6647).
• Legitimate senders retry because RFC 5321 requires it: “mail that cannot be transmitted immediately MUST be queued and periodically retried by the sender.”
• It is cheap and buys time for blocklists to catch up, but the cost is delay - the end user notices a lag only on the first message from a new sender.
• RFC 6647 recommends greylisting on the tuple (IP, MAIL FROM, first RCPT TO) and not applying it to authenticated submission.
• Tarpitting deliberately slows the connection to impose cost on bulk senders; it has no standards-track RFC and varies by implementation.
• Rate limiting caps throughput per source and operates independently of greylisting (RFC 6647).
• None of these is foolproof. They are layers that reduce load and buy time, not standalone solutions.

Greylisting: the mechanism

RFC 6647 defines the term at two scopes. Broadly, greylisting “refers to any degradation of service for an unknown or suspect source, over a period of time (typically measured in minutes or a small number of hours).” Narrowly - the common meaning - it “refers to generation of an SMTP temporary failure reply code for traffic from such sources.”

The logic is a deliberate exploitation of how badly-behaved senders cut corners:

“Spamware does not perform retransmission attempts after receiving an SMTP temporary failure. That is, if the spamware cannot deliver a message, it just goes on to the next address in its list since, in spamming, volume counts for far more than reliability.” - RFC 6647 §1.1

So the receiver tells an unfamiliar sender “try again later” (a 4xx code). A spam engine optimized for throughput moves on. A real mail server queues the message and retries - and on the retry, now recognized, it is let through.

First contact from an unknown source
  -> 450 try again later     (greylisted; receiver records the tuple)

Spamware: never retries -> message never delivered
Legit MTA: queues, retries after its delay -> recognized -> accepted

The triplet, over time

The state a greylister keeps is small. In the RFC’s recommended form it is a triplet - (client IP, MAIL FROM, first RCPT TO) - plus a timestamp. A first sighting is recorded and deferred; a retry that lands inside the accepted window is let through and the triplet becomes “known” for at least a week. Walked through one new sender:

t = 0       (1.2.3.4, news@sender.example, you@example.org) not in DB
            -> store triplet + timestamp, reply 450 "try again later"
t + ~min    legit MTA requeues; retry within the 1 min - 24 h window
            -> triplet matches a stored record -> ACCEPT, mark "known"
t + days    any further mail on this triplet (record kept >= 1 week)
            -> delivered immediately, no delay

spamware    sees the 450, never retries -> never stored as "known" -> never delivered

The end user therefore feels a delay exactly once per new sender, and never again while the record lives. A widely-cited (secondary) figure puts the typical first-message delay at around 15 minutes - driven by the sender’s retry schedule, not the receiver - though that number is not in the RFC and varies enormously by MTA.

Why legitimate senders survive it

The asymmetry only works because legitimate senders are obligated to retry. RFC 5321 (§4.5.4.1) requires that “mail that cannot be transmitted immediately MUST be queued and periodically retried by the sender,” and that “the SMTP client retains responsibility for delivery of that message.” RFC 5321 guidance puts the retry interval at at least 30 minutes with a give-up time of at least 4–5 days, though actual values vary widely by MTA. This is the contract greylisting depends on: a sender that does not retry is, per RFC 5321, “not fully SMTP-capable.”

The types of greylisting

RFC 6647 (§2) distinguishes five variants by what is recorded and when the 4xx is issued in the SMTP session:

Type	Matching key	Where the 4xx is issued
Connection-level	IP address (and/or hostname)	At TCP connect (421 close) or on all commands
HELO/EHLO-level	IP + HELO/EHLO parameter	After HELO/EHLO
MAIL-level	IP + MAIL FROM	After MAIL
RCPT-level	IP + MAIL FROM + first RCPT TO	After RCPT
DATA-level	Full tuple, optionally + content digest	On/after DATA

The finer the key, the more precisely it identifies a “known” sender - but also the more ways an innocent change between attempts can look like a new sender, which is the source of most failure modes below.

RFC 6647’s recommended implementation

The RFC does not leave implementers guessing. Its §5 recommendations are the closest thing to a “do it like this”:

Recommendation	Detail
Matching tuple	(IP, MAIL FROM, first RCPT TO) - only the first RCPT is needed “as legitimate MTAs appear not to reorder recipients between retries”
Retry window	SHOULD accept retries from 1 minute to 24 hours; retries after that window SHOULD be treated as a new message
Record timeout	SHOULD persist at least one week, so re-greylisting happens if an IP changes hands
Shared database	All inbound border MTAs in one administrative domain SHOULD share a greylisting database
CIDR clustering	MAY track /24 blocks instead of full addresses, to tolerate senders with clustered outbound servers
Authenticated sessions	Greylisting SHOULD NOT be applied to authenticated submission clients
4xx code	No strict rule; the “reasonable choices” are 421 (close immediately) or 450 (continue session)
Reply text	No recommendation - there is a genuine debate about whether to reveal that greylisting is in effect

That “share a database across border MTAs” point is critical and underlies one of the worst failure modes: a domain with several MX hosts that don’t share state will re-greylist a sender each time it retries to a different host.

Benefits

RFC 6647 (§3) and the surrounding analysis credit greylisting with several concrete advantages:

• It is cheap. “Rejecting email with a temporary 4xx error is very cheap in system resources. Most spam filtering tools are very intensive users of CPU and memory. By stopping spam before it hits filtering processes, far fewer system resources are used.”
• It buys time. “Delaying delivery also gives real-time blackhole lists and similar lists the time to identify and flag the spam source.” The retry may now be caught by a blocklist that did not know the source on first contact.
• It needs no user configuration. The end user “only notices delay on first message from a given sender.”
• It is a layer, not a cure. RFC 6647 frames it inside the array-of-techniques principle: no single mechanism is sufficient.

Costs and failure modes

This is the part a sender must understand, because nearly every greylisting problem shows up as delayed or lost legitimate mail, and the cause is usually a mismatch between attempts.

Failure mode	What happens
Slow retry	”Some popular MTAs do not retry failed delivery attempts for an hour or more,” causing painful delays for time-critical mail
Non-retrying senders	Some legitimate-but-broken MTAs never retry at all - greylisting then causes outright delivery failure
4xx treated as 5xx	An MTA that treats the temporary failure as permanent bounces the message and may drop the recipient from a list
Multi-MX	Retrying to a different MX host that doesn’t share the database → greylisted again
Multi-outbound / DHCP	Retrying from a different outbound IP → looks like a new source → greylisted again
DATA-level staleness	If the message changes between attempts (headers reformulated), the tuple differs → greylisted again
VERP / SRS / BATV	Per-message unique return-paths change MAIL FROM every time, defeating exact-match greylisting (mitigated by matching domain + start of local-part only)
NAT	Distinct MTAs behind one shared IP - the first is greylisted, later new MTAs from that IP slip past
Infrequent senders	A sender that mails a destination rarely may expire from the database and be greylisted nearly every time
Too-aggressive retry	Retrying faster than the minimum age can additionally trip rate limiting

Two structural limits round this out. Address-space saturation: “Bad actors that send mail with just enough frequency to avoid having their records expire will never be caught by this mechanism after the first instance” - so RFC 6647 recommends “combining greylisting with some form of reputation service.” And IPv6: the RFC warns the much larger address space “might even negate any benefits in using greylisting at all,” and makes the database far larger - greylisting’s applicability to IPv6 is explicitly uncertain.

Tarpitting

Tarpitting is the other connection-level idea: instead of deferring with a 4xx, the receiver deliberately slows the connection to impose time-cost on a sender that is trying to push high volume. It fits within RFC 6647’s broad definition of greylisting as “any degradation of service for an unknown or suspect source,” but as a distinct technique it is not standardized - there is no standards-track RFC for SMTP tarpitting, and the specifics are implementation conventions rather than protocol.

Common forms described in implementation literature include delaying the initial SMTP greeting (a “greet delay”) so that legitimate clients tolerate a small pause while high-volume operations accumulate cost; pausing to listen after the connection opens and dropping clients that “talk” before the server’s greeting (a violation of SMTP ordering, since a well-behaved client waits for the banner); and answering with slow, multi-line temporary responses that tie up the sender’s process. A slow multi-line reply uses the SMTP hyphen-continuation syntax to hold the session open, spacing the lines seconds apart:

220 mail.example.org ESMTP            (banner delayed several seconds)
... receiver pauses 15+ seconds between each 4xx line ...
451-please hold
451-still holding
451 temporary failure, try later     (final line ends the multi-line reply)

A patient, standards-compliant sender simply waits and retries; a throughput-optimized spam engine has its connection slots and time budget drained. Because these forms vary by product and are not governed by a standard, this library does not assert specific implementations, version numbers, or vendor histories beyond noting that the technique exists and is connection-level. The principle is the durable part: make abusive bulk sending slow and expensive without harming the patient, retry-capable legitimate sender.

Rate limiting

Rate limiting caps how much a single source may send over time (connections per IP per minute, messages per hour, and so on). RFC 6647 (§4.2) confirms it as a distinct, independent control: a greylisted client with an “aggressive” retry schedule “could be subjected to rate limiting by the MTA independent of the restrictions imposed by greylisting.” In other words, rate limiting runs alongside greylisting, not as part of it.

Beyond that confirmed relationship, the precise semantics of SMTP rate limiting are not defined by any single primary RFC - it is a server-side policy that each operator implements and tunes. Its general strength is throttling high-volume abuse from a single IP or network block; its general weakness is distributed (botnet) abuse, where no single source exceeds the cap. This library labels those general characterizations as exactly that - characterizations - rather than standardized facts.

The three controls side by side

Control	Mechanism	Standardized?	Cost to receiver	Main risk to a legitimate sender
Greylisting	Temporary 4xx on first sighting; accept on retry	Yes - RFC 6647 (applicability statement)	Very low - no content scan on first contact	A one-time delay, or repeated deferral if your identity shifts between retries
Tarpitting	Deliberately slow the SMTP conversation	No standards-track RFC	Holds a connection open (some resource)	Added latency, harmless if you simply wait and retry
Rate limiting	Cap connections/messages per source over time	No single RFC; RFC 6647 §4.2 confirms it operates independently	Low	Throttling or deferral if you send too fast from one IP

These are not alternatives so much as rungs that often run together - modern filters even fold greylisting into their scoring, treating “greylist” as a score-driven action rather than a fixed rule (see Rspamd architecture).

The layered-defence principle

Every one of these controls is a layer, and RFC 6647 says so directly: “Absent a perfect abuse-detection mechanism that incurs no cost, the current requirement is for an array of techniques to be used by each filtering system.” And, plainly: “Greylisting is obviously not a foolproof solution to avoiding abusive traffic.” Connection-level controls reduce the volume that reaches content scoring and buy reputation systems time; they do not replace authentication, content analysis, or complaint/bounce feedback. All layers are complementary.

What this means for you, and what Egressif does

Greylisting is the spam control most likely to make a legitimate operator think something is broken when nothing is - a first message to a new recipient arrives late, and the cause is a temporary failure your MTA is supposed to retry. The sender-side requirements are clear from the RFCs: retry correctly per RFC 5321 (queue on 4xx, sane intervals, don’t bounce a temporary failure), and don’t sabotage your own recognizability - shifting outbound IPs, per-message VERP return-paths, or reformulated headers between attempts all make a retry look like a brand-new sender and restart the clock.

Egressif handles exactly those behaviors: standards-compliant retry on temporary failures, a stable and consistent sending identity so greylisting recognizes the retry as the same sender, and disciplined sending rates that don’t trip independent rate limits. Greylisting then costs your mail a one-time delay at most - which is the design intent - rather than repeated deferrals or lost messages. We cannot remove another operator’s greylisting or rate limits, and we would not want to; we make sure your mail behaves the way the standards assume so those controls wave it through. Done right, a deferral is a delay and never a false positive; done wrong on the sending side - an MTA that bounces a 4xx, or an identity that changes between retries - it can become one.