SMTP Relay Explained: How It Works, Open Relay Risks & Setup

What Relay Actually Means in an Email's Journey

Very few emails travel directly from a sender's mail client straight to the recipient's mailbox in a single hop — almost every message passes through at least one, and often several, intermediate servers along the way, and each of those intermediate hops is performing relay: accepting a message it isn't the final destination for, and forwarding it onward toward the next step in its journey. A typical path might look like: your mail client submits a message to your provider's submission server (authenticated, on port 587 or 465), that provider's outbound infrastructure then relays the message to the recipient's mail server (on port 25, having looked up their MX records), and that recipient's server performs final delivery into the actual mailbox. The middle step in that chain — provider to recipient — is relay in its purest, most legitimate form, and it's this exact mechanism that, left unrestricted, became one of the most exploited weaknesses of early internet mail infrastructure.

The Historical Open Relay Disaster, Briefly

Through the 1990s and into the early 2000s, it was extremely common for mail servers — many run by small organizations, universities, and even early ISPs — to accept and relay mail from absolutely anyone, for any destination, with no authentication or restriction whatsoever. This wasn't originally a security oversight so much as a reflection of the internet's early, more trusting culture; the assumption that a connecting server was acting in good faith simply hadn't been seriously challenged yet at the scale it eventually would be. As spam became commercially lucrative, spammers rapidly realized that open relays let them send enormous volumes of unsolicited mail that appeared to originate from someone else's server and reputation entirely, hiding their own identity and sending infrastructure while borrowing the (initially clean) reputation of the exploited relay. Within a relatively short period, essentially every unrestricted, discoverable open relay on the internet was actively being abused, sometimes within hours of being detected by automated scanning tools built specifically to find them. This drove the mail server software industry to fundamentally rethink default configurations — restricting relay by default became the new standard, and remains the standard today specifically because of how catastrophically the earlier, more permissive default played out in practice.

How Automated Scanners Find Open Relays Today

It's worth understanding the actual mechanics of how an open relay gets discovered and abused, since the speed involved is genuinely surprising to anyone who assumes an obscure, low-traffic server is somehow safer by virtue of being unnoticed. Automated scanning tools — some run by security researchers cataloging vulnerable infrastructure, many more run directly by spam operations — continuously and systematically probe wide swaths of IP address space specifically looking for exactly the behavior an open relay exhibits: a server on port 25 that accepts a RCPT TO command for a domain it doesn't host, without requiring authentication first. These scans aren't targeted at specific, well-known organizations; they're broad, indiscriminate sweeps that will eventually reach any publicly reachable mail server regardless of how small or obscure the organization behind it is. Once discovered, an open relay is typically added to automated abuse tooling within hours, sometimes minutes, and begins receiving relay requests from spam-sending infrastructure looking to exploit exactly this kind of unrestricted server. This is precisely why "it's a small server nobody would bother targeting" is a dangerous assumption — the targeting isn't manual or selective, it's automated and comprehensive, and obscurity offers essentially no real protection against it, a lesson that has been relearned the hard way by countless small organizations who assumed their low profile was itself a form of security.

The Downstream Consequences of Being Discovered as an Open Relay

Beyond the immediate abuse itself, being discovered and exploited as an open relay carries a cascade of downstream consequences that can significantly outlast the initial incident. Major reputation and blocklist services — used by the vast majority of receiving mail servers worldwide to filter incoming mail — actively track and list IP addresses observed sending spam, including relay abuse traffic passing through a compromised open relay. Once an IP address lands on one or more of these blocklists, legitimate mail sent from that same server (including entirely unrelated, genuinely legitimate mail from the organization that owns it) can be rejected or heavily filtered by receiving servers checking those same blocklists, sometimes for a period of days or weeks even after the underlying open relay vulnerability has been fixed, since blocklist removal typically requires an explicit delisting request and review process rather than an automatic, instant reversal once the immediate cause is corrected. Organizations that have experienced this firsthand often describe the reputation recovery process as considerably more painful and time-consuming than the technical fix for the open relay misconfiguration itself, which reinforces why prevention — correct configuration from the start, and periodic testing thereafter — is so much more valuable than remediation after the fact.

Distinguishing True Open Relay From Adjacent Misconfigurations

That last row is worth specific attention, since it represents one of the more common real-world variants of accidental open relay exposure — not a complete absence of restriction, but a restriction so loosely scoped that it fails to meaningfully restrict anything in practice, which is functionally equivalent to no restriction at all from an abuse perspective.

How to Test Whether Your Server Is Accidentally an Open Relay

Testing relay restriction directly is the only reliable way to confirm a configuration is actually working as intended, rather than trusting documentation or assumed defaults. From a connection that should not be authorized — an external network, without providing credentials — attempt to send a test message through your server addressed to a domain your server doesn't host. A properly restricted server should refuse this attempt, typically with a "relay access denied" or similar error, at the RCPT TO stage of the SMTP conversation. If the server instead accepts the message and appears willing to forward it, that's a serious, immediate configuration problem requiring urgent correction, since it means the server is currently exploitable exactly as historical open relays were. Numerous free, public open relay testing services exist specifically for this purpose, and running a periodic check — not just once at initial setup — is good practice given how a subsequent configuration change could inadvertently reopen a restriction that was previously correctly locked down.

How Relay Rate Limiting Actually Gets Implemented

Beyond a simple authenticated-or-not binary decision, mature relay infrastructure typically implements rate limiting as a genuinely important secondary layer of protection, addressing a scenario authentication alone can't solve: a legitimate, correctly authenticated account that's been compromised, or an application with a bug causing it to send far more mail than intended. Rate limiting can be applied at several different granularities simultaneously — per-account limits (a given set of credentials can send no more than X messages per hour), per-IP limits (a given source IP can initiate no more than Y connections or messages in a window), and per-recipient-domain limits (protecting against a single sender overwhelming one specific destination, which can itself trigger reputation problems with that destination). A well-configured relay service typically applies graduated responses as limits are approached or exceeded — temporary throttling before an outright block, alerting administrators of unusual volume before automatically suspending an account, and clear, actionable error messages (typically 4xx temporary failures) rather than silent message dropping, so a legitimate sender hitting a limit understands what happened and can address it rather than simply losing mail with no indication why.

Relay Authentication Models Beyond Simple Username and Password

While SMTP AUTH with a username and password remains the most common relay authentication model, especially for smaller-scale or self-hosted setups, several alternative or complementary models are worth understanding for more sophisticated relay configurations. IP-based trust, already covered, works well for stable, known infrastructure but doesn't scale well to dynamic or cloud-hosted sending sources where the IP address might change. API-key-based authentication — common among transactional email providers — functions similarly to a password but is typically designed to be more easily scoped, rotated, and revoked independently per application or integration, without needing to change a shared account password that might affect multiple integrations simultaneously. Mutual TLS (client certificate authentication) represents a more rigorous model where the connecting client presents a cryptographic certificate as proof of identity, rather than a shared secret like a password, offering stronger guarantees against credential theft since the private key never needs to be transmitted at all, though it requires more sophisticated certificate management infrastructure than simpler password or API-key models and is correspondingly less common outside of larger, security-mature organizations that already have the supporting certificate lifecycle tooling in place for other purposes.

Real-World Use Cases

Real-Time Blackhole Lists and Their Role in the Open Relay Story

No discussion of open relay history is complete without covering the tooling that emerged specifically to combat it, since this tooling remains foundational to how spam filtering works even today, long after open relay abuse itself became far less common. Real-time Blackhole Lists (RBLs), sometimes called DNS-based Blocklists (DNSBLs), emerged directly in response to the open relay epidemic — community-maintained, continuously updated lists of IP addresses observed sending spam, including relay-abused open relays, queryable in real time by any receiving mail server as part of its own spam filtering decision. A receiving server checking an incoming connection's source IP against one or more RBLs could reject or heavily filter mail from any IP appearing on the list, creating a strong, tangible incentive for organizations to fix an open relay quickly once discovered and listed, since the alternative was watching all of their legitimate mail get rejected by a growing share of the internet. This mechanism — external, community-driven reputation tracking with real, immediate consequences for listed IPs — proved so effective that it became a permanent fixture of email infrastructure rather than a temporary response to a specific historical problem, and modern email reputation and blocklist services descend directly from this same basic model, even as the specific threats they're tracking have evolved considerably since the original open relay era.

Related Reading

For the specific error codes you'll encounter when relay is correctly restricted, see SMTP Connection Errors. For the authentication mechanisms that gate legitimate relay access, read SMTP Authentication. For which port relay traffic typically uses, see SMTP Ports Explained. For the encryption layer protecting relay credentials in transit, read SMTP TLS vs SSL. For a broader troubleshooting framework if your relay configuration isn't behaving as expected, see SMTP Troubleshooting. To test your own relay server's connectivity and behavior directly, use the SMTP Tester.