In computing, Sender Policy Framework (SPF) is an extension to the Simple Mail Transfer Protocol (SMTP). SPF allows software to identify and reject forged addresses in the SMTP MAIL FROM (Return-Path), a typical nuisance in e-mail spam. SPF is defined in Experimental RFC 4408. RAM (Random Access Memory) Look up computing in Wiktionary, the free dictionary. ... Simple Mail Transfer Protocol (SMTP) is the de facto standard for email transmission across the Internet. ... E-mail spam, also known as bulk or junk e-mail is a subset of spam that involves sending nearly identical messages to numerous recipients by e-mail. ...

Principles of Operation

Normal SMTP allows any computer to send an e-mail claiming to be from anyone. Thus, it's easy for spammers to send e-mail from forged addresses. This makes it difficult to trace back to where the spam truly comes from, and easy for spammers to hide their true identity in order to avoid responsibility. Many believe that the ability for anyone to forge sender addresses, a.k.a. Return-Paths, is a security flaw in modern SMTP, caused by an undesirable side-effect of the deprecation of source routes. E-mail spam, also known as bulk or junk e-mail is a subset of spam that involves sending nearly identical messages to numerous recipients by e-mail. ... Electronic mail, abbreviated e-mail or email, is a method of composing, sending, and receiving messages over electronic communication systems. ... An -mail address identifies a location to which e-mail messages can be delivered. ... Simple Mail Transfer Protocol (SMTP) is the de facto standard for e-mail transmissions across the Internet. ...

Further information: Sender Rewriting Scheme (SRS)

SPF allows the owner of an Internet domain to use special format of DNS TXT records to specify which machines are authorized to transmit e-mail for that domain. For example, the owner of the example.org domain can designate which machines are authorized to send e-mail whose sender e-mail address ends with "@example.org". Receivers checking SPF can reject messages from unauthorized machines before receiving the body of the message. Thus, principles of operations are quite similar to those of DNSBL, except that SPF exploits the authority delegation scheme of the real Domain Name System. The introduction to this article provides insufficient context for those unfamiliar with the subject matter. ... It has been suggested that this article be split into multiple articles. ... It has been suggested that this article be split into multiple articles. ... A DNSBL is a means by which an Internet site may publish a list of IP addresses that some people may want to avoid and in a format which can be easily queried by computer programs on the Internet. ... It has been suggested that this article be split into multiple articles. ...

The sender address is transmitted at the beginning of the SMTP dialog. If the server rejects the sender, the unauthorized client should send a Bounce message to that address. If the server accepts the sender, and subsequently also accepts the recipient(s) and the body of the message, it should insert a Return-Path header in the message's body in order to save the sender address. While the address in the Return-Path often matches other originator addresses in the mail header like "From:" or "Sender:" this is not necessarily the case, and SPF does not prevent forgeries of these other addresses. Simple Mail Transfer Protocol (SMTP) is the de facto standard for email transmission across the Internet. ... This article does not cite any references or sources. ... In computing, a client is a system that accesses a (remote) service on another computer by some kind of network. ... A bounce message, or Delivery Status Notification (DSN) message or, simply, a bounce is an automated electronic mail message from the receivers mail system, the message tells the sender that the message could not be delivered. ...

Spammers can send e-mail with an SPF PASS result if they have an account in a domain with a sender policy, or abuse a compromised system in this domain. However, doing so makes the spam easier to trace and prosecute.

The main benefit of SPF is to people whose e-mail addresses are forged in the Return-Paths. They receive a large mass of unsolicited error messages and other auto-replies, making it difficult to use e-mail normally. If such people use SPF to specify their legitimate sending IPs with a FAIL result for all other IPs, then receivers checking SPF can reject forgeries, reducing the amount of back-scatter.

SPF has potential advantages beyond helping identify unwanted e-mail. In particular, if a sender provides SPF information, then receivers can use SPF PASS results in combination with a white list to identify known reliable senders. Scenarios like compromised systems and shared sending mailers limit this use.

FAIL and forwarding

If a domain publishes an SPF FAIL policy, then legitimate mails sent to receivers forwarding their mail to third parties can be rejected and bounced if (1) the forwarder doesn't rewrite the Return-Path, unlike mailing lists, (2) the next hop doesn't white list the forwarder, and (3) this hop checks SPF. This is a necessary and obvious feature of SPF - checks behind the "border" MTA (MX) of the receiver can't work directly. A mail transfer agent or MTA (also called a mail transport agent, mail server, or a mail exchanger in the context of the Domain Name System) is a computer program or software agent that transfers electronic mail messages from one computer to another. ... An MX record or Mail exchange record is a type of resource record in the Domain Name System (DNS) specifying how Internet e-mail should be routed. ...

Publishers of SPF FAIL policies must accept this potential problem; they cannot expect that all receivers change their forwarding arrangements, i.e. clear at least one of the three critical conditions.

A technique called Sender Rewriting Scheme (SRS) is a way for mail forwarding services to avoid this problem. The introduction to this article provides insufficient context for those unfamiliar with the subject matter. ...

HELO tests

For an empty Return-Path as used in error messages and other auto-replies, an SPF check of the HELO-identity is mandatory. Actually it checks an artificial postmaster@mail.example.org address for a HELO or EHLO mail.example.org. A bounce message, or Delivery Status Notification (DSN) message or, simply, a bounce is an automated electronic mail message from the receivers mail system, the message tells the sender that the message could not be delivered. ...

With a bogus HELO identity the result NONE won't help, but for valid host names SPF also protects the HELO identity. This SPF feature was always supported as an option for receivers, and later SPF drafts including the final specification recommend to check the HELO always.

This allows to white list sending mailers based on a HELO PASS, or to reject all mails after a HELO FAIL. It can also be used in reputation systems (any white or black list is a simple case of a reputation system).

Implementation

Implementing SPF has two parts:

• Domains identify the machines authorized to send e-mail on their behalf. Domains do this by adding an additional record to their existing DNS information.
• Receivers can request and use SPF information. They use ordinary DNS queries, which are typically cached to enhance performance. Receivers then interpret the SPF information as specified and act upon the result.

Thus, the key issue in SPF is the specification for the new DNS information that domains set and receivers use. The records are laid out like this (in typical DNS-syntax):

 example.org. IN TXT "v=spf1 a mx -all" 

"v=" defines the version of SPF used. The following words provide mechanisms to use to determine if a domain is eligible to send mail. The "a" and "mx" specify the systems permitted to send messages for the given domain. The "-all" at the end specifies that, if the previous mechanisms did not match, the message should be rejected.

Mechanisms

Eight mechanisms are defined:

It has been suggested that IPv6 internet be merged into this article or section. ... Internet Protocol version 4 is the fourth iteration of the Internet Protocol (IP) and it is the first version of the protocol to be widely deployed. ... It has been suggested that IPv6 internet be merged into this article or section. ... An MX record or Mail exchange record is a type of resource record in the Domain Name System (DNS) specifying how Internet e-mail should be routed. ... FCrDNS, or Forward Confirmed Reverse DNS, is when an IP address has both forward (name -> IP) and reverse (IP -> name) DNS entries that match each other. ... A DNSBL is a means by which an Internet site may publish a list of IP addresses that some people may want to avoid and in a format which can be easily queried by computer programs on the Internet. ... “ISP” redirects here. ...

Qualifiers

Each mechanism can be combined with one of four qualifiers:

• + for a PASS result, this can be omitted, +mx is the same as mx.
• ? for a NEUTRAL result interpreted like NONE (no policy).
• ~ for SOFTFAIL, a debugging aid between NEUTRAL and FAIL.
• - for FAIL, the mail should be rejected (see below).

Modifiers

The modifiers allow for future extensions of the framework. So far only the two modifiers defined in the RFC 4408 are widely deployed.

exp=some.example.com gives the name of a domain with a DNS TXT record, which is interpreted using SPF's macro language to get an explanation for FAIL results, typically an URL, added to the SMTP error code. This baroque feature is rarely used. It has been suggested that this article be split into multiple articles. ... // Uniform Resource Locator (URL) is a technical, Web-related term used in two distinct meanings: In popular usage, it is a widespread synonym for Uniform Resource Identifier (URI) — many popular and technical texts will use the term URL when referring to URI; Strictly, the idea of a uniform syntax for...

redirect=some.example.com can be used instead of the ALL-mechanism to link to the policy record of another domain. This modifier is easier to understand than the somewhat similar INCLUDE-mechanism.

Error handling

As soon as SPF implementations detect syntax errors in a sender policy they must abort the evaluation with result PERMERROR. Skipping erroneous mechanisms cannot work as expected, therefore include:bad.example and redirect=bad.example also cause a PERMERROR.

Another safety guard is the maximum of ten mechanisms querying DNS, i.e. any mechanism except from IP4, IP6, and ALL. Implementations can abort the evaluation with result SOFTERROR when it takes too long or a DNS query times out, but they must return PERMERROR if the policy directly or indirectly needs more than ten queries for mechanisms, any redirect= also counts towards this processing limit.

A typical SPF HELO policy v=spf1 a -all needs three DNS queries: (1) TXT, (2) SPF, and (3) A or AAAA. This last query counts as the first mechanism towards the limit (10), in this example it's also the last, because ALL needs no DNS lookup.

Caveats

SPF normally only validates the domain of the envelope sender (in the Return-Path). Domains that share mail senders (e.g. with virtual hosting) can forge each others' domain. SPF does not validate that a given e-mail actually comes from the claimed user, because it operates at the network level.

SPF FAIL rejection

SPF FAIL policies can be an effective but dangerous tool. Some publishers of SPF policies try to avoid the dangers by using SOFTFAIL (designed for limited testing periods) instead of FAIL. But SOFTFAIL can be even more dangerous than FAIL with receivers rejecting FAIL and accepting SOFTFAIL tagged as potential junk.

A reject in a forwarding scenario is a clean decision. The forwarder will send an error message to the address in the Return-Path. Typically the error message (bounce) contains an explanation of the SMTP error and the failing (forwarded to) address. The original sender can then send his mail again directly to this address bypassing the forwarder, a crude emulation of the normal SMTP error code 551 user not local. A bounce message, or Delivery Status Notification (DSN) message or, simply, a bounce is an automated electronic mail message from the receivers mail system, the message tells the sender that the message could not be delivered. ...

However an accepted SOFTFAIL tagged as potential junk could be deleted by the final recipient. This is a user who has arranged his forwarding in a way that cannot work with SPF, this user could also be careless with checking his potential junk and simply delete it.

The same line of arguments also suggests that receivers should take the SPF recommendation to reject SPF FAIL seriously where possible. Accepting SPF FAIL results as potential junk can be more dangerous than simply rejecting FAILed mails. While senders with an SPF FAIL can be expected to know what it means, the same is obviously not the case for a receiver arranging his forwarding in a way that cannot work with SPF.

Checkpoints

Checking SPF behind the "border" MTA (MX) is not impossible, the relevant info is usually noted in a Received timestamp line added by one of the MXs of the receiver. But at this time it's too late to reject SPF FAIL, the checking entity can essentially only delete FAILing mail. A mail transfer agent or MTA (also called a mail transport agent, mail server, or a mail exchanger in the context of the Domain Name System) is a computer program or software agent that transfers electronic mail messages from one computer to another. ... An MX record or Mail exchange record is a type of resource record in the Domain Name System (DNS) specifying how Internet e-mail should be routed. ...

Experts should be able to get this right, but it's no plug and play situation, therefore the SPF specification recommends to check SPF only at the "border" (MX) in the SMTP session, not later. Later SPF checks can also have unexpected results if the publishers of sender policies don't plan modifications of their policy carefully with regard to DNS cache expiration. It has been suggested that this article be split into multiple articles. ...

DoS attack

A 2006 IETF Internet-Draft, SPF DoS Exploitation, discusses concerns related to the scale of an SPF answer leading to network exploits as a means to corrupt the DNS. This issue is also covered in the security considerations of the SPF RFC. The SPF project did a detailed Analysis of this draft and concluded that SPF does not pose any unique threat of DNS DoS.

What remains overlooked in this conclusion is that although there is a limit of 10 SPF mechanisms, each mechanism may invoke 10 queries targeting a victim for a total of 100 transactions per name resolved. In addition, the local-part macro can be employed to randomize subsequent queries, where none of the spammers resources are consumed. Any and all such traffic then represents an infinite gain DNS amplification attack. The spammer is able to spam and stage a completely free attack. The magnitude of this possible exploit is not common in other protocols.

History

The SPF concept was presented at YAPC and OSCON (O'Reilly Open Source Convention) in 2003, in a short paper titled "Repudiating Mail-From" written by Paul Vixie in 2002. Other predecessors were "Reverse MX" by Hadmut Danisch, and "Designated Mailer Protocol" by Gordon Fecyk. Yet Another Perl Conference, usually given as the abbreviation YAPC, are conferences discussing the Perl computer programming language, usually organised under the auspices of the Yet Another Society, a non-profit corporation for the advancement of collaborative efforts in computer and information sciences. External links YAPC YAPC Europe {stub} ... An OSCON (Open Source Convention) is an annual convention to talk about Open Source software, such as Linux, MySQL, Perl, and Python. ... Paul Vixie is the author of several RFCs and well known UNIX system programs, among them SENDS, proxynet, rtty and Vixie cron. ... Also see: 2002 (number). ... An MX record or Mail exchange record is a type of resource record in the Domain Name System (DNS) specifying how Internet e-mail should be routed. ...

In June 2003, Meng Weng Wong merged the RMX and DMP specifications and solicited suggestions from other programmers. Over the next six months, a large number of changes were made and a large community had started working on SPF. Meng Weng Wong Meng Weng Wong 黃銘榮 is a serial entrepreneur. ...

Originally SPF stood for Sender Permitted From and was sometimes also called SMTP+SPF, but it was changed to Sender Policy Framework in February 2004.

In early 2004, the IETF created the MARID working group and tried to use SPF and Microsoft's CallerID proposal as the basis for what is now known as Sender ID. The Internet Engineering Task Force (IETF) is charged with developing and promoting Internet standards. ... A Marid (Arabic : مارد ) in common mythology is a djinn related to the element of water. ... Sender ID is an anti-spam proposal from the MARID IETF working group, joining Sender Policy Framework and Caller ID. The Sender ID proposal is currently the subject of controversy regarding intellectual property and licensing issues: Microsoft holds patents on key parts of Sender ID and licenses those patents under...

After the collapse of MARID the SPF community returned to the original "classic" version of SPF. In July 2005 this version of the specification was approved by the IESG as an IETF experiment. On April 28th, 2006, the SPF RFC was published as RFC 4408. A Marid (Arabic : مارد ) in common mythology is a djinn related to the element of water. ... The Internet Engineering Steering Group is a body composed of the Internet Engineering Task Force Chair and Area Directors: Internet Area (int) Operations & Management Area (ops) Routing Area (rtg) Security Area (sec) Transport Area (tsv) Temporary Sub-IP Area (sub) and so on. ... Alternate meaning: Wikipedia:Requests for comment A Request for Comments (RFC) document is one of a series of numbered Internet informational documents and standards very widely followed by both commercial software and freeware in the Internet and Unix communities. ...

Controversy

Steven M. Bellovin has written a long e-mail dated January 5, 2004, discussing some of his concerns with SPF. Some of these include: Steven M. Bellovin is a researcher on networks, security and why the two do not get along. ...

• SPF uses TXT records in DNS, which are supposed to be free-form text with no semantics attached. SPF proponents readily acknowledge that it would be better to have records specifically designated for SPF, but this choice was made to enable rapid implementation of SPF. In July 2005, IANA assigned the Resource Record type 99 to SPF. SPF publishers may publish both record types and SPF checkers may check for either types. It will likely take many years before all DNS software fully supports this new record.
• As of the time he wrote his message, there was no consensus that this is the right way to go. Some major e-mail service providers have not bought into this scheme. Unless and until they do, it doesn't help much, either for their customers (who make up a substantial proportion of the user population) or for everyone else (since their addresses could be forged). It's worth noting that since this concern was raised, among others Google's GMail and AOL have embraced SPF.
• Bellovin's strongest concerns involve the underlying assumptions of SPF (SPF's "semantic model"). When using SPF, the SPF DNS records determine how a sender is allowed to send. That means that the owner of the domain will control how senders are allowed to send. People who use "portable" e-mail addresses (such as e-mail addresses created by professional organizations) will be required to use the domain owner's SMTP sender, which may not currently even exist. Organizations providing these "portable" addresses could, however, create their own Mail Submission Agents (MSAs) (RFC 4409) or offer VPNs. Besides SPF only ties the SMTP Return-Path to permitted MSAs; users are still free to use their RFC 2822 addresses elsewhere.

Jonathan de Boyne Pollard has written a separate diatribe against SPF that discusses alleged SPF's conflicts with mail RFCs, its ability to force ISPs to force their customers to use mail in particular ways, and the forwarding issue. The Internet Assigned Numbers Authority (IANA) is the entity that oversees global IP address allocation, DNS root zone management, and other Internet protocol assignments. ... A virtual private network (VPN) is a communications network tunneled through another network, and dedicated for a specific network. ... Simple Mail Transfer Protocol (SMTP) is the de facto standard for e-mail transmissions across the Internet. ...

Deployment

Despite its limitations, many people have decided that the pros of SPF outweigh its cons and have begun implementing SPF. Anti-spam software such as SpamAssassin version 3.0.0 and ASSP implement SPF. Many mail transfer agents (MTAs) support SPF directly such as Courier, CommuniGate Pro, Wildcat, Microsoft Exchange, and SmarterMail, or have patches/plug-ins available that support SPF, including Postfix, Sendmail, Exim, and Qmail. Many prominent domains decided to post SPF data for their domains as of mid-2004, including Amazon, AOL, EBay, Google, GMX, Hotmail, Microsoft, and W3C. SpamAssassin is a computer program released freely under the Apache License 2. ... The Anti-Spam SMTP Proxy (ASSP) server project is an Open Source platform-independent transparent SMTP proxy server that leverages numerous methodologies and technologies to both rigidly and adaptively identify spam. ... A mail transfer agent or MTA (also called a mail transport agent, mail server, or a mail exchanger in the context of the Domain Name System) is a computer program or software agent that transfers electronic mail messages from one computer to another. ... The Courier mail transfer agent (MTA) is an integrated mail/groupware server based on open commodity protocols, such as ESMTP, IMAP, POP3, LDAP, SSL/TLS, and HTTP. Courier provides ESMTP, IMAP, POP3, webmail, and mailing list services within a single, consistent framework. ... CommuniGate Pro is a Stalker software based Internet Communications Server providing various protocol services. ... Wildcat! is a bulletin board system (BBS) software package developed in 1986 by Mustang Software to create dial-up BBS operating under PC-DOS. It was later ported to Microsoft Windows. ... Microsoft Exchange can mean: Microsoft Exchange Server Microsoft Exchange Client: the native and bundled client of Exchange server up to version 5. ... Postfix is a free software / open source mail transfer agent (MTA), a computer program for the routing and delivery of email. ... Sendmail is a mail transfer agent (MTA) that is a well known project of the open source, free software and Unix communities, which is distributed both as free software and proprietary software. ... Exim is a mail transfer agent (MTA) used in Unix-like operating systems. ... qmail is a mail transfer agent that runs on Unix. ... Amazon. ... It has been suggested that AOL search data scandal be merged into this article or section. ... eBay headquarters in San Jose eBay North First Street satellite office campus (home to PayPal) eBay Inc. ... Google Inc. ... Hotmail is a free webmail e-mail service, which is accessible via a web browser. ... Microsoft Corporation, (NASDAQ: MSFT, HKSE: 4338) is a multinational computer technology corporation with global annual revenue of US$44. ... The World Wide Web Consortium (W3C) is a consortium that produces standards—recommendations, as they call them—for the World Wide Web. ...

In a survey published 2007 5% of the .com and .net domains had some kind of SPF policy - this might include overall useless policies like v=spf1 ?all.^[1] 2007 is a common year starting on Monday of the Gregorian calendar. ...

See also

• E-mail authentication overview
• DomainKeys
• MARID
• Sender ID
• Sender Rewriting Scheme
• Sender Signing Policy

Ensuring a valid identity on an e-mail has become a vital first step in stopping spam, forgery, fraud, and even more serious crimes. ... DomainKeys is an e-mail authentication system designed to verify the DNS domain of an E-mail sender and the message integrity. ... A Marid (Arabic : مارد ) in common mythology is a djinn related to the element of water. ... Sender ID is an anti-spam proposal from the MARID IETF working group, joining Sender Policy Framework and Caller ID. The Sender ID proposal is currently the subject of controversy regarding intellectual property and licensing issues: Microsoft holds patents on key parts of Sender ID and licenses those patents under... The introduction to this article provides insufficient context for those unfamiliar with the subject matter. ... In computing Sender Signing Policy (SSP) is a proposed part of the E-mail authentication scheme DomainKeys (now DKIM). ...

External links

• SPF homepage and news
• SPF Testing Site
• Mailing list archives
• SPF Testing site (Page is blank June 23, 2007)
• ONLamp report about DNS extensions (January 2007)
• emailbattles: SPF Claws Sender-ID (Aug 2005)
• Canadian recommendation for ISPs (May 2005)
• Interview with co-author W. Schlitt (Mar 2005)
• David Woodhouse discusses flaws in SPF (Jan 2005)
• SpamCop FAQ entry about bogus bounces discusses also SPF (2005)
• M. Wong's Deployment White Paper (PDF, Nov 2004)
• The Register: Spammers embrace email authentication (September 2004)
• CircleID interview with co-author M. Wong (Jun 2004)
• Brad Knowles considers SPF as harmful (May 2004)
• Jonathan de Boyne Pollard's list of the flaws in SPF (2004)
• Diagram of e-mail flow and SPF's impact (PDF, PNG )
• MIT Spam Conference Handout on SPF (Jan 2004)
• Steve Bellovin expresses doubts (Jan 2004)