The Dark Art of Email Deliverability

We often get questions over the technologies which help get your legitimate email delivered.  There are three main ones:

• SPF – Used to lock down email sources to your servers
• DKIM – To authenticate your sent emails
• DMARC – For real world email reporting and DKIM enforcement

Email is vital to any organisation. Ensuring your emails are delivered can seem like a dark art. There are many systems at play to cut down on spam and other fraudulent emails. Over 80% of the email we receive at Dogsbody Technology is filtered out as SPAM. It can be hard for a system to recognise your legitimate email.

This makes email one of the more difficult technologies to get right.

“The Dark Arts are many, varied, ever-changing, and eternal”.
– Professor Severus Snape

Foreword

Our example features three characters: Alice, Bob and Sybil.

• Alice is sending emails to Bob.
• Bob receiving emails through Google’s Gmail.
• Sybil is sending emails to Bob pretending to be Alice.

SPF, DKIM and DMARC is checked by the receiving email system and will affect how it handles the email. Larger email providers: Gmail, Hotmail, Yahoo, etc., are more likely to check the use of these technologies as they suffer from more SPAM than smaller hosts.

Sender Policy Framework; SPF

SPF is a DNS record which identifies all email sources permitted for your domain.

When an email is received the SPF record is looked up by the receiving mail server and the source IP from the email is then cross-referenced to check that it is in the SPF record.

Example SPF record for Alice’s server, example.com:

example.com. IN TXT "v=spf1 ip4:198.51.100.13 ~all"

e.g. when Alice sends an email from her domain (example.com) to Bob, Gmail receives this email and gets the source IP (198.51.100.13). Gmail then looks up the SPF record for ‘example.com’ which lists ‘198.51.100.13’ as an authorised server. Gmail knows Alice has allowed this and passes the email onto Bob.

When Sybil emails Bob as ‘example.com’ (Alice’s domain), Gmail receives the SPAM email and gets the emails source IP (203.0.113.37). When Gmail looks up the SPF record for ‘example.com’ it doesn’t list ‘203.0.113.37’ so Gmail will flag the email as SPAM.

This means that all sources sending email for your domain needs to be listed. This may include Mandrill, Google Apps for business, Socket labs, AWS SES, FreeAgent, etc.

DomainKey Identified Mail; DKIM

DKIM is a signature added to the header of an email before it is sent into the public internet. This signature shows that you are taking ownership of your email and allows any recipients to prove that.

The DKIM signature is generated with a private key and is based on the email’s contents, this make it unique to you and your email. For the receiving server to authenticate the signature a public key related to the private key is published in a DNS record. With the public key the receiving server can confirm that the DKIM signature was generated by your key. If the public key doesn’t match the signature the email will be considered as SPAM.

Example DKIM record for Alice’s server, example.com:

selector._domainkey.example.com IN TXT "v=DKIM1; p=MIGfMA0GCSq..."

For example when Alice sends an email to Bob, her server will generate and add a DKIM header to it. When Gmail receives this email it sees the DKIM header and looks up the DKIM public key record. It will check the DKIM signature against the public key. Since the email was sent through Alice’s servers the signature matches and gmail will pass the email onto Bob.

When Sybil emails Bob as Alice, Sybil’s server can generate a DKIM header, but when Gmail receives it and looks up the Signature against example.com (Alice’s) DKIM public key it will not match and Gmail will flag the email as spam. This also means that if Sybil intercepts a signed email before it reaches Gmail and changes its contents, the signature will no longer match which will flag Gmail that it had been changed.

Note that DKIM is only checked reactively, when the header is seen in an email. This is unlike SPF records which Gmail will check pro-actively.

Domain-based Message Authentication, Reporting & Conformance; DMARC

DMARC is a DNS record containing information regarding which anti-spam technologies the sending domain is using, and telling receivers to report back what they see.

When an email is received the receiving server looks up the DMARC record for the sender. This record includes flags detailing if SPF and/or DKIM are in use. The receiving server is then able to use this extra information to ensure that the received email is authenticated as the sender intended. If an email comes in which doesn’t meet the SPF and DKIM information provided by the DMARC record, the DMARC record also specifies guidance of how the receiver should act in the case of an authentication failure. e.g. If the message should be let through, put in junk or rejected completely.

Additionally, each ISP that supports DMARC is encouraged to send a report to the contact specified in the DMARC record. The report shows totals of email received and how the ISP dealt with these emails. The ISP can send a report at any time after it starts receiving emails, this is often daily.

Example DMARC record for Alice’s server, example.com:

_dmarc.example.com IN TXT "v=DMARC1; p=quarantine; rua=mailto:dmarc@example.com;"

e.g. When Alice sends an email to Bob, Gmail will receive her email and look up the example.com DMARC record. Gmail will then check that the email it received uses DKIM and SPF to authenticate as set in the DMARC record. It does and Gmail will pass it onto Bob.

If Sybil sends an email as example.com to Bob, Gmail will receive it and look up the DMARC record for example.com. It then checks that SPF and DKIM were used to authenticate the email and it will fail as neither SPF or DKIM will match the DMARC record. Gmail may then follow the DMARC policy ‘quarantine’ and put Sybil’s email into Bobs Junk mail folder. Gmail can then report to Alice that it received an email from the server ‘203.0.113.37’ (example.net) spoofing example.com, and that it failed SPF and DKIM checks.

DMARC reports will come from each receiving server and a spammer can easily send thousands and thousands of emails in an instant. These reports can quickly build up. Dmarcian is a tool which will break down these report into usable graphs and summaries helping you understand them.

These reports show all sources of emails sent from the domain and if DKIM and/or SPF was used to authenticate them, providing real world information on how your domain’s emails are being received out on the internet. It shows if there are any unknown sources impersonating the domain. All of this helps to understand the domain’s email reputation and how deliverable emails are.

DMARC gives you information. Information is power.

Conclusion

Globally SPAM has been going down and it will continue to do so only if more email systems implement these technologies. These have been just one example of how SPF, DKIM and DMARC can protect a domain. There are many more implementations that have their own specific requirements. If you want to improve the deliverability or want more information on how your emails looks to others, these technologies will give you this advantage. Don’t let Sybil spoof you.

Let us build your perfect email infrastructure.