Langa Letter: E-Mail--Hideously Unreliable - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Enterprise Applications
10:17 PM
Fred Langa
Fred Langa

Langa Letter: E-Mail--Hideously Unreliable

A recent test by InformationWeek columnist Fred Langa shows that up to 40% of valid E-mails never reach the recipient. Here's what it all means to you.

The Test
The basic test concept was simple: I'd send one plain text, attachment-free E-mail to each volunteer. The content of the E-mail would simulate normal, safe, business or interpersonal correspondence. It would contain no deliberate or obvious spam or virus-filter triggers (e.g., no spamlike components, such as offers to enlarge this or shrink that; no attachments; no viruses; no HTML; no embedded scripts; etc.). The subject line also would be plain and general, neither designed to trigger nor avoid spam filters. To count as a successful delivery, the test E-mail had to be seen and responded to by a human; not by a mailbot, responder, or other auto-reply mechanism. The body of the message contained instructions for the volunteer either to send a new, original E-mail, or to forward the test E-mail to a unique reply mailbox. This was similar to the original sign-up process the volunteers had already used to join the test, and so wasn't a high hurdle. But it did ensure that simple automatic reply mechanisms couldn't skew the test results. Instead, only human-generated responses that arrived at the designated reply mailbox would count as a complete, successful communication.

I took steps to ensure that the test E-mails would be treated completely normally; subject to all the standard E-mail processing and filters that might be in place at the volunteers' ISPs, mail servers, or desktops. That meant that I wouldn't send the E-mails from a "" or other address that the volunteers might already associate with me, or already have in their "whitelists." (Whitelisted addresses are known to be safe; mail from those addresses is preapproved, and bypasses normal filtering.) So, I made arrangements to send the test E-mails from a name and address the volunteers hadn't seen before.

I'd planned the above basic approach when I thought I'd have maybe 500 volunteers. But with more than 10,000 volunteers available, I also was able to add tests for several additional E-mail variables. I did this by splitting the pool of volunteers into large subgroups. Each subgroup got a slightly different test E-mail; or got E-mails that were sent out in slightly different ways. (I'll detail the variables in a moment.) Each subgroup was given its own unique mailbox to reply to, so I could track the response rates separately.

I sent all 10,979 test E-mails from a private E-mail account, using normal desktop E-mail tools, during East Coast business hours on Nov. 17 and Nov. 18, a Monday and a Tuesday. I left the response mailboxes--the mailboxes by which the volunteers could acknowledge that they got the test E-mail--open for a week. I ran no mail or spam filters on the response mailboxes so all acknowledgements would be delivered untouched and intact. Recall that all the volunteers were expecting some kind of test E-mail to arrive, although they didn't know exactly when, where, or how. Recall also that these were motivated, E-mail-savvy test subjects. Given this, the overall response was astonishingly poor: Of the 10,979 test E-mails I sent, I received 6,551 acknowledgements; a gross success rate of just 60%; or more pointedly, a failure rate of 40%.

But not all the subgroups yielded the same performance. To see and understand the differences and to have a basis for analyzing the gross results, we need to go into more detail:

Test Group One
This test group was closest to the small-scale test I'd originally envisioned; intended to simulate an original, unanticipated (i.e. non-whitelisted or preapproved) correspondence from an unknown, but nonhostile personal sender.

This test group comprised 1,500 individually composed, addressed, and mailed messages. (This was by far the most labor-intensive test.) The messages were in plain text with no attachments and sent one at a time. Each message had one recipient--one of the volunteers. (I used the volunteers' addresses in the order in which I'd received them--first volunteer, second volunteer, third volunteer, etc.) The E-mails were sent from a name and E-mail account the volunteers had not seen before: Liam Sugob, or [email protected] (a temporarily valid personal and E-mail address I set up at my Freetune.Com domain). The E-mails carried a totally generic subject line of "Hello."

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
2 of 6
Comment  | 
Print  | 
More Insights
Get Your Enterprise Ready for 5G
Mary E. Shacklett, Mary E. Shacklett,  1/14/2020
Modern App Dev: An Enterprise Guide
Cathleen Gagne, Managing Editor, InformationWeek,  1/5/2020
9 Ways to Improve IT and Operational Efficiencies in 2020
Cynthia Harvey, Freelance Journalist, InformationWeek,  1/2/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Flash Poll