Langa Letter: E-Mail--Hideously Unreliable - InformationWeek
Software // Enterprise Applications
10:17 PM
Fred Langa
Fred Langa

Langa Letter: E-Mail--Hideously Unreliable

A recent test by InformationWeek columnist Fred Langa shows that up to 40% of valid E-mails never reach the recipient. Here's what it all means to you.

The Test
The basic test concept was simple: I'd send one plain text, attachment-free E-mail to each volunteer. The content of the E-mail would simulate normal, safe, business or interpersonal correspondence. It would contain no deliberate or obvious spam or virus-filter triggers (e.g., no spamlike components, such as offers to enlarge this or shrink that; no attachments; no viruses; no HTML; no embedded scripts; etc.). The subject line also would be plain and general, neither designed to trigger nor avoid spam filters. To count as a successful delivery, the test E-mail had to be seen and responded to by a human; not by a mailbot, responder, or other auto-reply mechanism. The body of the message contained instructions for the volunteer either to send a new, original E-mail, or to forward the test E-mail to a unique reply mailbox. This was similar to the original sign-up process the volunteers had already used to join the test, and so wasn't a high hurdle. But it did ensure that simple automatic reply mechanisms couldn't skew the test results. Instead, only human-generated responses that arrived at the designated reply mailbox would count as a complete, successful communication.

I took steps to ensure that the test E-mails would be treated completely normally; subject to all the standard E-mail processing and filters that might be in place at the volunteers' ISPs, mail servers, or desktops. That meant that I wouldn't send the E-mails from a "" or other address that the volunteers might already associate with me, or already have in their "whitelists." (Whitelisted addresses are known to be safe; mail from those addresses is preapproved, and bypasses normal filtering.) So, I made arrangements to send the test E-mails from a name and address the volunteers hadn't seen before.

I'd planned the above basic approach when I thought I'd have maybe 500 volunteers. But with more than 10,000 volunteers available, I also was able to add tests for several additional E-mail variables. I did this by splitting the pool of volunteers into large subgroups. Each subgroup got a slightly different test E-mail; or got E-mails that were sent out in slightly different ways. (I'll detail the variables in a moment.) Each subgroup was given its own unique mailbox to reply to, so I could track the response rates separately.

I sent all 10,979 test E-mails from a private E-mail account, using normal desktop E-mail tools, during East Coast business hours on Nov. 17 and Nov. 18, a Monday and a Tuesday. I left the response mailboxes--the mailboxes by which the volunteers could acknowledge that they got the test E-mail--open for a week. I ran no mail or spam filters on the response mailboxes so all acknowledgements would be delivered untouched and intact. Recall that all the volunteers were expecting some kind of test E-mail to arrive, although they didn't know exactly when, where, or how. Recall also that these were motivated, E-mail-savvy test subjects. Given this, the overall response was astonishingly poor: Of the 10,979 test E-mails I sent, I received 6,551 acknowledgements; a gross success rate of just 60%; or more pointedly, a failure rate of 40%.

But not all the subgroups yielded the same performance. To see and understand the differences and to have a basis for analyzing the gross results, we need to go into more detail:

Test Group One
This test group was closest to the small-scale test I'd originally envisioned; intended to simulate an original, unanticipated (i.e. non-whitelisted or preapproved) correspondence from an unknown, but nonhostile personal sender.

This test group comprised 1,500 individually composed, addressed, and mailed messages. (This was by far the most labor-intensive test.) The messages were in plain text with no attachments and sent one at a time. Each message had one recipient--one of the volunteers. (I used the volunteers' addresses in the order in which I'd received them--first volunteer, second volunteer, third volunteer, etc.) The E-mails were sent from a name and E-mail account the volunteers had not seen before: Liam Sugob, or (a temporarily valid personal and E-mail address I set up at my Freetune.Com domain). The E-mails carried a totally generic subject line of "Hello."

2 of 6
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of the Cloud Report
As the use of public cloud becomes a given, IT leaders must navigate the transition and advocate for management tools or architectures that allow them to realize the benefits they seek. Download this report to explore the issues and how to best leverage the cloud moving forward.
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on for the week of November 6, 2016. We'll be talking with the editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll