A recent test by InformationWeek columnist Fred Langa shows that up to 40% of valid E-mails never reach the recipient. Here's what it all means to you.
The basic test concept was simple: I'd send one plain text, attachment-free E-mail to each volunteer. The content of the E-mail would simulate normal, safe, business or interpersonal correspondence. It would contain no deliberate or obvious spam or virus-filter triggers (e.g., no spamlike components, such as offers to enlarge this or shrink that; no attachments; no viruses; no HTML; no embedded scripts; etc.). The subject line also would be plain and general, neither designed to trigger nor avoid spam filters.
To count as a successful delivery, the test E-mail had to be seen and responded to by a human; not by a mailbot, responder, or other auto-reply mechanism. The body of the message contained instructions for the volunteer either to send a new, original E-mail, or to forward the test E-mail to a unique reply mailbox. This was similar to the original sign-up process the volunteers had already used to join the test, and so wasn't a high hurdle. But it did ensure that simple automatic reply mechanisms couldn't skew the test results. Instead, only human-generated responses that arrived at the designated reply mailbox would count as a complete, successful communication.
I took steps to ensure that the test E-mails would be treated completely normally; subject to all the standard E-mail processing and filters that might be in place at the volunteers' ISPs, mail servers, or desktops. That meant that I wouldn't send the E-mails from a "langa.com" or other address that the volunteers might already associate with me, or already have in their "whitelists." (Whitelisted addresses are known to be safe; mail from those addresses is preapproved, and bypasses normal filtering.) So, I made arrangements to send the test E-mails from a name and address the volunteers hadn't seen before.
I'd planned the above basic approach when I thought I'd have maybe 500 volunteers. But with more than 10,000 volunteers available, I also was able to add tests for several additional E-mail variables. I did this by splitting the pool of volunteers into large subgroups. Each subgroup got a slightly different test E-mail; or got E-mails that were sent out in slightly different ways. (I'll detail the variables in a moment.) Each subgroup was given its own unique mailbox to reply to, so I could track the response rates separately.
I sent all 10,979 test E-mails from a private E-mail account, using normal desktop E-mail tools, during East Coast business hours on Nov. 17 and Nov. 18, a Monday and a Tuesday. I left the response mailboxes--the mailboxes by which the volunteers could acknowledge that they got the test E-mail--open for a week. I ran no mail or spam filters on the response mailboxes so all acknowledgements would be delivered untouched and intact.
Recall that all the volunteers were expecting some kind of test E-mail to arrive, although they didn't know exactly when, where, or how. Recall also that these were motivated, E-mail-savvy test subjects. Given this, the overall response was astonishingly poor: Of the 10,979 test E-mails I sent, I received 6,551 acknowledgements; a gross success rate of just 60%; or more pointedly, a failure rate of 40%.
But not all the subgroups yielded the same performance. To see and understand the differences and to have a basis for analyzing the gross results, we need to go into more detail:
Test Group One
This test group was closest to the small-scale test I'd originally envisioned; intended to simulate an original, unanticipated (i.e. non-whitelisted or preapproved) correspondence from an unknown, but nonhostile personal sender.
This test group comprised 1,500 individually composed, addressed, and mailed messages. (This was by far the most labor-intensive test.) The messages were in plain text with no attachments and sent one at a time. Each message had one recipient--one of the volunteers. (I used the volunteers' addresses in the order in which I'd received them--first volunteer, second volunteer, third volunteer, etc.) The E-mails were sent from a name and E-mail account the volunteers had not seen before: Liam Sugob, or Liam@freetune.com (a temporarily valid personal and E-mail address I set up at my Freetune.Com domain). The E-mails carried a totally generic subject line of "Hello."
Building A Mobile Business MindsetAmong 688 respondents, 46% have deployed mobile apps, with an additional 24% planning to in the next year. Soon all apps will look like mobile apps – and it's past time for those with no plans to get cracking.