PrintWednesday, July 29, 2009
Email Testing Layers Peeled Back
According to the eROI newest study on the Use of Analytics in Email Marketing Campaigns, of the 37% of marketers who don't test their email campaigns, 33% say it's because they do not know how, 27% say they don't have time and 13% say their platform doesn't have testing capabilities. For those that are testing, 85.2% test subject lines, 54.8% test calls to action, 50.9% test designs, 49.1% test copy, 41.7% test offers, 36% test timing of campaigns.
With 73% of marketers planning to increase email as a priority in their future marketing plans, eROI set out to discover why more than a third of marketers are not testing their campaigns, and for those that are testing, what is being tested and its perceived relative importance.
The study notes that in a 2006 Marketing Sherpa article by Anne Holland survey results proved that testing increases ROI. The responses show that "in every case more than 50% of marketers improved ROI (even if only moderately) by testing." The current eROI study uncovered some of the main reasons for not testing.
Reasons Some Marketers Are Not Testing Email Campaigns
Reason |
% of Respondents |
Don't know How |
32.84% |
My Campaign Timeline Is Too Short |
27.36% |
Platform Doesn't Have Testing Capabilities |
13.43% |
Other |
11.94% |
I Don't See The Value In It |
8.46% |
Platform Doesn't Have Capabilities I Want |
5.97% |
Source: eROI Testing in Email Campaigns, July 2009
For those that are testing, they are testing a wide variety of elements including design, frequency, calls to action and day/time sent. The report shows this breakdown for email content testing:
• 85.2% test subject lines
• 54.8% test calls to action
• 50.9% test designs
• 49.1% test copy
• 41.7% test offers
• 36.8% test timing of campaigns
The survey unveiled the latest trends in testing time of day releases, with 49.4% currently find sending mid-day (10am-2pm) to be best, while start of the business day (6AM - 10AM) showed to be second best:
Time of Day For Testing (% of Respondents)
| Day Part | % Preferred |
| Mid-Day | 49.4% |
| Start Of Business Day | 35.06% |
| Early Morning | 9.09% |
| End Of Business Day | 3.90% |
| Early Evening | 2.60% |
Source: eROI Testing in Email Campaigns, July 2009
34.91% of marketers that test timing are testing frequency. Of those, 33.3% send weekly, 28.21% send bi-monthly, 28.21% send as necessary (no set frequency) and the rest are divided equally between sending quarterly, daily, more than once a week and monthly.
Frequency That Email Marketers Test Their Email Campaigns
(% of Respondents Testing Frequency)
| Frequency | % Testing |
| Weekly | 33.33% |
| As necessary | 28.21% |
| Bi-monthly | 28.21% |
| Daily | 2.56% |
| Monthly | 2.56% |
| More than once a week | 2.56% |
| Quarterly | 2.56% |
Source: eROI Testing in Email Campaigns, July 2009
As a check list and comparison to individual practice, the study includes an appendix that shows statistics on which variables are most often tested by marketers on each element of their emails. The report concludes that the reader might find a new testing opportunity for the next campaign.
Variables Tested By Marketers (% of Those Testing Each Element)
| Element Tested | % Testing |
| Subject Line | |
| Tone/personality | 40.81% |
| Length | 33.55% |
| Personalization | 20.09% |
| Other | 5.56% |
| Body Copy | |
| Length | 29.71% |
| Headlines | 28.29% |
| Tone | 20.00% |
| Personalization | 19.14% |
| Other | 2.86% |
| Offer Variables | |
| Limited (time or quantity) | 28.69% |
| Percentage vs. Dollar | 25.8% |
| Perceived value vs. Price | 20.08% |
| Free vs. Reduced | 14.75% |
| Live content | 4.51% |
| Other | 6.45% |
| Call to Action | |
| Type (links, buttons, banners, text only) | 29.06% |
| Copy | 26.85% |
| Placement | 26.60% |
| Number of placements | 16.26% |
| Other | 1.23% |
| Design/image variables | |
| Image Placement | 25.39% |
| Amount of imagery | 17.15% |
| Complexity | 16.93% |
| Background | 11.58% |
| Accent color/shading | 9.58% |
| Text only | 9.58% |
| Dynamic Imagery | 8.24% |
| Other | 1.56% |
| Other Elements | |
| Text vs .html | 22.58% |
| From name | 15.75% |
| None | 13.47% |
| Pre-headers | 12.9% |
| From address | 10.44% |
| Alt-tags | 7.78% |
| Emailwidth | 7.78% |
| Footers | 7.59% |
| Other | 1.56% |