How to optimize email campaigns? Part I

When a data scientist is asked to optimize an email campaign, it just means improving targeting, finding best time / day to send the message, improving segmentation of contact lists to get better metrics (opens, clicks, conversions, opens-to-clicks ratio, clicks-to-conversion ratio, revenue per lead etc.)

However this is only one aspect of email campaign optimization. Full optimization consists of

Improve targeting and create better segments - what data scientists do (discussed above)
Improve deliverability - what software engineer and computer scientists do, see below
Improve subject line, landing page, number of clickable links and position within the email message, and wording / text in general - what marketing people do, but I believe data scientists should be involved too, e.g. to process unstructured data, create email categories, perform "on the fly" A/B testing

These three aspects are (unfortunately) always treated as separate silos handled by very different teams in any company (analytics team for #1, engineering / IT or vendor for #2, and product / marketing for #3). Here I discuss how some aspects of #2 can be handled by a data scientist.

Case study: how a data scientist reduced email bounce rate from 30% to 17%

High bounce rate typically occurs with imported lists. Improving open rate / bounce rate / unsubscribe rate is critical and should be the #1 priority. Without perfect deliverability metrics, your email ends up in junk boxes, unread, spam filters and may not be delivered at all by Gmail, Hotmail, Yahoo or some other providers, resulting in very poor performance no matter how great your text message is. Also, it will be very hard for you to grow your list.

Anyway, here's how we improved our bounce rate. A more complicated methodology (because it involves the two other silots) applies to unsubscribe and open rates. Our mailing list in question had about 1.4MM contacts.

Algorithm:

Step A (test)

Test a message on the first 100K contacts (randomly selected from the 1.4MM list)

Compute bounce rate for each domain name (e.g. ibm.com)
Compute bounce rate for each domain extension (e.g. .com)
Compute bounce rate for small domains (domain names with < 3 contacts)
Create a blacklist of extensions or domain names with a bounce rate > 40%, based on the 100K messages

Step B (outside the test)

When sending messeges outside the 100K test, delete email addresses from small domains (as their bounce rate was far above average), and from blacklisted extensions and domain names.

Note that by doing so, you will also rerroneously delete good contacts (false positives) from your mailing list. But by carefully choosing the algorithm threshold parameters (40% and 3 in this example), the benefits will far outweight the drawbacks.

1 member likes this

▶ Reply to This

Replies to This Discussion

Permalink Reply by Mirko Krivanek on August 21, 2012 at 4:49am

Regarding #3 (improve subject line etc.), it might work better to not overuse keywords or subject lines that are working very well, as their efficiency decreases exponentially over repeated uses. Also, the first sentence of your message is critical: this sentence will help the user decide whether or not she is going to read your message, delete it or report it as spam. In many email clients, all the user sees before opening a message is the subject line, the first line of the message and the "from" field (email address + name). The from field is also very important.