We get asked a lot about A-B testing for e-mail marketing. When you are sending to a large group, testing is a vital part of the process. However, when your list is so small that you would have to be testing groups of 20 or smaller, the results won’t be as clear, so it’s smarter to just send to the whole group.
Ideally, each test group should be at least 100 contacts. if you have an extremely large contact list, bigger is okay. In fact, the bigger the test groups, the easier it will be to see distinct differences.
Remember that a good open rate is somewhere around 20% , so with groups of 100, you will probably be seeing 10-20 opens per groups. You want to be able to judge the different groups by the number of opens and the number of link clicks. If one group has 10 opens and one has 20 you can clearly see that the group that had 20 opens yielded a better result. However, if you are testing groups of 1,000 and one group has 100 opens versus a group that has 200 opens, that’s a bigger difference so you can feel more confident that the test group with 200 opens is a significant difference.
Here are a few items that should be A-B tested
Subject Line
This is what directly influences someone to open an e-mail. If you only test one thing, this should be it. Subject lines ideally should be straight forward and not try to be too creative. Test out saying what is in your e-mail a few different ways and see what people respond to the best.
Content
Your content should also be short and to the point. If you want to send out an article or e-book, send out a link to it, not the whole thing. That said, make a couple different versions of your email. Keep in mind that many people either click to open the email in their browser and don’t scroll down or open it on a mobile device. Because of this, you need to keep the most important information in the first paragraph.
Image versus no Image
This can be a tricky one. People like things that are visual, but many people have images blocked in their e-mail browser. Because this can be hit or miss, it is a good thing to test putting an image in the e-mail for one group and keeping it out for the others.
Merge Fields
Putting merge fields (like someone’s first name) with in the subject line can actually yield more opens– or look super cheesy. You won’t know how people will respond until you test it, and from our experience it is something worth trying.
