What You’ll Learn
- How A/B testing can help you send better performing emails
- How A/B testing works in Campaigns451
- How to conduct an A/B test
How A/B Testing Works, What to Test
A/B Testing is a communication type in Campaigns451 that lets you send versions of an email to a portion of your audience, and then automatically sends the “winning” version to the rest of your recipients.
You can test:
- Up to 3 subject lines
- Up to 3 sender names
- 2 versions of the content
When you set up your test, you will select what percentage of your audience should be “testers.” For A/B tests, you will send to a segment rather than a workflow. The test audience won’t know they are part of a test; the email will look like any other email you send in Campaigns451.
After the message is sent and the recipients have had time to open it and click links, the version with the better response rate will be labeled the winner and will be sent to the remaining people in the chosen segment.
Create an A/B Test
Creating an A/B testing communication is similar to creating a basic email. When you reach the step where you select the communication type, choose A/B Testing, and start following these steps:
1. Choose which variable you want to test:
Subject Lines: Choose 2 or 3. You'll enter the text in the Setup tab.
From Names: Choose 2 or 3. You'll enter the text in the Setup.
Content: You can test 2 versions of the content. Enter a short description for each version. You'll add the content in the Content tab.
2. If you're testing the subject line or from name, go to the Setup tab. You'll see fields for the number of versions you're testing. Enter the text for each version of the subject line or sender name, depending on which variable you're testing.
3. If you're testing content, go to the Content tab. If you haven't done so yet, follow the steps for choosing a template in Create an Email. With your template in place, click on the row you want to have different versions of and select one of these options:
Add Variation (Duplicate)
Use this option when content will be mostly the same, but you only need a few adjustments. It will copy the current row and label the new row as Variation B. When the email is sent, it will send either the original (Variation A Default) or the new variate row (Variation B).
Add Variation (New)
Use this option if you want to start a new row from scratch. It will add a brand new row with no content or formatting. The new row (Variation B) will be inserted where the original, Variation A row is when the full Version 2 email is sent.
4. With your rows in place for your content test, you'll now need to enter the content for the two versions of the message by either editing the duplicate row or building the new variation row.
5. Whether you're testing subject lines, from names, or content, go to the Settings tab and click Test Strategy. This is where you'll indicate how you want the test to run. Make selections for the following:
What percent of the audience do you want to receive "test" messages. Keep in mind that the email will look like any other email to them.
The percentage you choose will be further divided by how many versions of your email are being tested. For example, if you choose 30% as your test audience and you are testing 3 versions of the subject, 10% will get version 1, another 10% will get version 2, and yet another 10% will get version 3. After the winner is determined the remaining 70% will receive the winning version.
Determine Winner By
Select the metric that will determine the winning version. Options include open rate and click rate.
Choose open rate for testing things like the subject, but not the content. Changes in the content won't affect open rate since the recipient won't be able to see the difference before clicking.
You'll also need to indicate a time frame to determine the winner. After all, recipients need time to check their email.
6. Continue with the preview, test, and sending steps you would for a regular email.
7. Once your email has sent according to the test strategy and sending settings (immediate or scheduled), make sure to see which version won, and how the versions performed overall.
You can check the performance by going to the Analytics section of Campaigns. Learn more about analytics.