Creating Variants and Launching an Experiment

In this article, you'll learn how to create variants of a campaign that you can test against a control campaign. For some examples of the changes you might consider for the type of variant you're creating, see Campaign Variant Types.

This article focuses on email campaigns, but variants are created the same way for other campaign types, with different fields to experiment with.

# Table of contents

# Creating variants for from name, subject line, or preheader

The steps involved in creating a variant that tests opens and open rates is the same whether you're testing from name and sender, subject line, or preheader, you just specify values for different fields.

1. On the Create variations page, edit the field(s) that match the Experiment with value you chose during setup.

   For example, maybe you want to create a variant of a subject line that includes an emoji.

   

2. If you want to add another variant, click Add new variation.

3. When you're done creating variants, click Save & Review.

4. Be sure to review all variants—you want to avoid changing an experiment after you launch it, as doing so may cause inconsistent results.

5. Click Save & Continue.

6. Launch the campaign variants for the experiment.

   If you're creating an experiment from a new campaign, when you return to the Review & Launch page:

   • For a blast campaign, schedule the campaign so the experiment automatically sends with it.

   • For a triggered or journey campaign, activate the campaign and then manually launch the associated experiment.

   If you're creating an experiment for an active campaign (blast or triggered), the experiment will send the next time the campaign sends.

   NOTES

   • For a journey campaign, you can alternatively let a campaign visitor automatically activate the campaign after you enable the journey.

   • For recurring campaigns, experiments are associated only with the first instance of a campaign, and are not relaunched with each recurring send.

# Creating send time variants

You've already specified a send time for the control campaign. Now, specify send times for each variant you create.

1. On the Create variations page, edit Variation name if you want it to reflect the type of change you're making.

2. Specify Send date and Send time for this variant.

   

   IMPORTANT

   If your campaign's send lists have highly dynamic membership, launch the send time experiment as close to your campaign's first send as possible. This way, the list's membership at the experiment's start will be similar to what they were at the start of the campaign.

3. If you want to add another variant, click Add new variation.

   Schedule any additional variants at least one hour ahead of the one you created before it.

4. When you're done creating variants, click Save & Review.

5. Click Save & Continue.

6. Launch the campaign variants for the experiment.

   If your campaign isn't yet scheduled or activated, do so now. Experiments for scheduled (blast) campaigns will send with the campaign, but experiments for recently activated triggered campaigns must be manually activated.

   Experiments for recurring campaigns are associated only with the first instance of a campaign, and are not relaunched with each recurring send.

# Creating variants of a message body

Experiment with the message body to measure the effectiveness of a campaign (and its variants) on clicks, purchases, or custom events.

For example, maybe you want to experiment with a winback campaign that currently offers churned users a 25% discount on their next purchase. Perhaps you want to see whether they convert more readily when you highlight what they've missed while they've been gone.

1. On the Create variation page, at the bottom of the page, click Save Experiment to enable editing.

   

2. Click Edit Variation at the bottom of the page to open the template editor, and click Edit Design.

3. Make the changes you want to test, and click Save Design & Exit.

   

4. From the Create variation page, click Save & Review.

5. Click Save & Continue.

6. Launch the campaign variants for the experiment.

   If your campaign isn't yet scheduled or activated, do so now. Experiments for scheduled (blast) campaigns will send with the campaign, but experiments for recently activated triggered campaigns must be manually activated.

   Experiments for recurring campaigns are associated only with the first instance of a campaign, and are not relaunched with each recurring send.

# Creating a Send Time Optimization test

To test whether a blast campaign has a better open rate with Send Time Optimization enabled (compared to the campaign's scheduled send time), create an STO experiment.

1. To associate an STO experiment with a new campaign, after you attach a template and update its content, in Conversions and Experiments, click Edit and then Add Experiment.

2. On the Experiment Setup page, choose Send Time Optimization and click Create.

   If Send Time Optimization isn't visible, your Iterable project doesn't yet have enough historical user engagement data to calculate optimized send times.

3. From the Create variation page, click Review, and in the preview window, click Save & Continue.

4. On the Review & Launch page, make sure that Enable Send Time Optimization is checked.

   

   Specify the maximum number of hours (between 6 and 24) after the campaign's configured send time (whether immediate or scheduled) that Send Time Optimization can send messages. STO optimizes send times at a one-hour granularity and sends messages at the top of the hour.

   WARNING

   If you add a rate limit to your campaign, some messages may send outside of your STO window. Rate limits are a beta feature; talk to your Iterable customer success manager to add them to your account.

5. Schedule the campaign to send now or at a specific time.

   NOTE

   Specifying a recurrence pattern for a Send Time Optimization campaign causes each recurrence to use Send Time Optimization. However, STO experiments do not recur.

# Holdout group tests

It's not possible to create a standalone holdout group experiment. Instead, create one of the types of experiments mentioned in this article, and when you're setting it up, add a holdout group. See Setting up an Experiment

You don't have to add a variant to a holdout experiment if you don't need one. If you're only interested in measuring campaign conversions and holdout group conversions, simply skip the variant setup step. When your experiment is launched, it will be sent to the users who are not part of the holdout group.

# Next steps

After an experiment launches, you can monitor the results on the Experiments index page. To learn more, read Managing Experiments for information about the statuses reported. And, see Experiment Winner Selection to learn when and how Iterable selects a winning campaign variant and what happens after a winner is selected.