Send Time Optimization Experiments

This document describes how to create Send Time Optimization experiments with email blast campaigns. It's not currently possible to use Send Time Optimization experiments with push notifications.

For email blast campaigns, Send Time Optimization (STO) attempts to maximize the Email Open Rate metric (defined as Unique Email Opens or Clicks / Total Emails Delivered). To do this, it uses machine learning to analyze each recipient's historical engagement behavior (in the same Iterable project), and then chooses a send time when they're likely to open the message.

Send Time Optimization experiments use optimized times for half of a blast campaign's recipients and the campaign's configured send time for everyone else. They allow you to measure the improvement Send Time Optimization makes on a campaign's Email Open Rate.

# Table of contents

To create an email blast campaign with an associated STO experiment, follow these steps:

# 1. Create an email blast campaign

To create an email blast campaign, navigate to Messaging > Campaigns and click Create New Campaign. On the New Campaign Screen:

1. Specify a Campaign Name, choose whether to use a Custom Conversion Metric, and add Labels as necessary.

2. For Campaign Type, choose Blast.

3. For Message Medium, choose Email.

4. Select the list to which you'd like to send the campaign.

5. Click Continue to Templates.

6. Select an existing template and click Continue to Design, or click Create a New Template. Configure your template as necessary.

You'll now be on the Edit Template screen.

# 2. Create an STO experiment

To use STO for half of your campaign's recipients and a configured send time for the rest, create an STO experiment. To do this:

1. On the Edit Template screen (where you left off in the previous step), click Create A/B Experiment:

   

2. On step 2 (Setup) of the experiment creation sequence, enter an Experiment Name and select Send Time Optimization in the Experiment With section:

   

   WARNING

   If the Send Time Optimization option isn't visible, it's because your Iterable project does not have enough historical user engagement data to calculate optimized send times.

3. On step 3 (Design) of the experiment creation sequence, click Review (in the upper-right corner) to continue on.

4. On step 4 (Review) of the experiment creation sequence, click Save & Continue (in the upper-right corner) to continue on.

You'll now be on the campaign's Review & Launch screen.

# 3. Send or schedule the campaign

On the Review & Launch screen:

1. Notice that the Send Time Optimization toggle has been enabled.

   

   WARNING

   If the toggle isn't visible, it's because your Iterable project doesn't yet have enough historical engagement data to support the Send Time Optimization machine learning models.

2. Specify the maximum number of hours (between 6 and 24) after the campaign's configured send time (whether immediate or scheduled) that Send Time Optimization can send messages. Send Time Optimization optimizes send times at a one-hour granularity and sends messages at the top of the hour.

3. To send the campaign immediately, click Send Campaign Right Now.

4. To schedule the campaign for later or have it recur, click Schedule Campaign For Later. Specifying a recurrence pattern for an Send Time Optimization campaign causes each recurrence to use Send Time Optimization. However, STO experiments do not recur.

   WARNING

   After a campaign begins to send, you cannot disable Send Time Optimization.

# 3. Monitor experiment results

To monitor the results of an STO experiment:

1. Wait until the experiment is active (after the campaign's specified send time).

2. Navigate to Messaging > Experiments.

3. Select a tab: Completed, Running, Draft/Scheduled, or Canceled/Errored (depending on the experiment's state).

4. Click an experiment's name to view its variations and their associated metrics. The variations will look similar to this:

   

   WARNING

   For STO experiments, do not click the Use Variation button.

   Iterable sends the experiment's Control variation to half of the campaign's recipients at its configured send time, and the STO variation to the other recipients at per-user optimized send times.

5. Use the Showing X Metrics drop-down menu in the upper-right corner of the table to choose which columns the table should display.

6. To choose metrics to analyze with charts and graphs, use the Metrics to Analyze drop-down menu below the table:

   

   Since STO optimizes Email Open Rate, this is a good metric to analyze.

To learn more about experiment metrics, read A/B Experiment Metrics.

# Further reading

• Limitations of Send Time Optimization.
• Creating Blast Campaigns
• Using Send Time Optimization with Workflows