Setting up an Experiment

After you create a new experiment, the Experiment Setup page appears, where you can define what you're experimenting with, the portion of your user list to include in an experiment, and how to pick a winner.

The options in this article are for an email experiment. The available options vary, depending on the campaign's channel type.

# In this article

# Experiment setup options

On the Experiment Setup page:

# Experiment name

Provide a name that makes this experiment easy to search for and identify among others.

# Experiment with

Select the option you chose when you planned your experiment (see Planning an Experiment). This option is used for all of the variations you create for a given experiment.

# Experiment type (send time)

For Send Time experiments, specify when to send the control campaign for the experiment. Variations are sent to your entire user list, with all variations randomly distributed equally. You can't choose to send variations a minimum number of times or to a subset of users.

If you choose Send in user's time zone, and the user doesn't have a time zone specified, the experiment is sent based on the campaign's time zone.

# Experiment type (not send time)

For all other experiment types, use Randomly split variations to specify what portion of your user list to include in an experiment. The control campaign and variations are sent in roughly equal proportions whether you set the option to yes or no.

• To send to an entire user list, set Randomly split variations to Yes.

  With this setting, blast campaign experiments end when all campaign messages are sent. Triggered campaign experiments run until you select a winner. With this setting, you must select a winner, Iterable doesn't pick one for you.

• To send to a subset of users before sending to the whole list, set Randomly split variations to No.

  For triggered and journey campaigns:

  

  • Sends per variation defines the minimum number of times Iterable sends each variation (round robin) before selecting a winner. When a winner is picked, it's used for 90% of the campaign's future sends. Other variations are used for the remaining 10%, in equal distribution. Iterable monitors variations on an ongoing basis, changing the distribution of sends until the experiment completes.

  For blast campaigns:

  

  • Percentage of list to include in experiment defines the percentage of a send list that Iterable should send this experiment to.

  • Select winning experiment variation after specifies how long Iterable sends campaign messages before selecting a winning variation. After this period of time, Iterable selects a winner for the campaign's remaining messages. Be sure the time you specify gives campaign recipients enough time to interact with it.

# Holdout Groups

Set Holdout Groups to Yes to set aside a portion of a send list to whom Iterable doesn't send a blast campaign, but for whom conversions are still tracked. This allows you to compare the conversion performance of users who don't receive a campaign to the conversion performance of those who do.

• Percentage of the list specifies what percentage of users won't receive the campaign, including variations. Instead, these users each receive a Send Skip event with:

  • eventName set to emailSendSkip, pushSendSkip, smsSendSkip, inAppSendSkip, or webPushSendSkip
  • campaignId set to your campaign's ID
  • reason set to ExperimentHoldOut

• Attribute all custom conversion events specifies how many hours Iterable should attribute custom conversions by users in the holdout group to this experiment. After this period, conversions for members of the holdout group will no longer count towards the group's overall performance.

  A holdout group's attribution window can differ from a campaign's attribution window.

# Select winner based on

Under Select winner based on, choose the metric to use to measure the best performing experiment.

Iterable selects the winner by dividing the following metrics by the total number of messages sent:

• Opens - Unique Opens or Clicks / Total Sends
• Clicks - Unique Clicks / Total Sends
• Custom conversions - Unique Conversions (for the chosen custom event) / Total Sends. To optimize on purchase, select the iterable_purchase event.

See Metrics Definitions for information about these metrics.

Refer to Experiment Summary beneath your selection to see how winning variations and experiment variations will be distributed.

# Next steps

When you're done, click Create (in the upper-right corner). You'll be taken to the Create variations page so you can finish defining your experiment.

See Creating Campaign Variations and Launching Experiments for next steps.