This article explains how to work with Iterable's Experiments page to find information about the experiments you create.
In this article
Accessing experiments
To view your experiments, go to Messaging > Experiments.
On the Experiments page, you can:
- Create a new experiment.
- Edit an existing experiment.
- Delete an experiment that's in Draft or Ready state. Deleted experiments are permanently deleted, they aren't archived.
- View the criteria used to select a winning campaign.
- View the status of an experiment.
- Review when the experiment started and ended.
- Find out who created an experiment and when, by hovering the date under Last modified.
- Find out who last modified an experiment and when, using the avatar under Last modified.
To display overview and performance details for a specific experiment, click the experiment’s name to open Experiment Analytics.
Experiment statuses
The status reported on the Experiments page provides important information about the state of a campaign and the variants being sent to users in an experiment.
-
Draft - An experiment has been created, but not launched.
For blast campaigns, if the campaign is scheduled, experiments (except for Send Time) will automatically launch when the campaign is sent. If not, it will be sent with the campaign once you schedule it.
For triggered and journey campaigns (and blast campaigns with Send Time experiments), you’ll need to manually launch the associated experiment. See Launching an Experiment for details about manually launching an experiment.
-
Running - Iterable is still waiting for a winner to be declared. Campaigns have only been sent to the subset being used for testing.
If the experiment was configured to have Iterable select a winner, Iterable is still waiting for the experiment test duration to complete (blast) or for the minimum number of sends per variant to be met (triggered).
If the experiment was configured to manually select a winner, Iterable will continue testing all variants until a winning variant is manually selected or the experiment is ended.
Finished - Either Iterable or a user has ended the experiment and variants are no longer being tested. If a winning variant was found and the associated campaign is still running, all future users will receive that winning variant. If no winning variant was found and the associated campaign is still running, all future users will receive the control variant.
Winner found - The experiment is still running and Iterable is using a multi-armed bandit approach to pick a winning variant. To optimize your experiment, that variant is sent to 90% of your campaign's future sends, with the other variants being sent to the remaining 10%. Iterable continues to monitor variants for the duration of the experiment, selecting a new winner if new user preferences surface. If a new winning variant is found, it becomes the current winner, and 90% of new users will start to receive that variant.
Sorting experiments
To sort your project's experiments by name, status, launch date, or last modified date, click the column headers on the table.
Searching for an experiment
Type an experiment's name or ID in the search bar in the upper-right corner of the page. The experiments list will automatically refresh to show experiments that match your search phrase.
Filtering the experiments view
To filter your experiments by status, choose one of the status filters along the top of the page. Alternatively, to filter by other criteria, click Filters and select relevant criteria. Options include:
Experiments - Modified by, Test subject, Winning criteria, Winner selected, and split variants
Campaigns - Campaign type, Campaign medium, Campaign label
Viewing experiment analytics
Once an experiment has finished, review its analytics for insight about how users responded to the variants you tested. You can view high-level details about an experiment on the Overview tab, and more detailed performance-related information on the Performance tab.
NOTE
You can view analytics for active experiments at any time, but it's best to let them finish before interpreting results. Evaluating an experiment's analytics prematurely can lead you to form incorrect conclusions.
To view the analytics associated with an experiment, navigate to Messaging > Experiments and click the experiment you're interested in.
For details about the type of data you can review, see Experiment Analytics.