**Glossary of Statistical Terms:**

**Confidence: **The likelihood that the difference in conversion rates between a given variation and the control or baseline is not due to chance. Your statistical significance level reflects your confidence level.

**Difference Interval:** Difference intervals tell you the range of values where the difference between the original and the variation actually lies. The difference interval is a confidence interval of the conversion rates that you can expect to see when implementing a given variation. Think of it as your "margin of error".

For example if my open rate is 50% and my difference interval is + 2%, then my difference interval is from 48% to 52%.

**Confidence Level:** What percentage of confidence you can say the given outcome will occur within the difference interval--this is set at 90% for experiments in Iterable.

**The A/B Test Report**

The A/B test report uses email events or conversions (purchases or custom events) to calculate the confidence intervals, open, click, purchase or conversion rates, and statistical significance.

An example of an A/B test report is shown here:

- Open Rate: this column shows the percentage of unique email opens as well as the difference interval
- Click Rate: this column shows the percentage of unique clicks opens as well as the difference interval
- Revenue: this is the total revenue generated from the variation
- Purchases: this is the number of purchases made from that variation
- Revenue / M: this is the revenue generated from the variation per thousand emails sent
- Purchase (or Conversion) Rate: this column reports the number of conversion events received and the number of view events received as well as the difference interval
- Improvement: this column reports the percentage Improvement of the Test variation compared to the baseline
- Confidence: the likelihood that the difference in conversion rates between a given variation and the control or baseline is not because of chance

The values used in the report are calculated as noted below.

## Conversion Rate and Conversion Rate Change for Variations

For each variation the following is calculated:

- Conversion rate:

- The percentage change of the conversion rate between the Test variation and the Control variation:

## Comments

0 comments

Please sign in to leave a comment.