Summary Metrics: Incremental Revenue

As a summary metric, Incremental Revenue represents the total additional revenue generated by an Insider campaigns during an A/B testing. The calculation of this metric is based on the comparison between the performance of the control groups and that of the variants of relevant campaigns.

Incremental revenue is one of the important indicators to determine if a variant is a winner, and quantify the contribution of Insider. It measures the impact of a campaign on revenue, and highlights the extra income generated by one group compared to the control group.

For example, imagine you and another customer visit an electronics store. A sales associate accompanies you, and you end up purchasing a smart phone and a smart watch while you were planning to purchase only a smart phone. The sales associate does not help the other customer, and they purchase only the smart phone and leave the store. The extra smart watch you purchase is an example of incremental revenue.

Summary (overall) Incremental Revenue does not directly show the total of variant (campaign-level) incremental revenues. To provide a precise generated extra revenue value, summary incremental takes unique revenue into consideration.

We examine unique order IDs to calculate the unique revenue. For example, if a user views three variants/personalizations but makes a single purchase, the purchase is attributed to each variant separately. However, in summary metrics, the sale is counted as a single sale rather than three.

Based on this logic, the formula for Summary Incremental Revenue is as follows:

Formula: Unique Revenue x ΣIncremental (revenue) / ΣVar (revenue)

Below is a sample calculation of Incremental Revenue.

Unique Revenue

 

Sum of Incremental Revenue

 

Sum of Variant's Revenue

 In the examples above, you can see the unique revenue, sum of incremental revenue, and sum of variant revenue. When you place them in the formula, it gives out the Incremental Revenue displayed in the table below.

Incremental Revenue = Unique Revenue x ΣIncremental (revenue) / ΣVar (revenue)
Incremental Revenue = 1.720.512,49 x 843.945, 99 / 1.760.313, 20
Incremental Revenue = 824.864,35

The following video explains how to calculate incremental revenue, and offers some example cases.

Important Things to Keep in Mind

• There is no such thing as “negative” incremental revenue. Incremental revenue is designed to showcase increases in values due to the nature and definition of the word “incremental”, which means to show a value if it is increasing. Negative values are not displayed on the panel, instead the incrementel revenue is displayed as 0.
• Incremental revenue of a month be higher than that of 3 months. Summary Incremental Revenue is an aggregated total. If you have positive values for 2 months and receive a negative value for 1 month, the overall value for 3 months can be lower than the incremental value of a 1 month-period. Let's say you have 300.000 incremental values in April, 450.000 in May, and 0 in June. As you receive a negative incremental metric in June, it is displayed as 0. If you decide to take three months, you will see a value less than 750.000 as the June has negative incremental value
• Summary Incremental Revenue is unique while Variant Incremental Revenue is not.
• Daily vs. Monthly Incremental Revenues have a difference. Summation of daily values may not be equal to the total for the month. As an overall calculation, we take an aggregated total. Negative incremental revenues contribute to this aggregated monthly total and affect the calculation.