Info |
---|
Calculate The document describes how to calculate CRM Incrementality on topline sales, isolate this impact from Loyalty, understand specific Campaign & Journeys performances & drivers |
Table of Contents | ||
---|---|---|
|
Which analysis can we run?
...
CRM Control Group Pre-Post: this proves the overall value of CRM (how do we impact same guests’ sales if we send communication vs if we don’t)
CRM Behavior Trend: for the test group, affected by CRM communication, we should be looking at their average frequency and check on a monthly basis, to ensure that we are maintaining & sustaining the growth
Specific Ad Hoc Campaign & Journeys Incrementality: on a single campaign and journey level, we will use the delta between test group conversion & control group conversion to understand the incrementality. This is helpful to assess the performances of a specific campaign, or incentive. Conversion is defined as loyalty online and/or in-store transactions in the next 5 days
Overall Journeys & Ad Hoc incrementality: we will take the weighted average of the conversion delta between control & test group for all the ad hoc campaigns, as well as for the full journeys.
...
Through this methodology, we want to measure the impact of Loyalty & CRM over total sales. On top of that, we want to isolate the impact of CRM from the one of loyalty. This proves the overall value of CRM (how do we impact same guests’ sales if we send communication vs if we don’t)
...
Methodology
Create 2 control groups:
Control group for opted in people, to measure CRM incrementality over total sales
Control group at campaign level, to assess efficacy of individual campaign & incentives and optimize the CRM strategy
...
How to set up the groups
1, Open your Braze account and click on Segments
...
Test: 10k
Control: 1K
Unique per week
Week 1 | Week 2 | Week 3 | Week 4 | |
---|---|---|---|---|
Test | 120 | 121 | 130 | 133 |
Control | 12 | 11 | 11 | 12 |
Events Total
Week 1 | Week 2 | Week 3 | Week 4 | |
---|---|---|---|---|
Test | 140 | 141 | 160 | 173 |
Control | 12 | 12 | 11 | 14 |
Monetary Value
Week 1 | Week 2 | Week 3 | Week 4 | |
---|---|---|---|---|
Test | 800 | 810 | 780 | 790 |
Control | 80 | 81 | 79 | 80 |
Frequency = events total by week by cohort / fixed cohort size
...
$ value = (% incremental conversion)*($ average check test group - $ average check control group)
...
*(average check)
This information is available here: add nena's dashboard once ready
4. Overall Ad Hoc Campaign & Journeys Incrementality:
It applies the same methodology listed above (so delta between test & control for both conversion rate and check) but instead of doing it on a campaign level, we will implement it on:
Weighted average for all ad hoc campaigns incrementality
Weighted average for all journeys incrementality
Example:
Journeys | Test Group | Control Group | Conversion Test | Conversion Control | Incremental Conversion |
---|---|---|---|---|---|
Welcome Flow | 10,000 | 100 | 300 (30%) | 20 (20%) | +10% |
Activity Flow | 5,000 | 50 | 100 (20%) | 5 (10%) | +10% |
Journeys | Test Group | Control Group | Check Test | Check Control | Incremental Check |
---|---|---|---|---|---|
Welcome Flow | 10,000 | 100 | 8 | 8.5 | +6.2% |
Activity Flow | 5,000 | 50 | 9 | 9 | +0% |
The impact is the weighted average (weighted for the people who received the campaign) of conversion, and check, for all the campaigns & journeys.
Deployment
The methodology has been deployed in:
EMEA:
BK UK
BK DE
BK NL
BK CH
BK AT
BK France (partially)
APAC:
BK Korea
BK NZ
LAC
BK Mexico