...

Which analysis can we run?

• CRM Control Group Pre-Post: this proves the overall value of CRM (how do we impact same guests’ sales if we send communication vs if we don’t)

• CRM Behavior Trend: for the test group, affected by CRM communication, we should be looking at their average frequency and check on a monthly basis, to ensure that we are maintaining & sustaining the growth

• Specific Ad Hoc Campaign & Journeys Incrementality: on a single campaign and journey level, we will use the delta between test group conversion & control group conversion to understand the incrementality. This is helpful to assess the performances of a specific campaign, or incentive. Conversion is defined as loyalty online and/or in-store transactions in the next 5 days

• Overall Journeys & Ad Hoc incrementality: we will take the weighted average of the conversion delta between control & test group for all the ad hoc campaigns, as well as for the full journeys.

1. CRM Control Group Pre-Post:

Through this methodology, we want to measure the impact of Loyalty & CRM over total sales. On top of that, we want to isolate the impact of CRM from the one of loyalty

...

. This proves the overall value of CRM (how do we impact same guests’ sales if we send communication vs if we don’t)

...

Methodology

Create 2 control groups:

• Control group for opted in people, to measure CRM incrementality over total sales

• Control group at campaign level, to assess efficacy of individual campaign & incentives and optimize the CRM strategy

...

How to set up the groups

1, Open your Braze account and click on Segments

...

5. Import the file in Braze by clicking on create a segment and selecting the CSV file

...

6. Include the cohort as a control group for all campaigns, both automated campaigns & journeys. To exclude the control group from the campaigns, in the Target Audience select Consumer Membership and Is Not part of the control group

...

7. Export all the cohorts (opted out, opted in, control) in Amplitude by:

...

• Select User ID

• 

• Create the cohort and when you are done click on Save

...

...

• Give your cohort a title and click on Save

...

7. Braze: Include the cohort as a control group for all campaigns, both automated campaigns & journeys. To exclude the control group from the campaigns, in the Target Audience select Consumer Membership and Is Not part of the control group

...

8. Amplitude: Create a new analysis, selecting segmentation and use as event Online Loyalty Transaction Claimed & In Store Loyalty Transaction Claimed all together (all RS). You can adjust the timeframe where the condition have to be verified, and the period you want to look at.

...

Info
By selecting “daily” the unique users will be counted each day. By selecting weekly, the unique users will be determined on a weekly basis. Therefore the sum of the weekly unique does not correspond to the monthly unique

9. Include as segmentation criteria where cohort = test group and control group

...

10. Extract unique, event totals and monetary value (note: to extract monetary value, select group by subtotal and then click on properties, and select sum of property value)

...

11. Calculate frequency, check and monetary value per user by using as denominator the fixed cohort

12. Calculate the pre-post NoC by comparing the behavior of the test group with the behavior of the control group. Eg:

Fixed cohorts

• Test: 10k

• Control: 1K

Unique per week

	Week 1	Week 2	Week 3	Week 4
Test	120	121	130	133
Control	12	11	11	12

Events Total

	Week 1	Week 2	Week 3	Week 4
Test	140	141	160	173
Control	12	12	11	14

Monetary Value

	Week 1	Week 2	Week 3	Week 4
Test	800	810	780	790
Control	80	81	79	80

Frequency = events total by week by cohort / fixed cohort size

Check = total sales by week by cohort / total events by week by cohort

Monetary value = frequency x check

2. CRM Behavior Trend

We want to monitor the evolution, for both check & frequency, of our test group to ensure that, through activations, we are sustaining growth month over month

Methodology

Check test group frequency, and check, on a monthly - weekly basis

Example: https://analytics.amplitude.com/burgerkinguk/notebook/qk83d5g/breadcrumbs/chart/wxgl7wc/edit/2gs1j4j

...

3. Specific Ad Hoc Campaign & Journeys Incrementality:

We want to understand the impact of specific journeys or initiatives on overall conversion & user engagement. This can help us identify where we should allocate our resources and which are the initiatives that drive sales within the overall CRM strategy

Methodology

Each campaign has a control group. Note: The control group for journeys is 10%, the control group for ad-hoc campaigns is 5% (as they have a broader audience reach). The control group does not receive any communication from that specific ad hoc campaigns or journey.

Therefore, we can compare the average conversion rate (online or in-store loyalty transaction) in the next 5 days for the 2 groups. The delta between test & control is the incremental sales

CRM Campaign or Journey Incrementality:

% incremental conversion = % conversion rate test group - % conversion rate control group

On top of that, we should also monitor the average check of test group vs control group to understand the $ value of the campaign.

Incremental value:

$ value = (% incremental conversion)*($ average check test group - $ average check control group)*(average check)

This information is available here: add nena's dashboard once ready

4. Overall Ad Hoc Campaign & Journeys Incrementality:

It applies the same methodology listed above (so delta between test & control for both conversion rate and check) but instead of doing it on a campaign level, we will implement it on:

• Weighted average for all ad hoc campaigns incrementality

• Weighted average for all journeys incrementality

Example:

Journeys	Test Group	Control Group	Conversion Test	Conversion Control	Incremental Conversion
Welcome Flow	10,000	100	300 (30%)	20 (20%)	+10%
Activity Flow	5,000	50	100 (20%)	5 (10%)	+10%

Journeys	Test Group	Control Group	Check Test	Check Control	Incremental Check
Welcome Flow	10,000	100	8	8.5	+6.2%
Activity Flow	5,000	50	9	9	+0%

The impact is the weighted average (weighted for the people who received the campaign) of conversion, and check, for all the campaigns & journeys.

Deployment

The methodology has been deployed in:

EMEA:

• BK UK

• BK DE

• BK NL

• BK CH

• BK AT

• BK France (partially)

APAC:

• BK Korea

• BK NZ

LAC

• BK Mexico