What are we trying to measure?

Through this methodology, we want to measure the impact of Loyalty & CRM over total sales. On top of that, we want to isolate the impact of CRM from the one of loyalty

Methodology

Create 2 control groups:

• Control group for opted in people, to measure CRM incrementality over total sales

• Control group at campaign level, to assess efficacy of individual campaign & incentives and optimize the CRM strategy

How to set up the groups

1, Open your Braze account and click on Segments

2. Select from the drop down list Custom Attributes

3. Create a new segment with the following characteristics:

• MarketingEmail = True OR

• MarketingPush = True

By relying on the OR logic, we are selecting all guests who have opted in to at least one channel (either email or push, or both)

4. Export this cohort and randomly select 10% of the users. This 10% will be the fixed control group

5. Import the file in Braze by clicking on create a segment and selecting the CSV file

6. Export all the cohorts (opted out, opted in, control) in Amplitude by:

• Clicking on User Data and then CSV Export Email Addresses

• In the export, only keep the User ID column

  

• Open Amplitude and click on Create New

• Click on User Cohorts

• Select import from cohort file

• Select User ID

• 

• Create the cohort and when you are done click on Save

• Give your cohort a title and click on Save

7. Braze: Include the cohort as a control group for all campaigns, both automated campaigns & journeys. To exclude the control group from the campaigns, in the Target Audience select Consumer Membership and Is Not part of the control group

8. Amplitude: Create a new analysis, selecting segmentation and use as event Online Loyalty Transaction Claimed & In Store Loyalty Transaction Claimed all together (all RS). You can adjust the timeframe where the condition have to be verified, and the period you want to look at.

9. Include as segmentation criteria where cohort = test group and control group

10. Extract unique, event totals and monetary value (note: to extract monetary value, select group by subtotal and then click on properties, and select sum of property value)

11. Calculate frequency, check and monetary value per user by using as denominator the fixed cohort

12. Calculate the pre-post NoC by comparing the behavior of the test group with the behavior of the control group. Eg:

Fixed cohorts

• Test: 10k

• Control: 1K

Unique per week

Events Total

Monetary Value

Frequency = events total by week by cohort / fixed cohort size

Check = total sales by week by cohort / total events by week by cohort

Monetary value = frequency x check