Context

Currently, there is no easy way to track the percentage of deliveries that failed due to wrong addresses since this information is not tracked in amplitude. There is, however, a delivery_status_change event in amplitude triggered every time an order transitions from one delivery status to another.

Example of delivery_status_change event in Amplitude.

Proposed Solution

A simple change that allows us to start measuring the percentage of orders with errors due to wrong addresses is including a Cancellation Reason attribute to the event. For example:

New attribute example.

Solution Hypothesis

Using the cancellation reason as reduce delivery errors KPI will provide us with meaningful results as long as the following assumptions are true:

Delivery Partner operators correctly identify wrong addresses scenarios, and close the order with issues using the correct status
Deliver Partner System is sending the correct value for wrong addresses, as defined in Partner's API documentation (At least BK ES and BK PT are guaranteed to send it).

Defining Reduce Delivery Erros KPI

The existing name attribute of the event is an interpolation of the old and new status (${OLD_STATUS}-${NEW_STATUS}), therefore we can create a segment to measure the total number of orders with errors and compare the percentage of them caused by wrong delivery addresses. These are the metrics that can be used:

A: delivery_status_change that contains ORDER_ERROR
B: delivery_status_change where ‘Cancellation Reason’ = WRONG_ADDRESS
Formula: TOTALS(B)/TOTALS(A)

Below there is an example of the proposed segment where the KPI is 0.67% for Oct 15.

The reason the KPI is 0 for for the other days is that this is a POC and the code is not deployed.

New attribute implementation

How are update delivery status event triggered?

The sequence diagram below shows what happens between Partners API receiving a cancellation event and Amplitude receiving the “delivery_status_change” event.

Tasks Breakdown

[intl-packages] Fix RBI Cancellation type in intl-deliveries

There is a mismatch between the IRbiOrder type defined in packages and actual type saved in dynamo. This happens due to a wrong casting on the update-delivery-event lambda. Add the missing fields to TDeliveryCancellation.

OBS: The new fields must be optional to avoid breaking types across the plataform

[intl-packages] Add new Attribute to delivery_status_change event in intl-mparticle

Include new attribute in deliveryStatusChangeEventProperties.

POC code:

this.logger.info({ newOrder }, '>>> Before checking cancellation reason');
const cancellationReason = (newOrder as any).delivery?.cancellation?.reason;

// delivery status change
if (newOrder.delivery?.status !== oldOrder.delivery?.status) {
  const deliveryStatusChangeEventProperties: LogCrmEventProps = {
    ...baseProperties,
    eventAttributes: {
      'Cancellation Reason': cancellationReason,
      'Customer ID': resolvedUserId,
      Name: `${oldOrder.delivery?.status}-${newOrder.delivery?.status}`,
      'Transaction RBI Cloud Order ID': rbiOrderId,
    },
    eventName: 'delivery_status_change',
    eventType: CUSTOM_EVENT_TYPE,
  };

Iberia Collaboration

Tech Refinement - Measure Reduce delivery error success

Analytics