Versions Compared

Version Old Version 1 New Version Current
Changes made by

Caes, Guilherme

Caes, Guilherme

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Note

For the full context of why these changes are necessary, please check the [Opportunity] Reduce delivery address errors - core document.

🛠️ Potential Solutions

Solution

Why not?

Add a modal informing the customer if the city changes after an address update

The customer already has selected an address that includes a city on the previous page.

Bias google predictions to locations close to the original address selection

The same thing might happen if two cities are close by (e.g. Madrid and Alcorcón)

Force the customer to choose a valid street name in the previous page (restricting API predictions to addresses) and don’t give the option to change streets without going back to the previous page.

image-20240814-203245.png

In the UK the preferred method of search is by typing the zip code on the search bar and filling the city on the form. Therefore, this change would have a big impact on their current experience.

Adding an extra step to the form if the customer doesn’t select an address with street name. The user experience would be divided into two cases:

Selecting an address that includes a street

  1. Customer selects their street from the dropdown and is redirected to the form

  2. Customer includes the street number

  3. Customer clicks on “Deliver Here” button

Selecting a zip code or city

  1. Customer selects city/zip from dropdown

  2. Customer inputs a street name from dropdown and click next (or swipe right)

  3. Customer includes the street number

  4. Customer clicks on “Deliver Here” button

Adds more friction to the user experience.

Updating coordinates when street name changes.

In this case, the main problem of the address and coordinates diverging would remain, after all, the prediction API results would not be shown to the customer. Signaling prediction failure with a modal is possible, but ideally, we shouldn’t let the customer proceed with an unrecognized street.

✅ Proposed Solution

The proposed is an amalgamation of the concepts listed in the above section since each on its own wouldn’t fix the current errors that occur. To achieve the best results, we propose the following changes.

Info

All possible use cases are listed in Figma and also on the https://rbictg.atlassian.net/wiki/spaces/IBC/pages/edit-v2/4868702220#%F0%9F%A7%91%E2%80%8D%F0%9F%9A%92-QA-Plan section. The changes can be broken into a series of minor improvements listed below.

Delivery selection workflow

In the current implementation, although the customer can change the street name, the update is not reflected in the address coordinates, which is the only information used to determine which restaurant will take the order. This is misleading since the only way the user can notice the update didn’t take effect is by checking that the zip code or map coordinates haven’t changed.

...

  • The coordinates are updated (reflected on the map)

  • The address is updated with Google’s reverse geolocation service (reflected on the pinned details)

  • The street number is cleared (forces the user to put it again to make sure it is the right one)

Edge case 1: typing street number changes city

One of the canceled deliveries due to problems with address selection that happens today is when a customer changes the street number and the app changes the city, for example, try searching for “Calle de nuestra senora de Guadalupe, Madrid” and then changing the street number to “3”. Spoiler: it changes the city to Leganes, even though 3 is a valid number for this street in Madrid .

...

For example, if on the previous page, the customer searched for an address in Madrid, new predictions in the same city will be considered more likely.

Edge case 2: street number doesn’t exist

Another edge case is a valid street number which is not yet recognized by the API. In this scenario, we can allow the user to proceed, but ask display a warning the number wasn’t recognized and ask for confirmation.

...

Conclusion

To make sure these improvements will indeed reduce the amount of delivery errors, we believe this is a perfect scenario to be deployed using an A/B testing strategy to compare data between users this feature activated and not, and measure the actual impact of the changes.

(warning)Potential Challenges

N/A

💰 Cost

To estimate the additional cost of including a dropdown for the street name we need to measure the amount of requests to Google’s Autocomplete service.

...

Now, if we have an average of 12 requests per session, and in the busiest month of the last year we had 82,845 sessions, we reach an estimation of 994,140 requests, which is outside the range of Google’s calculator.

Configuration

The only configuration required will be the creation of a new feature toggle in Launch Darkly to display the improvements (enable-reduce-delivery-errors). This feature toggle will be used to split users into control group and variation group so we can measure the success of the proposal.

📈 Metrics

Main KPI

To effectively measure success, will have to track the percentage of orders that resulted in delivery errors. Currently, delivery updates generate the backend event “delivery_status_change” as can be seen below:

...

However, in case of order errors, there are many possible reasons such as wrong address, payment problems, etc. So we’ll need to add another attribute specifying the cancellation reason and the main KPI for this feature will be the percentage of cancellations with “Wrong Address” and if we see an improvement compared with the control group.

Impact on user experience

For measuring the impact on user experience, we can evaluate what happens between the “Page View“ event when the customer accesses the store locator page in delivery mode and the “Button Click” event when the user clicks on the “Deliver Here” button. By analyzing these two we can measure:

  • Average time it takes to get from one event to the other

    • Tell us if users are having a worse experience than before.

  • Average number of “Manual Location Search“ events triggered in between.

    • Tell us if users are having to select multiple street names or if they are getting it right on the first try.

🗓️ Delivery Plan

TBD

🧑‍🚒 QA Plan

Scenarios with successful Google predictions APIs

Use Case

Steps

Result

Flow in Figma

SCENARIO 1: Search by street number and name

  1. Search by street name and number on “Delivery Address” page

  2. Click on the “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240909-161443.png

SCENARIO 2: Search by street name

  1. Search by street name in “Delivery Address” page

  2. Street name is filled automatically.

  3. Fill street number with a valid value

  4. Click on the “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240909-164745.png

SCENARIO 3: Search by post code

  1. Street name and number are blank.

  2. Types and select a street name from dropdown

  3. inserts the number and check address is updated

  4. Click on the “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240909-165322.png

SCENARIO 4: Search by city

  1. Street name and number are blank.

  2. Types and select a street name from dropdown

  3. inserts the number and check address is updated

  4. Click on the “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240909-170744.png

SCENARIO 5: Search by street name and number, then change street name.

  1. Search by street name and number

  2. Street name and number are filled

  3. Change street name and select from dropdown

  4. Street number is cleared

  5. User cannot proceed until street number is filled

  6. Fill street number

  7. Click on the “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240909-171117.png

SCENARIO 6: Search by street name, change it and fill the street number.

  1. Search by street name

  2. Street name is filled

  3. Change street name and select from dropdown

  4. Fill street number

  5. Click on the “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240909-171338.png

SCENARIO 7: Search by street name, select a street number and change street

  1. Search by street name

  2. Street name is filled

  3. Fill street number

  4. Change street name and select from dropdown

  5. Street number is cleared

  6. User cannot proceed until street number is filled

  7. Fill street number

  8. Click on the “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240909-174159.png

Scenarios with invalid street name

Use Case

Steps

Result

Flow in Figma

SCENARIO 1: Search by street name and change to a name that doesn’t exist

  1. Search by street name and number

  2. Fill street name with a street that doesn’t exist

No results option appear on the dropdown

image-20240909-174634.png

SCENARIO 2: Search by street name, fill street number and type street name that doesn’t exist

  1. Search by street name

  2. Fill street number

  3. Change street name to a value that doesn’t exist

No results option appear on the dropdown

image-20240909-175325.png

SCENARIO 3: Search by postal code, and then type street name that doesn’t exist

  1. Search by postal code

  2. Fill street name with invalid value

No results option appear on the dropdown

image-20240909-175355.png

SCENARIO 4: Search by city and type a street name that doesn’t exist

  1. Search by city name

  2. Fill street name with invalid value

No results option appear on the dropdown

image-20240909-181733.png

Scenarios with invalid street number

Use Case

Steps

Result

Flow in Figma

SCENARIO 1: Search by street name and number, and change number to invalid value

  1. Search by street name and number

  2. Street number and number are filled

  3. Type an invalid street number

  4. Click on the “Deliver Here” button

  5. Confirmation modal appears

  6. Click on “Yes, deliver here” button

Restaurant is selected, and “DeliveryRestaurant” query is sent with payload that:

  • Has coordinates of the last valid number

  • Has street number typed by the user

image-20240909-182149.png

SCENARIO 2: Search by street name, and change number to invalid value

  1. Search by street name

  2. Street name is filled

  3. Type an invalid street number

  4. Click on the “Deliver Here” button

  5. Confirmation modal appears

  6. Click on “Yes, deliver here” button

Restaurant is selected, and “DeliveryRestaurant” query is sent with payload that:

  • Has coordinates of the street

  • Has street number typed by the user

image-20240909-185127.png

SCENARIO?: Search by street name and number, and change number to invalid value + customer is far away

Scenarios where the postal code changes

Use Case

Steps

Result

Flow in Figma

SCENARIO 1: Search by postal code and change city afterwards

  1. Search by postal code

  2. Postal code appears in map pin

  3. Change street name

  4. Postal code no longer appears in map pin

  5. Fill street number

  6. New postal code appears on map pin

  7. Click on “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240910-112333.png

Scenarios where the city changes on address confirmation page

Use Case

Steps

Result

Flow in Figma

SCENARIO 1: Search by street name and number and city changes after changing street name

  1. Search by street name and number

  2. Street name and number are filled

  3. Change street name to a street in another city

  4. Street number is blank and a “city has been changed” warning appears in yellow on top of the map

  5. Fill new street number

  6. Click on “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240910-113308.png

SCENARIO 2: Search by street name and city changes after changing street name

  1. Search by street name

  2. Street name is filled

  3. Type a valid street number

  4. Change street name to a street in another city

  5. Street number is blank and a “city has been changed” warning appears in yellow on top of the map

  6. Fill new street number

  7. Click on “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240910-113827.png

SCENARIO 3: Search by post code and city changes after changing street name

  1. Search by post code

  2. Change street name to a street in another city

  3. Street number is blank and a “city has been changed” warning appears in yellow on top of the map

  4. Fill new street number

  5. Click on “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240910-114047.png

SCENARIO 4: Search by city name and city changes after changing street name

  1. Search by city name

  2. Change street name to a street in another city

  3. Street number is blank and a “city has been changed” warning appears in yellow on top of the map

  4. Fill new street number

  5. Click on “Deliver Here” button

Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

image-20240910-114253.png

⚠️ Call-outs

Note

Tip: Document here any improvement or discussion regarding the feature