Document Status	DRAFT
Document Owner(s)	@Caes, Guilherme
Reviewers	@Raphael Ferreira Gomes

For the full context of why these changes are necessary, please check the [Opportunity] Reduce delivery address errors - enhancements document.

Potential Solutions

Solution	Why not?

Solution	Why not?
Add a modal informing the customer if the city changes after an address update	The customer already has selected an address that includes a city on the previous page.
Bias google predictions to locations close to the original address selection	The same thing might happen if two cities are close by (e.g. Madrid and Alcorcón)
Force the customer to choose a valid street name in the previous page (restricting API predictions to addresses) and don’t give the option to change streets without going back to the previous page. Form address second page	In the UK the preferred method of search is by typing the zip code on the search bar and filling the city on the form. Therefore, this change would have a big impact on their current experience.
Adding an extra step to the form if the customer doesn’t select an address with street name. The user experience would be divided into two cases: Selecting an address that includes a street Customer selects their street from the dropdown and is redirected to the form Customer includes the street number Customer clicks on “Deliver Here” button Selecting a zip code or city Customer selects city/zip from dropdown Customer inputs a street name from dropdown and click next (or swipe right) Customer includes the street number Customer clicks on “Deliver Here” button	Adds more friction to the user experience.
Updating coordinates when street name changes.	In this case, the main problem of the address and coordinates diverging would remain, after all, the prediction API results would not be shown to the customer. Signaling prediction failure with a modal is possible, but ideally, we shouldn’t let the customer proceed with an unrecognized street.

Proposed Solution

The proposed is an amalgamation of the concepts listed in the above section since each on its own wouldn’t fix the current errors that occur. To achieve the best results, we propose the following changes.

All possible use cases are listed in Figma and also on the [Solution] Reduce delivery address errors - enhancements | 🧑‍🚒 QA Plan section. The changes can be broken into a series of minor improvements listed below.

Delivery selection workflow

In the current implementation, although the customer can change the street name, the update is not reflected in the address coordinates, which is the only information used to determine which restaurant will take the order. This is misleading since the only way the user can notice the update didn’t take effect is by checking that the zip code or map coordinates haven’t changed.

Another possible issue is that the user might type a street name that doesn’t exist and there is no validation to inform them.

Requiring the customer to select a street name from the dropdown (i.e. the predictions given by Google’s Autocomplete API) will make sure that the street coordinates are valid. Also, on the address confirmation page, the predictions are restricted to addresses, unlike the previous page where city or zip codes were valid.

Customer changing street name

Once a value from the dropdown is selected, the following effects take place:

The coordinates are updated (reflected on the map)
The address is updated with Google’s reverse geolocation service (reflected on the pinned details)
The street number is cleared (forces the user to put it again to make sure it is the right one)

Edge case 1: typing street number changes city

One of the canceled deliveries due to problems with address selection that happens today is when a customer changes the street number and the app changes the city, for example, try searching for “Calle de nuestra senora de Guadalupe, Madrid” and then changing the street number to “3”. Spoiler: it changes the city to Leganes, even though 3 is a valid number for this street in Madrid .

We cannot control if Google’s autocomplete API considers one address more likely than the other, but we can influence it by biasing the results to certain coordinates. In this case, we can use bias for the API to give more weight to predictions close to the location selected on the previous page.

For example, if on the previous page, the customer searched for an address in Madrid, new predictions in the same city will be considered more likely.

Edge case 2: street number doesn’t exist

Another edge case is a valid street number which is not yet recognized by the API. In this scenario, we can allow the user to proceed, but ask display a warning the number wasn’t recognized and ask for confirmation.

Conclusion

To make sure these improvements will indeed reduce the amount of delivery errors, we believe this is a perfect scenario to be deployed using an A/B testing strategy to compare data between users this feature activated and not, and measure the actual impact of the changes.

Potential Challenges

N/A

Cost

To estimate the additional cost of including a dropdown for the street name we need to measure the amount of requests to Google’s Autocomplete service.

For this estimation, let’s consider the “Manual Location Search“ event that is triggered when the customer selects an address from the dropdown, or when they start and stop typing. For July this year, which was the peak in the last 6 months, the event was triggered 215,376 times for PLK ES (checking via Amplitude).

However, the same session may trigger more than one “Manual Location Search“ event, so to make sure we have unique sessions, we have to take into consideration that 132,531 of these events had no address selected, which leaves us with 82,845 unique sessions.

To proceed further, we’ll need to make some assumptions. There’s a lot of variance in the length of street names in Spain, from “Calle Mayor” with 11 characters to “Calle de nuestra senora Guadalupe“ with 33. However, the user doesn’t need to type all characters, only enough for the right prediction to appear on screen, so let’s make a guess that on average, 15 characters will be enough. Excluding the starting 3 characters that do not trigger the search, this leaves us with 12 requests per session.

Now, if we have an average of 12 requests per session, and in the busiest month of the last year we had 82,845 sessions, we reach an estimation of 994,140 requests, which is outside the range of Google’s calculator.

Configuration

The only configuration required will be the creation of a new feature toggle in Launch Darkly to display the improvements (enable-reduce-delivery-errors). This feature toggle will be used to split users into control group and variation group so we can measure the success of the proposal.

Metrics

Main KPI

To effectively measure success, will have to track the percentage of orders that resulted in delivery errors. Currently, delivery updates generate the backend event “delivery_status_change” as can be seen below:

However, in case of order errors, there are many possible reasons such as wrong address, payment problems, etc. So we’ll need to add another attribute specifying the cancellation reason and the main KPI for this feature will be the percentage of cancellations with “Wrong Address” and if we see an improvement compared with the control group.

Impact on user experience

For measuring the impact on user experience, we can evaluate what happens between the “Page View“ event when the customer accesses the store locator page in delivery mode and the “Button Click” event when the user clicks on the “Deliver Here” button. By analyzing these two we can measure:

Average time it takes to get from one event to the other
- Tell us if users are having a worse experience than before.
Average number of “Manual Location Search“ events triggered in between.
- Tell us if users are having to select multiple street names or if they are getting it right on the first try.

Delivery Plan

TBD

QA Plan

Scenarios with successful Google predictions APIs

Use Case	Steps	Result	Flow in Figma

Use Case	Steps	Result
SCENARIO 1: Search by street number and name	Search by street name and number on “Delivery Address” page Click on the “Deliver Here” button	Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates
SCENARIO 2: Search by street name	Search by street name in “Delivery Address” page Street name is filled automatically. Fill street number with a valid value Click on the “Deliver Here” button	Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates
SCENARIO 3: Search by post code	Street name and number are blank. Types and select a street name from dropdown inserts the number and check address is updated Click on the “Deliver Here” button	Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates
SCENARIO 4: Search by city	Street name and number are blank. Types and select a street name from dropdown inserts the number and check address is updated Click on the “Deliver Here” button	Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates
SCENARIO 5: Search by street name and number, then change street name.	Search by street name and number Street name and number are filled Change street name and select from dropdown Street number is cleared User cannot proceed until street number is filled Fill street number Click on the “Deliver Here” button	Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates
SCENARIO 6: Search by street name, change it and fill the street number.	Search by street name Street name is filled Change street name and select from dropdown Fill street number Click on the “Deliver Here” button	Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates
SCENARIO 7: Search by street name, select a street number and change street	Search by street name Street name is filled Fill street number Change street name and select from dropdown Street number is cleared User cannot proceed until street number is filled Fill street number Click on the “Deliver Here” button	Restaurant is selected, and “DeliveryRestaurant” query has the right address and coordinates

Scenarios with invalid street name

Use Case	Steps	Result	Flow in Figma

Use Case	Steps	Result
SCENARIO 1: Search by street name and change to a name that doesn’t exist	Search by street name and number Fill street name with a street that doesn’t exist	No results option appear on the dropdown
SCENARIO 2: Search by street name, fill street number and type street name that doesn’t exist	Search by street name Fill street number Change street name to a value that doesn’t exist	No results option appear on the dropdown
SCENARIO 3: Search by postal code, and then type street name that doesn’t exist	Search by postal code Fill street name with invalid value	No results option appear on the dropdown
SCENARIO 4: Search by city and type a street name that doesn’t exist	Search by city name Fill street name with invalid value	No results option appear on the dropdown

Scenarios with invalid street number

Use Case	Steps	Result	Flow in Figma

Use Case

Steps

Result

Flow in Figma

SCENARIO 1: Search by street name and number, and change number to invalid value

Search by street name and number
Street number and number are filled
Type an invalid street number
Click on the “Deliver Here” button
Confirmation modal appears
Click on “Yes, deliver here” button

Restaurant is selected, and “DeliveryRestaurant” query is sent with payload that:

Has coordinates of the last valid number
Has street number typed by the user

SCENARIO 2: Search by street name, and change number to invalid value

Search by street name
Street name is filled
Type an invalid street number
Click on the “Deliver Here” button
Confirmation modal appears
Click on “Yes, deliver here” button

Restaurant is selected, and “DeliveryRestaurant” query is sent with payload that:

Has coordinates of the street
Has street number typed by the user

SCENARIO?: Search by street name and number, and change number to invalid value + customer is far away

Scenarios where the postal code changes

Use Case	Steps	Result	Flow in Figma