4 Ways To Measure Offline Aha Moment

Products with physical services have two kinds of aha moments: online & offline. A Product Manager can't afford to optimize for only one.

‍

Take, for example, when a customer books an Uber:

1. User enters where they want to go
2. Gets matched to a driver
3. Driver is on the way. => Online aha
4. Driver picks them up
5. On the move
6. Dropped at the destination. => Offline aha

The challenge comes in measuring both in a meaningful way.

The online aha is the easy part. It all happens on the app /web & you can inspect the entire (digital) user journey.

However, apart from a few aspects, you don't have visibility on the quality of interactions during the entire physical journey leading up to the offline aha.

Ex: How did the driver greet the customer? Did they help them with their luggage? Did they follow their training procedures? Did they talk too much? Were they on the phone? Did they play loud music?

‍

"Aatir, the customer will complain if the driver does this."

First, not everyone has the desire, time or energy to complain.

If you hear a complaint about your service, rest assured, there are a few others who experienced the same but chose to remain mum.

Secondly, while a five-star rating control is indicative, it hardly reveals any meaningful data on what exactly to improve.

This becomes even more important when you realize that the offline aha supersedes the online one.

Someone may have a breeze in booking a ride or a hotel but an obnoxious experience when entering the car or checking in, taking the entire experience to the gutter.

No one gives a rating just based on how easy it was to place the order.

In owned marketplaces (where each provider essentially represents the service ex: Careem, Uber etc.), the quality of the offline aha serves as a signal to decide which providers to promote (or eject).

In neutral marketplaces (where the service is merely connecting consumers with independent providers ex: Foodpanda, Booking.com), the quality of the offline aha needs to be democratized to keep consumers informed.

What other options are there to measure offline ahas then apart from star ratings?

1. Dogfooding

Wherever possible, getting employees to regularly make use of the service themselves (with some training on what to look for) & report back issues.

2. Incentivized reviews

Asking for detailed reviews for some value e.g. Booking.com's Genius benefits.

3. Recordings (for video-based services)

Ex: In tele-health & consultancy services, recording those 1:1s for random sampled quality analysis later on helps. [provided permission is granted & privacy policy allows]

4. Derivative Analytics

Deducing quality parameters from app analytics e.g. efficiency of the route that an Uber driver took, sentiment analysis of chat exchange etc.

‍

What other metrics do you track for offline ahas?