A Methodology for Crafting Awesome Experiences – Part 7

Part 1 – Introduction
Part 2 – Strategic Measurement Framework
Part 3 – Design
Part 4 – Validation
Part 5 – Implement
Part 6 – Measure

If you have not been following the series – or for a refresher – please read the previous entries.

I was always bothered by the shallow measurements in two critical areas of experience management: at the moment-of-truth (when customer and organization interact) and for the overall experience. Using this methodology we are measuring moment-of-truth by integrating feedback events into them as part of the experience design.  That feedback is collected when it happens, focused on effectiveness, and uses escalation if necessary.

The measurement for the overall  experience is something usually addressed poorly.  Most organizations simply deliver a satisfaction survey and ask a question about overall satisfaction with the entire experience.  However, it is very hard for a customer to place a value in overall satisfaction in an experience where they may have been good and bad elements (and we won’t get into the discussion of using customer satisfaction as a metric).  That question would also not measure the effectiveness of the experience.

I want to introduce you to the end-to-end efficiency and effectiveness index (EEX), which accomplishes two things:

1. It provides a method to measure and monitor the overall experience.
2. It consolidates all the moment-of-truth feedback measurements into an index that can be adapted to specific goals of measurement.

There are two stages to compose this index.  First, each moment-of-truth is measured:

The three vertical ovals in the chart above represent the three elements present in each interaction between customers and the organization.  Each moment-of-truth must have a feedback event and the smaller bubbles in the chart represent what is potentially a feedback event (which one is chosen depends greatly on the process, what we are trying to measure, and the purpose of the moment-of-truth)m represented with a numeric value (or a quality-value transformed to numeric).  If there is more than one feedback event in the moment-of-truth, those values are to be aggregated into a common index (the big bubble at the bottom of the chart) — or you have to choose (and document the choice) which one you will use to incorporate into the EEX.

Second, we measure all these moment-of-truth in the experience:

Of course, the individual events will still be reported and followed separately, and will likely have workflows associated with individual values and thresholds.  The index does not override the business’ need to focus on each element of the process to understand weak spots and potential improvement.

Finally, we insert all the values we collected from all those feedback events into a formula to calculate the EEX:

There are two variables in this formula.  First, the feedback (F) refers to the numeric value of each feedback event created and deployed in the experience we designed.  Each will we assigned a weight which specifies how important that specific part of the experience is to the overall value of it.  For example, if your strategy right now is to focus on the value of online interactions (web, email, chat, SMS, Social Media) then you can assign a higher weight to functions that happen online.  The score will then be higher (if you are doing a good job) or lower (if you are doing a poor job).

By using weights you can quickly compare any specific portion of the experience versus the overall benchmark (a benchmark is created by assigning each interaction measured equal weights).  Let’s use  an example.

• You are measuring the experience for change of address.
• There are six steps in your web site to change an address, and four of them are moment-of-truth and have a feedback event associated with them.
• If you insert those four values (F1, F2, F3, F4) to the above formula and assign an equal weight (W1, W2, W3, W4) of .25 to each, you can get the benchmark value for the experience (the average effectiveness per customers’ comments).
• If you want to highlight one of the steps over the others (let’s say that you have been having problems with the confirmation of the old address before allowing the customer to enter the new one), you can then increase the weight for that feedback event to .40, and decrease the value of each of the other three events to .20.
• The resulting value can then be compared to the benchmark to determine the effectiveness in the overall experience
  • If the value from altering the weights is higher – then that portion is performing above the overall function.
  • If it is lower, it is performing below the overall function.

Finally, how to use this index — I know you will be tempted to use this index as a single metric to represent the effectiveness or efficiency of the experience.

Don’t do it.

I wrote before about how using single metrics for measurement is wrong, and it does not represent the true value of what it intends to measure.  Instead, use it as an internal metric to let you know when an experience is performing below par, or when a specific channel or portion of the same is lagging.

You can use it in dashboards, scorecards, or reports as you wish – but make sure it is part of a larger reporting structure.  At the very least, when you use the index (as you are supposed to do with any index) you have to make sure you include all the values that led to that specific index – in this case, the different feedback values you gathered in the different moment-of-truth.

We reached the end of this methodology, and I am anxious to hear your comments.  What do you think?  Is this something you could use? Can you see the value for using it to craft your awesome methodologies?  Am I missing something? Is there something I should change / add / delete?  Please leave me your comments here or send me an email with your questions, comments, or concerns.

Thanks for reading.