A Methodology for Crafting Awesome Experiences – Part 6

Part 1 – Introduction

Part 2 – Strategic Measurement Framework

Part 3 – Design

Part 4 – Validation

Part 5 – Implement

If you have not read the previous five parts, please do so to understand the context.

And so it is we reach the last of the 4 phases of crafting awesome experiences: measurement.

Whenever I talk about this methodology, this is the part where the debate begins.  One of the most cherished best practices of Customer Experience Management and Feedback Management is consistent measurement.  Not simply at the end of an experience or function, but throughout.  Especially for feedback management (an integral part of CEM) it is crucial to collect feedback at different points in the experience to understand which part may need improvements versus simply measuring the entire experience when it is over.  We even discussed measurement and inserting feedback events in the middle so the experiences as we designed in Part 3 of this series.

So why, then, am I talking about measurement at the end?

I am totally in favor or measuring feedback and other metrics at different moments in the experience.  As a matter of fact, when developing the Customer Experience Map (in the design phase), you will see you must find and map the moments of truth – those specific instances in each experience where a customer is likely to make a determination as to their satisfaction and loyalty.  Loyalty is earned or lost at those moments

What is the difference between that constant, consistent, on-going measurement and what this post covers?

The measurement we do at the end is to make sure that the experience we created hits its goals and objectives.  The way to measure once the experience is completed and deployed is by comparing metrics to older metrics – how things work now versus how they use to work.  There are three types of measurements you must do at the end:

1. Effectiveness.  This one is the one that you cannot miss.  Did the change reach its goal?  If you were changing an experience because it took too long to complete before – what is the time to complete now? what was you goal when you started? is the new process better than the old one – even if the goal was not attained?  The bottom line is that you must be able to reach a goal you set before to consider the new experience better.  Alas, you can still improve an experience even if you don’t reach the goals.  Even if your goal is not achieved at first, it may necessitate to do two or three iterations to reach a specific number, improvements must be documented as successful results.
2. Regression. Did you break something else in the process?  Is another process taking longer because of what you did?  Did you generate a new process that is not well documented or measured?  The bottom line on regression testing it to make sure that the rest of the experiences work the same – or better – than before.  While we usually focus on what did not work when doing regression measuring, sometimes you may find something else that works better -unintentionally – than before.  That should also be documented as any improvement can be used to justify the existence of the experience management program.
3. Incremental. Is there anything new that must be changed or measured because of the results of this new experience?  This is a very interesting question to ask, and the reason why experience management is an iterative process. As you make changes to one process, another one may now need to be evaluated.  There are some dependencies that are not easily addressed at first, or easily found.  The purpose of incremental measurement is to understand and prioritize the next phase of experience crating.

A couple more things on the end-result measurement.  First, you are measuring at a higher level than a specific function or even the experience.  You are not measuring any of the three layers in the Feedback Model (effectiveness, satisfaction, or loyalty) but you are measuring overall effect on the existing and new experiences, whether the goal and objective you set early were achieved, and whether or not the design of the experience can be called a success.

You are still going to have to measure effectiveness and satisfaction directly from the customers as they use this new or improved experience – and that is going to happen separately and with different expectations. I just want to make sure you understand that this has nothing to do with the experience itself, rather with the design process and whether it reached its goals.

Second, you must remember that crafting awesome experiences is an iterative process, it takes time and several repetitions to make an experience perfect (OK, better and way more improved – there are not perfect experiences).  You will have to measure the changes as you go along, and make sure improvements happen after each iteration – regardless of how customers feel.  Your goals for improving or creating new experiences are different from what customers expect to get.

They want better, easier, simpler, faster.  You want better integration, more automation, easier processing, and less time to complete.  It may sound the same,  but the main difference is that what you measure is back-office processes that clients cannot see and what they measure is a perception and a impression on how it works.

After you prove that the experience improved, achieved its goals (or least moved in the right direction), and works without breaking anything else or creating new problems then you are ready for deployment.

And to start measuring the perception and impression from customers in the shape of effectiveness and satisfaction.

And for that, as I said in the first two parts, you will need a measurement framework (part 2 of this series), and an understanding of what you are measuring.

Which is the next and last part of this series.

What do you think so far?  Interesting? Any Comments? Thoughts?