Mystery Shopper Research: Performance Standards (Part 5)

In Part 5 of our Mystery Shopper Research Series, we explore how to select and measure Performance Standards.

In this Part 5 post we will –

Look at Performance Standards – how to select and measure them
Share more examples

If you’re new to this series and want to see the other articles you will find them all here in our Introduction to the Series –

https://www.omnitouchinternational.com/introducing-our-mystery-shopper-research-series/

Let’s review where we are in our 7-Step Design Process for a Mystery Shopper Program

In this post we cover Step 4 in our 7-Step Design Process.

As a reference here is a summary description of each of the 7 Steps:

What do I want to learn: Define and document what you aim to learn from your research program – your Research Objectives
What Scenarios do I want to test: Define specific journey and/or touchpoint scenarios to be tested / Categorize each scenario as either existing Customer, prospective Customer or Observation
Special Profiling & Selection Needs: Consider special profiling needs for the Mystery Shoppers as to what they need to ‘be’ or ‘have’
Select Performance Standards: Select and define the Performance Standards to be scored and exactly how they are to be scored
Plan Volumes & Timing: Develop a fieldwork calendar that lays out the finalized volumes and timing of Mystery Shopper interactions
Design Reporting: Layout report design and flow – decide how will results be presented to stakeholders both when finalized and as they come in (red flags)
Design & Execute Training: Design the paths and scripts for Mystery Shoppers to execute as well as the tasks and equiprment along the way

With Performance Standards, you decide what specific behaviors and conditions you want to measure

You’ve reached an important step in your Mystery Shopper Research program design.

In this step you decide what behaviors and/or conditions you want to specifically measure and report numerically. Everyone is going to want to know ‘how we did’ when the program has ended.

And the objective scores from your Performance Standards – supported by qualitative comments where relevant – will help you tell the story of how you did.

The possibilities for what you want to measure with Mystery Shopper are nearly endless. We’ve measured literally thousands of different behaviours and conditions over the years.

So to help our Clients imagine what can be measured I share this list. Which always begins with the question –

How well do we ______?

Deliver service
Drive sales
Provide an effective digital journey
Adhere to policies & procedures
Know products & services
Convey brand image
Pick up and handle sales signals
Manage queues and response times
Handle questions about competitors
Handover Customers from one person or function to another
Give bad news when we have to
Compare to the competition
Show empathy
Present the right environment
Use branded words & phrases
Upsell / Cross-sell

As you can see – just from this short list – the opportunities to select meaningful Performance Standards are many and varied.

Selecting the right Performance Standards matters

As you get specific about what you want to learn – which is where it all begins in Step 1 of the Design Process – the Performance Standards you want to or should measure in your Mystery Shopper program begin to emerge.

For example, if your Research objective is to evaluate the impact of a sales skills training that you rolled out to the Frontline, it’s likely that you’ll want to evaluate the sales behaviors which were covered in that training program.

Or if your Research objective is to understand the Customer Service performance of handling product enquiry across the 10 markets in which you operate, you’ll want to focus in on the behaviors needed to effectively handle that particular product enquiry.

For example you could measure how well smart questioning was used. Was there clarity in why the product was right (or not) for the Customer. How easy (or not) was it to convey how to enjoy the future benefits of the product.

Perhaps your Research objective is to understand how best in class organizations say ‘no’ to their Customers when they have to. If so, then you’ll likely select behaviors around empathy, positive language and specific techniques related to saying no.

The point that I hope shines through in these real life examples is that the Performance Standards you select need to align to your overall Research objective.

Or put another way, ask yourself – “What’s the business question we’re trying to answer?” And then relentlessly choose the Performance Standards that will help you to answer that question.

Resist the temptation to let everyone in the company jump in and suggest measurements

If you allow different departments and functions to jump in and suggest the measurements they want to have – typically for their own reasons – you’re going to end up with a Frankenstein program.

Remember Frankenstein?

He’s made out of different body parts taken from different bodies. And in Mystery Shopper Research, it’s pretty easy to spot a Frankenstein program

Because there’s no central theme or direction to follow.

Which also means that when the Mystery Shopper sccores come out, it will be different to organize and guide coherent response actions.

Here’s a model I use to help select Performance Standards

When we teach Quality – and Mystery Shopper Performance Standards are very much about Quality – we share that there are three key areas of input to consider.

And that it makes sense to look at each input area one by one to be sure you’ve thought of everything.

Input Area 1 – Who are we? (Our organization)

What is our Vision, Mission and Values? What are our core brand attributes? How do we describe and present ourselves to the world?

Because the story you tell about yourself sets expectations in the mind of your Customer.

We always read our Client’s website when thinking about what Performance Standards to measure. One Client had the word ‘friendly’ on their site in 17 different places.

So we recommended that (as a result) they should measure ‘friendliness’ in their program.

Input Area 2 – What do they want? (Customer expectations)

This simple question links to Customer expectations. Because CX Management, at its simplest, is about meeting and exceeding Customer expectations.

Do your Customers want you to be patient? Then measure patience.

Do your Customers tell you that they want easy ways to reach you? Then measure the ease of access – including response times and effectiveness.

If your company has a strong Customer research capability, you’ll have a good reliable source of input here. If not, then admittedly it will be more difficult.

Input Area 3 – What do we have to do? (Regulatory)

Many industries – such as utility providers, public sector agencies and financial institutions – must follow regulatory laws & requirements.

They must do or not do something or risk having fines assessed or even their business closed down.

When and where relevant, you may decide to include some level of regulatory Performance Standards in your Mystery Shopper Research program.

When Clients get stuck when selecting Performance Standards, I think this model can help get them ‘unstuck’.

Each Performance Standard is scored – here’s how

Let’s start with binary scale ‘Compliance Standards’

Some Performance Standards are best measured on a binary scale – yes or no or 1 point/zero points.

A binary yes/no scale measures whether something was done (or not).

For example, the Greeting was delivered (or not). The Verification was conducted (or not).

Please never try to measure a binary yes/no behavior on a range-based scale. That wastes everybody’s time and doesn’t take any part of the experience forward.

Typically there’s not a lot of skill or nuance required for a Compliance Standard. The standard was either delivered to expectation or not delivered to expectation.

Now let’s look at range-based ‘Calibre Standards’

Other Performance Standards are best measured on a range-based scale. A scale such as 3,2 1 or Excellent, Good, Fair and Poor.

A range-based scale measures how well something was done (or not).

For example, the Tone of Voice was good which means that it was better than fair but not (yet) at excellent.

Range-based or calibre Performance Standards involve a degree of performance.

And each step up or down in that degree is clearly defined. Your requirement to score ‘excellent’ for Empathy should look different than your requirement to score ‘good’ for Empathy.

Sometimes you have to think about whether the standard should be compliance or calibre

For example, for an observation-based standard like Cleanliness, we could decide to have a yes/no measurement for Cleanliness.

Or a range-based evaluation of cleanliness such as Excellent, Good, Fair or Poor.

Think about what you’re trying to achieve and how it fits in with your overall research objectives. You’ll make the right choice.

Here’s a question – which standards do my Customers tend to remember / feel more after they’ve interacted with us?

Compliance? Or calibre? 99% of the time when I ask this question in training rooms I hear the correct answer –

Calibre.

Because calibre standards tend to pick up on the humanity, feeling, degree and/or style of the interaction. It’s where we can shine.

I don’t think Customers get that excited by your Verification or Closing process. Not that those aren’t important. Many are very important.

They just don’t carry the same emotional resonance.

Scores for Performance Standards represent the objective data in your Mystery Shopper Research program

Every Performance Standard in the program is individually measured and has a score attached to it.

Of course, this means that every Performance Standard needs to be clearly defined. And the scoring logic for standard also needs to be clearly defined.

Because scoring Perfomrance Standards should never be based on the individual opinion of the Mystery Shopper.

It’s not ok if Mystery Shopper ‘Bob’ gives a score of 3 out of 3 while Mystery Shopper ‘Tanya’ gives a score 1 out of 3 – for the same demonstrated behavior.

Programs that allow Mystery Shoppers to score whatever they think or feel like is where many Mystery Shopper programs fail.

Because in these programs, the Mystery Shopper scores are treated as if they’re coming from real Customers. But they’re not real Customers.

Mystery Shoppers are stand-ins for Customers who have been given instructions to follow a pre-written scenario across a pre-selected set of journeys and/or touchpoints and make observations along the way.

How to ensure your scores are accurate

The term calibration refers to the concept that whoever is involved in scoring Performance Standards across a Mystery Shopper program scores in the same way.

That everyone has the same understanding of the definition of the standard and how to distinguish between the levels of performance for eachstandard.

Whether it’s a Compliance Standard or a Calibre Standard.

Sometimes, depending on the simplicity or small scale of the program, this is possible to do with all the Mystery Shoppers.

But when you’re dealing with a more complex or bigger scale program, calibrating all the Mystery Shoppers isn’t possible.

To address this reality, we have our own small in-house Team of Quality Assurance people who for some programs do all the scoring – using audio or video recordings or transcripts.

Or – for example when the Mystery Shopper is doing their evaluation in a different language – our Quality Assurance person will interview the Mystery Shopper to ensure accurate scoring – along with supporting qualitative input where relevant.

Thank you for reading!

In this Part 5 post in our Mystery Shopper Research series we went deep into the topic of Performance Standards.

Your properly selected and calibrated Performance Standards ensure that your Organization can trust their Mystery Shopper data as a basis for improvement.

In the next post in this series we will wrap up our 7-Step Design Process including –