A Customer Experience Mystery Shopper program is something very particular and special.
This article is part of our Mystery Shopper Research Series — a full set of insights on designing, running, and learning from Mystery Shopper programs.
A Local Association Introduced a Customer Experience Mystery Shopper
Some time back in Singapore, one of the local industry Associations introduced a Customer Experience-based Mystery Shopper Award into its portfolio.
I thought – Cool. It will be interesting to see what a Customer Experience-based Mystery Shopper Award looks like as per a global vendor.
A short time later, I was helping a hospitality Client set up the Quality Assurance program for their Contact Centre.
A group of 20 senior folks were gathered together in the conference room, and we were in the midst of selecting & defining quality standards when one of them stopped and asked –
“Hey Dan – did you know that we entered the Customer Experience Mystery Shopper Award this past year?”
“No,” I said – “How did it go?”
Here’s What Happened Next in That Conversation
“Well – we aren’t so sure.
Because in this workshop I’m getting a sense of the complexity that goes into setting & measuring quality – but I’m not so sure it was this rigorous in our Awards entry.”
He continued…
“I have the final report from the Mystery Shopper vendor here on my laptop – can we flash it up and talk about it?”
But of course!
The cover slide whirred up on the screen.
Opening slide – very formal – The Customer Experience Mystery Shopper Report.
We were all ready. And then, next slide…
THE GREETING – score 98%
What? The Greeting? Oh – ok. Anyway 98%
Then the next slide…
GET THE CUSTOMER NAME – score 97%
Oh…really?
And it carried on from there.
Slide after slide after slide reported on a compliance measurement – even the Hold Technique was featured.
As we hit slide 20+ something someone in the room turned to me and asked – “So Dan – you look a bit pale – what do you think?”
Well it was an easy question to answer.
“Well guys. What you have here is a wonderfully presented compliance report – but I haven’t seen anything yet that even remotely measures or talks about the Customer Experience.”
And the room agreed.
There Was One Final Measure Called ‘Customer Experience’
Interestingly – in this report there was a final measurement slide that said CUSTOMER EXPERIENCE (yes – that’s right – a single slide).
Score – 58%.
But the legend was unclear as to how the score was derived.
After some discussion around the table we guessed that perhaps this was the personal score or viewpoint of the Mystery Shopper.
So let’s put it on the table right now.
If the first thing that comes to your mind when planning a Customer Experience-based Mystery Shopper program is THE GREETING – then you are on the wrong track.
There’s a Lot of Compliance-Based Mystery Shopper Work Going On Out There
There are certainly valid reasons for having a solid compliance-based Mystery Shopper program.
They’re used extensively in the banking & finance industry – especially for ensuring regulatory compliance.
In the Public Sector, compliance-based programs provide a basic ‘minimum-standard’ dipstick – that provides high level assurance that when an email gets sent – it receives a reply.
Or when a telephone call is placed, some kind of basic response – along with basic courtesy – is provided.
While it’s rare to see a Public Sector program skew heavily to the Customer experience (now there’s an opportunity I’d love to be part of!) compliance-based programs ensure a level of essential service is provided.
Another example of a smart compliance program is ensuring that things work the way they are supposed to work.
That when a certain telephone number is dialed at a certain time of day – that the call goes to the right place (you’d be surprised how many times it doesn’t).
Or when a certain set of IVR options or digital instructions are followed, that the Customer ends up where they were supposed to and got what they were supposed to get.
Sanity Check the Plumbing
As channels overlap, confirm that journeys actually route the way you intend — calls to the right place at the right time, IVR paths that land correctly, and digital instructions that don’t dead-end.
These checks are “compliance” in spirit but essential prerequisites for any CX-based study.
It’s Not Right to Say That Compliance-Based Programs are ‘Bad’ While CX-Based Programs Are ‘Better’
We’re not here to say one kind of program is better than another.
It’s always about defining what you want to learn and then figuring out the best way to learn it.
But there’s a big opportunity in putting together a solid Customer Experience-based Mystery Shopper program.
And no – adding the phrase Customer Experience in front of something doesn’t make it so.
Have You Asked Your Team?
If you are considering a compliance-based Mystery Shopper program ask yourself – what am I going to learn from this program that I couldn’t learn from my own Team Members?
Assuming that you’re not conducting Mystery Shopper because you have to (such as described earlier for the finance or public sector), I’d ask myself a very simple question before starting.
What am I going to learn from this Mystery Shopper program that my own Team Members – both Management & Frontline – wouldn’t have already picked up on?
Once, when I met with a fancy hotel chain, the resident Trainer told me (in a very proud tone) that their Mystery Shopper – apparently a hotel expert who traveled the world – had picked up that the wheels on the room service trolley were squeaky.
And before I could think (and perhaps keep my mouth shut) I blurted out – “Why would you pay someone for that?
Shouldn’t your Room Service People & Supervisor pick up on that?
What kind of culture exists around here if your own Team Members wouldn’t find and fix such matters on their own?”
At the end of the day, if the Mystery Shopper program looks and feels ‘police-based’ it will be wildly unpopular – and that makes improvement efforts very difficult.
Because when a program has limited credibility – it automatically has limited impact.
The key is always to define the purpose – a set of objectives for the program – that will resonate with Stakeholders and set the Organization up for future success.
So What Does a Real Customer-Experience Based Program Look Like?
There is no one single model – that’s the beauty of deep dive research – and we share a few models here from our work with innovative Clients.
Let’s Start With ‘The Brand’
Colin Shaw of Beyond Philosophy says that a brand is perception – nothing more, nothing less. It is what you think and feel about that company: an opinion, a viewpoint, an expectation.
So the Customer experience is the journey the Customer has with your brand.
When you look at it like that – then opening the Mystery Shopper design conversation with a discussion of the brand makes a lot of sense.
If your brand proposition incorporates things like trust, or accuracy, or ownership – then these values can be codified and studied during the Mystery Shopper journey.
The gaps between the ‘brand’ and the ‘Customer experience’ can be identified for further action.
We Conducted a Brand-based Mystery Shopper
One of our favorite Customer experience-based Mystery Shopper programs was with a high end hotel where the GM & Team wanted to focus exclusively on brand values.
So we designed everything to effectively measure the success (or gap) in bringing brand values to life.
It was a privilege to work with such forward thinking management and I share this example in many of my talks and workshops on Customer experience.
What About Emotion?
If you’re executing a real Customer-Experience Mystery Shopper Program then studying the emotion is a must.
One of my favorite things about the rise of Customer Experience is the inclusion of emotion in business discussions.
For too long, Customers (and Employees as well) have been treated as batches of numbers, or ‘segments’ that are expected to behave and perform in certain ways.
If they follow ‘your rules’ – then they can get what they want or what they need.
Read any established Customer Experience authority and you’ll quickly see how powerfully the topic of emotion comes up.
In fact, Bruce Temkin argues that more than 50% of the Customer Experience is driven by emotion — so how can anyone ignore that?
So in our work designing Customer-experience based Mystery Shopper programs we always talk about emotions.
During the course of booking a dining reservation what emotion do we want to ‘bring out’? It’s definitely not using the Customer’s name 2 times!
Let’s be frank – if you don’t know what emotions you are trying to evoke – how will your Team Members know?
Testing emotion is one of the best things you can do in a Customer Experience-based Mystery Shopper program.
Use the 3Es to Frame Your Study
The 3Es represent a useful measure of Customer Experience quality:
Effective: Did the job get done (e.g., open the account, file the claim)?
Easy: How much work did the Customer expend (transfers, repeats, channel-hopping)?
Emotional: Did they leave feeling respected, reassured, or relieved?
Designing scenarios and capture fields around the 3Es sparks higher-value dialogue than compliance points ever will.
We Also Like The Diary Approach to Recording Thoughts & Feelings Over the Customer Journey
For a well known theme park, we conducted a series of lengthy (6 – 8 hour) Mystery Shopper visits that incorporated thoughts & feelings.
Structured in a diary format and supported by photographs, each final report was quite lengthy.
After we completed each visit, we boiled down our observations across the journey into a number of themes.
We then cross referenced all the themes across all the visits.
Mystery Shopper research – like Focus Groups – is a deep dive qualitative research methodology. It lends itself beautifully to this kind of study.
What was also great about this program was that no score was assigned.
It was about designing an observation through the eyes of a ‘stand-in’ Customer to document the Customer journey.
The report became legendary and we still have company management write to us now and then on how useful (and revolutionary) the approach had been for them.
At the Very Minimum Consider This
At minimum, if you use Mystery Shopper for Customer Experience, upgrade your measures beyond compliance.
Sure – compliance standards are easy to measure. But they have very little to tell you with regard to the thoughts, emotions, feelings & Customer journey.
With compliance standards, you can get excellent marks and still deliver a lousy experience. And just adding the phrase ‘Customer Experience’ to something doesn’t make it so.
Thank You for Reading!
I regularly share stories, strategies, and insights from our work across Contact Centers, Customer Service, and Customer Experience. If this resonates, I’d love to stay connected.
You can drop me a line anytime, or subscribe via our website.
Daniel Ord
[email protected]
www.omnitouchinternational.com
