Home Mystery Shopper

Mystery Shopper

Share

What we’ve learned about conducting Mystery Shopper Research on Chatbots

by OmniTouch International OmniTouch International No Comments

In this article, we share some of what we’ve learned about conducting Mystery Shopper research on Chatbots. 

You mean there’s Mystery Shopper research on Chatbots?

We recently completed our fourth Mystery Shopper research program on a Client’s chatbot offering for Customers.

And when we share that we conduct Mystery Shopper research on Chatbots the first thing we usually hear is this –

“There’s such a thing as Mystery Shopper research on Chatbots?”

Because almost any time you hear someone in the CX or Contact Centre industry talk about Chatbots, it’s on things like AI, machine learning, natural language processing and the like.

All cool and important stuff.

But inwardly focused.  You’ll never hear a Customer say –

“You know I used that NLP enhanced Chatbot at the bank/telecom/insurance company  today and I have to say that they have the best decision tree ever.”

For a Customer it’s another channel.  One they expect will help them achieve their goals.

And Mystery Shopper – when well designed – is a great tool to help ensure the Customer perspective is considered.

Some of our Clients prefer the term ‘Digital Contact Audit’ rather than ‘Mystery Shopper Research’.

But whatever you call it the goal is to improve the Customer Experience of the Chatbot.

 

Why did these organizations pursue Mystery Shopper research on their Chatbots?

 Firstly a big thanks to the four Clients – all in Asia Pacific – for engaging in the Chatbot research.

And even though the Mystery Shopper research briefs came from four different organizations, there was significant commonality across all the briefs.

In summary, the commonality sounded like this.

Client:  “We’re known in the market  for service.  And we’ve got a Chatbot.  So the Chatbot has to represent our service focus well or it could hurt our brand.

And of course, the better the service delivered by the Chatbot, the better the containment rate will be within the Chatbot channel.

 But we think we’re suffering internally from knowing ‘too much’ about our own organization. 

Too much about the Chatbot tech, about the products & services we offer and about we do things around here. 

 So we’re looking for a Customer Experience based Mystery Shopper program that’s going to look at the Chatbot from the Customer perspective – not our own.

And it’s been great to see how some organizations are raising the bar on their Mystery Shopper research.

Moving the research beyond typical compliance measurements to research that embraces the Customer Experience.

 

The parameters selected for the Chatbot research

To achieve the research objectives, we set the following key parameters for the Chatbot research.  And these parameters have served us well.

The use of Journeys

We defined, with the Client, which Customer journeys to study. So each Mystery Shopper program consisted of a defined number of specific journeys.

To help explain and define what a Customer journey looked like – and what we would be studying – we used this simple script’ as a guide.

Customer:  “Because _______I wanted to find out ________  so that I could _________.  (all blanks to be filled in)

 Customer:  “Because I’m bringing my family on holiday in June, I wanted to find out about the opening hours & entry fees at the theme park, so that I can  understand the budget. if any promotions apply and better plan for the trip overall.” 

Customer:  “Because I’m travelling overseas next week, I wanted to find out how to avoid unnecessary or unexpected roaming charges on my  mobile phone.” 

And the individual journeys that were studied, were selected for different reasons.

Sometimes the Client chose to study the journeys with the highest number of visits.

Or they chose the journeys with the highest number of opt-outs to other channels.

Often the selected journeys were those that had attracted the lowest satisfaction rating.

Because one of the great things about Mystery Shopper research is that you can choose what you want to learn.  And that approach worked perfectly for the Chatbot research.

The Customer Experience lens we used

To bring structure and clarity to assessing the Customer Experience delivered by the Chatbot, we used the 3 Levels of Customer Experience model.

A model that any Certified Customer Experience Professional (CCXP) or Customer Experience professional will be familiar with.

The 3 Levels are:

  • Effectiveness (Met my needs)
  • Ease (Was easy)
  • Emotion (what I thought and how I felt)

We assigned a scoring mechanism to each level, as well as to the overall performance and we developed a ‘dashboard’ to indicate which journeys needed improvement and at which levels.

And of course all scores were supported and informed by qualitative input.  Because when you’re assigning a score related to Customer Experience, that score must be backed up by the qualitative rationale.

The Customer Experience Mystery Shopper Program – are you on track?

A key learning that came out of the Mystery Shopper research

A key Customer Experience learning that came out of the research across all the Clients was this.

Much of the language built into the Chatbot assumed high Customer familiarity with the various product names, industry terms and context in use at that organization.

All things that a dedicated Employee of the organization would intuitively understand.  But not necessarily Customers.

When the Customer didn’t understand the language presented by the Chatbot,  it took them 3 -4 times as long to complete the chat – either successfully (they got what they needed eventually) or unsuccessfully (they needed to rollover to human assisted service).

And in Client discussions, we learned that some of the Chatbot script writing had been delegated out to different departments.

So for example, if the Chatbot was going to answer a Finance-related question, the ‘answer’ to that question was developed by the Finance Department.

On the surface, the approach seemed logical.

But it resulted in approaches and language that varied by department.  And by the competency within each department to write in a Customer-oriented manner.

Terror in the Boardroom – and the impact on your Mystery Shopper research

In closing

We have found that our Mystery Shopper work on Chatbots has been as rewarding as the work we do on other channels.

And a few parting tips:

  • Bring Customers into the picture – ask them to evaluate their journeys with you
  • Think deeply about the need that ‘drove’ the Customer to use your Chatbot
  • Consider where the Customer goes ‘after’ engaging with the Chatbot – is it all done?
  • Keep in mind that sometimes the very best Employees have the most difficult time thinking like a Customer – and understand that doesn’t come from a lack of Customer-centricity

Thank you for reading!

Daniel and Marcus

[email protected] / [email protected]

 

I’ll be presenting my Keynote speech at the Customer & User Experience Expo in London

by OmniTouch International OmniTouch International No Comments

I’ll be presenting my Keynote speech – What kind of experience does your Contact Centre deliver? – at the Customer & User Experience Expo in London next week.

More than 5,000 Customer engagement professionals are expected to descend on ExCel London (the ExCel Exhibition Centre) for the Expo, Europe’s largest Customer experience event of the year.

I look forward to meeting & engaging with those making the trek to London!

I’ve put the details for the Exhibition at the end of this short post.

So what will I talk about?

Over the years I’ve managed many Contact Centre based Mystery Shopper programs.  And my responsibility in these programs was to advise management how to improve the quality of the experience they delivered to their Customers.

In this Keynote speech I’m going to share stories from my work with Universal Studios, the Singapore Government and an award winning hotel to help Contact Centre folks

  • Proactively define the type of service ‘we deliver around here’
  • Understand and use the 3 key inputs to select Performance standards for quality conversation
  • Consider a measurement approach that addresses the needs of Customers today

I’ll also share some rather hilarious (and real life) examples of quality standards gone wrong.  Because sometimes it’s not just what to do – it’s what not to do.

I’ll close the session with a list of tips to help your Centre deliver a better Customer experience.

I guarantee a few laughs – and real life stories & examples are always the best.  You’ll love the story about Kung Fu Panda and also the 12 Teeth.

 

Exhibition details

  • Wednesday, 27 March, and Thursday, 28 March, 10:00 a.m.– 5:00 p.m.
  • ExCel Exhibition Centre, Sandstone Lane, London
  • http://www.cu-experienceshow.co.uk/
  • I speak on the 28th at 11:00AM and again at 2:45PM at the designate Keynote Theatre

See you in London next week!

Daniel

 

 

 

 

 

 

 

 

The Customer Experience Mystery Shopper Program – are you on track?

by OmniTouch International OmniTouch International No Comments

If you want to conduct a Customer Experience Mystery Shopper there are some do’s and don’ts you need to know.

A Customer Experience based Mystery Shopper program involves a lot more than just tacking on the phrase ‘Customer Experience’ in front of ‘Mystery Shopper’.

The Customer Experience Mystery Shopper Award

Some time back one of the local Awards Clubs introduced a Customer Experience Mystery Shopper Award into its portfolio.

Organizations could apply for the Mystery Shopper program and potentially win an award.

Cool I thought – it would be interesting to see what a Customer experience-based Mystery Shopper Award looks like as per a global Mystery Shopper provider. 

I got my wish

A short time later, I was helping a hospitality Client set up their Quality Assurance program.

A group of 20 senior folks were gathered around a conference table and we talking about how to select & define quality standards.

Then the head stopped and asked –

Hey Dan – did you know that we entered the Customer Experience Mystery Shopper Award this past year?

No I said – how did it go?

Well – we aren’t so sure. Because in this workshop I’m getting a sense of the complexity that goes into setting & measuring quality – but I’m not so sure it was this rigorous in our Awards entry.

He continued…

I have the final report from the Mystery Shopper provider here on my laptop – can we flash it up and talk about it?

But of course!

The cover slide whirred up on the screen.

Opening slide – very formal – The Customer Experience Mystery Shopper Report.

We were all ready. And then, next slide…

THE GREETING – score 98%

What? The Greeting? Oh – ok. Hmmm.  Anyway 98%.

 Then the next slide…

GET THE CUSTOMER NAME – score 97%

Oh…really?  This is a Customer Experience Mystery Shopper?

And it carried on from there.

Slide after slide after slide reported a compliance measurement.

Even the Hold Technique was featured.

As we hit slide 20+ someone in the room turned to me and said  – So Dan — you look a bit pale – what do you think?

Well it was an easy question to answer.

Well guys.

What you have here is a wonderfully presented compliance report – but I haven’t seen anything yet that even remotely measures or talks about the Customer experience.

And the room agreed.

Finally ‘Customer Experience’ appeared

As we carried on viewing the deck, there was a final measurement slide that said CUSTOMER EXPERIENCE.  It was a single slide.

Score – 58%.

But the legend was unclear as to how the score was derived.  That started a lively conversation – where did 58% come from?

We guessed that perhaps this was the personal score or viewpoint of the Mystery Shopper.

If so that’s a big issue.

Because Mystery Shoppers aren’t real Customers.  Once you pay and instruct someone to execute a certain scenario they’re not a real Customer.

Sure – their personal opinions can be a source of insight.

But one slide with the opinion of the Mystery Shopper is not a Customer Experience Mystery Shopper program.

There’s a lot of valid compliance-based Mystery Shopper work

Let’s put something on the table right now.

If the first thing that comes to your mind when planning a Customer Experience Mystery Shopper program is THE GREETING – then you’re on the wrong track.

But there are certainly valid reasons to conduct compliance-based Mystery Shopper.

They’re used extensively in the banking & finance industry.

Especially for ensuring regulatory compliance.

In the Public Sector, compliance-based programs provide a basic ‘minimum-standard’ dipstick.

While it’s rare to see a Public Sector program skew heavily to the Customer experience compliance based programs ensure a level of essential service is provided.

Another example of a smart compliance program is ensuring that things work the way they are supposed to work.

That when a certain telephone number is dialed at a certain time of day – that the call goes to the right place.

You’d be surprised how many times it doesn’t.

Or when a certain set of IVR options or digital instructions are followed, that the Customer ends up where they were supposed to.

As channels proliferate and overlap, it’s important to ensure that channel mechanisms work the way they are supposed to.

I sometimes call these the Omni-channel Mystery Shopper program.

So what does a Customer Experience Mystery Shopper look like?

There is no one single model – that’s the beauty of deep dive research – and we share a few models here from our work with innovative Clients.

Let’s start the discussion with brand

Colin Shaw of Beyond Philosophy says that a brand is perception – nothing more, nothing less.

It’s what you think and feel about that company.  An opinion, a viewpoint, an expectation.

So the Customer experience is the journey the Customer has with your brand.

When you look at it like that – then opening the Mystery Shopper conversation with a brand discussion makes a lot of sense.

If your brand proposition incorporates things like trust, accuracy or ownership – then these values can be codified and studied during the Mystery Shopper journey.

Then the gaps between the brand and the Customer experience can be identified.

One of our favorite brand-based programs

One of our favorite Customer Experience Mystery Shopper programs was with a high end hotel.

The GM & Team wanted to focus exclusively on brand values.

So we designed everything to effectively measure the success in bringing brand values to life.

All the scenarios were designed around brand values. And rather than scores we documented the measures of success.

I share this example in many of my talks and workshops on Customer experience.

The study of emotion is a must

One of my favorite things about the rise of Customer Experience is the inclusion of emotion in business discussions.

For too long, Customers – and Employees – have been discussed as batches or ‘segments’ that are expected to behave and perform in certain ways.

If they follow ‘your rules’ – then they can get what they want or what they need.

But if you read any established Customer experience authority you’ll note how quickly the topic of emotion comes up.

Bruce Temkin argues that more than 50% of the Customer experience is driven by emotion.

So in our work designing Customer Experience Mystery Shopper programs we always talk about emotions.

If you don’t know what emotions you are trying to evoke – how will your Frontliners know?

Testing emotion is one of the best things you can do in a Customer Experience Mystery Shopper program.

We like The Diary approach to recording thoughts & feelings 

For a well known theme park, we conducted a series of lengthy (6 – 8 hour) Mystery Shopper visits that incorporated thoughts & feelings.

Structured in a diary format and supported by photographs, each final report was quite lengthy.

After each visit we were able to boil down observations across the journey into a number of themes.

We then cross referenced all the themes across all the visits.

Mystery Shopper research is a deep dive qualitative research methodology – and lends itself beautifully to this kind of study.

The report became legendary and we still have company management write to us now and then on how useful the approach had been for them.

There are 3 aspects to a Customer Experience

Forrester teaches a very useful way to look at ‘a’ or ‘the’ Customer Experience.

Is the experience ‘effective’ – i.e. does the job get done.  Examples could be opening a bank account, making an insurance claim, getting tech support, hiring a venue for an event.

Is the experience ‘easy’ – how much work did the Customer have to put in to get their solution.  Examples include being transferred around, repeating needs multiple times, having to visit multiple channels to get help.

Is the experience ’emotional’ (in the right way of course) – did the Customer leave feeling respected, relieved or reassured?

We’ve found that designing Mystery Shopper work around the 3 ‘E’s to be very helpful in stimulating dialogue and insight.

At the very minimum

At the very minimum – if you are ready to use Mystery Shopper as a Customer experience tool, consider upgrading your measurements beyond compliance standards.

Sure – compliance standards are easy to measure.

But they have very little to tell you with regard to the thoughts, emotions, ease and ‘success’ of the experience.

Adding the phrase ‘Customer Experience’ to something doesn’t make it so.

Thank you for reading!

Daniel Ord and Marcus von Kloeden

 

 

Terror in the Boardroom – and the impact on your Mystery Shopper research

by OmniTouch International OmniTouch International No Comments

I met up with a friend who works in another Mystery Shopper research firm.

We like to compare notes on Mystery Shopper research and share practices that enable great outcomes.

Over a recent coffee we talked about how senior leadership, and their reactions to Mystery Shopper results, have a direct link to the success of the program.

Terror in the Boardroom

My friend shared this story.

“Dan – here’s what happened…most of the Mystery Shopper results were ‘ok’.

Nothing spectacular, but for an organization of their scale, the essential compliance KPIs were being met.

But one of their Customer touchpoints really struggled with their turnaround time commitment.

Rather than receiving a reply within 2 – 3 days, reply time-frames ranged from one week to no reply received within the time-frame promised.

We knew what we were getting into when we took the program.

But even we were taken aback when – after submitting the final results – the Client asked us to edit out the poor results.

And not just once – we had to redo the complete deck and set of reports three times before they were satisfied. 

Later on a Service Quality Manager told us what happened.

The Blame Game

When senior management saw the poor results for turnaround time, they yelled at the Participants and launched into assigning blame.

Of course, the Participants were stunned into silence.

And the unspoken message came across loud and clear.

It’s safer to hide bad results then to risk angering Senior Management.

Clearly a company culture issue.  And one that kills any chance at systemic improvement.

Avoid Terror in the Boardroom

It’s sad to see a viable Mystery Shopper program go down in flames due to fear of Senior Management.

The Mystery Shopper Agreement

I’d suggest is asking Senior Management sign a simple agreement when the Mystery Shopper program is approved.

Perhaps something like this:

The purpose of our Mystery Shopper program is to ________.  It’s likely we will uncover things that we want to hear – and things that we don’t.

We will resist the natural urge to cleanse results to make them look better.

We can only get better if we truly know how we’re doing – and for CX-based Mystery Shopper programs, how our Customers are experiencing us.  

With this in mind, we will take the good with the bad, the great with the not so great, look at results in perspective – and use them to help us move forward. 

Let your Research Partner present findings

Mystery Shopper Research Partner

Your Research Partner is in the best position to share methodology, compare and contrast findings with other organizations and give specific examples of both the good and not so good results with ideas for improvement.

The Research Partner operates outside the politics of the organization.  That brings an important level of objectivity and credibility to the process.

When the Research Partner doesn’t present – it’s left to someone within the organization to share findings

But when findings are presented ‘in-house’, a lot of context, examples and recommendations go missing.

And the politics can be more ‘highly charged’.

We hope these few words on Mystery Shopper research are helpful to you.

Avoid terror in the boardroom!  And thank you for reading,

Daniel

How Mystery Shopper Research contributes to the Customer Experience

by OmniTouch International OmniTouch International No Comments

In this short article I share how Mystery Shopper Research contributes to the Customer Experience by looking at the first dimension of Customer Experience.

It’s great to see everybody studying Customer journeys

Customer journey review has a powerful impact on the decisions you make around serving Customers.

And Mystery Shopper researchers have been studying Customer journeys for years.

Let’s see where Mystery Shopper fits into the Customer Experience.

There are 3 dimensions to the Customer experience

An article on Customer Experience triggered this article on Mystery Shopper research.

Bruce Temkin wrote that there are three dimensions to consider when studying a Customer experience –

  1. The actual experience – what really happened
  2. The Customer’s perception of the experience – how the Customer perceived the experience
  3. The Customer’s reaction to the experience – what the Customer does after the experience

The achievement of the perfect Customer experience occurred at Dimension #2 – the Customer’s perception of the experience.

And that makes absolute sense.  Because as we know, perception = reality.

Dimension #3 – the Customer’s reaction to the experience – is where the Return on Investment lies.

Reactions like brand advocacy, telling X number of other people and posting positive reviews on social media.

“The perfect Customer experience is a set of interactions that consistently exceed the needs and expectations of a Customer…

While the outcome of delivering great customer experiences will hopefully turn many Customers into advocates, I don’t think an experience is any less great if a Customer keeps her satisfaction to herself.”

I think Mystery Shopper is a brilliant tool to assess Dimension #1 – the actual Customer experience – or what really happened.

So what really happened?

Mystery Shopper tells you what ‘really happens’.  Not what you hope will happen or not what you think happens.

Because what really happens drives the Customer’s perception of what happened.

You can measure simple compliance aspects – the plasma screens were working, the flowers were fresh, the name tags were worn, the live chat session was picked up quickly.

Or you can evaluate more complex behaviours – empathy was demonstrated, the upsell attempt was well executed, the ambience was warm.

You can evaluate simple journeys – like buying a mattress.

Or more complex journeys like making an insurance claim.

If things work the way you designed them to work, you generate better Customer emotion, perception and memory.

The Customer Experience Mystery Shopper Program – are you on track?

But things don’t always work the way you think they do

Sometimes, the actual experience doesn’t work the way senior management thinks it does.

For example, the calls didn’t get picked up, the emails were not replied to, the service quality was not up to standard.

Or to get digital, the FAQ system didn’t load, the ecommerce process failed, the online submission got stuck, the IVR sent the call to the wrong queue.

Too often assumptions are made about ‘how well’ things are actually working.

I remember a very memorable Mystery Shopper program where not a single email was replied to within a 2 week despite a 3 day turnaround promise.

Sometimes the bad news is hidden out of fear

It’s a bad scenario – but it happens a lot.

Internal Employees ensure that data is ‘scrubbed’ before it is presented to senior management.

That’s because, the fear of senior management reprisal is so strong that the cultural belief is that it’s better to mask bad results than deal with the senior fallout.

Not a great recipe for Customer Experience.

Three examples of what was ‘really happening out there’ which ended up surprising everyone

In this last section we share three  examples of what we uncovered with Mystery Shopper research:

A software company

We were conducting Mystery Shopper Research on the email touchpoint of a global tech company. We were writing emails in Chinese to test their China operations team.

After 3 days we had not received a single reply. This was strange because the promised turnaround time was 24 hours.

We alerted the Client who then alerted their internal Operations folks.

It turned out that because of an incorrect setting in the webserver, the Chinese language emails had been inadvertently forwarded to the U.S.

And the U.S. team had not actioned on their side to ask why they were receiving these emails in Chinese.

A Telecoms company

We were conducting a review of Customer service enquiries for the prepaid services of a well known mobile company. We had been instructed to press a certain sequence of options on the IVR menu.

All options ‘read out’ correctly as we pressed each button but when we reached the last option the phone line was automatically disconnected.

We reported this to the Client within two days of discovery but the intelligence wasn’t action-ed.

A couple of months later, we were presenting the results to the Board.  When we shared the finding about the IVR, a very senior person at the table told us that this was impossible and such a thing would never happen at their company.

A member of the audience grabbed the high tech speakerphone on the conference table, pressed the sequence we had cited and oops – the call was disconnected.

Of course the room went silent so we diplomatically pulled the attention back to the presentation at hand.

Now that was a stressful moment.

A beer company

We were engaged to test the promotional capabilities of brand ambassadors for a European beer brand.

If the brand ambassador mentioned just one or two ‘promotional’ phrases to the Customer s part of the Guest ordering a beer, they received an on-the-spot gift voucher as a reward.

At the start of the program, the Client was convinced that their top-down training and campaign mechanics had been widely disseminated across the Team.

But early into the program, the scores on sharing the promotional messaging came in significantly lower than expected.

This finding triggered innovative discussions on how to better get the brand message out to the Team.

We were impressed because this was the right way to receive the message – not allocate blame – but find solutions.

Follow-up Mystery Shopper work validated that the new innovations had worked.

In closing

Here I’ve shared just 3 simple examples of how professional Mystery Shopper research can be used to validate Dimension #1 – what really happened with the Customer experience.

Armed with accurate Dimension #1 intelligence, the organization can ensure that it has put everything into place that it needs and wants to create a particular experience for its Customers.

From there, VOC programs, VOE programs, Unsolicited Feedback and Ethnographic Research can take over for Dimension #2 – the Customer’s perception of the experience.

And of course, from there, we hope that the Customer will bring us ROI through the actions taken after the experience such as referring us to others.

Thank you for reading!

Daniel

 

Dear Contact Centre Agents – please stop pushing Customers to self-help

by OmniTouch International OmniTouch International No Comments

This article examines the Customer experience impact of pushing Callers to self help options as a method to improve Contact Center productivity.

There are some odd approaches to achieving Contact Center productivity.

One of my least favorite is what I call Tai Chi’ing the Customer.

Here’s what it sounds like –

Good morning this is Andrew, how may I help you?

Hi Andrew, Siti here. Can I ask how to apply for the scholarship?

Sure Siti.

It’s all on the website.

Go to abc.com and you’ll find everything there.

Oooh ah – punch – go away – ack!

Short, sweet – unhelpful.

It’s Tai Chi’ing when you push someone to self-help without offering to help first

Can you imagine you’re sunning at the local swimming pool and you see someone struggling to stay afloat in the deep end?

Look! (you shout) Just grab that orange floaty thing a few meters from you and you’ll be fine!

Unlikely – I assume you’d jump in an help – wouldn’t you?

 

Designed journeys have exception handling too

Sure – perhaps the digital journey had been planned such that the Customer would have utilized the website.

After all the concept of opti-channel refers to the best channel for a particular Customer performing a particular task.

But when you offer multiple channels, you make a promise to honor the Customer regardless of which channel (or channels) they decide to use.

When I work with students in CX courses I explain it this way –

“When your Customer wakes up in the morning they have a choice – a choice in how they interact with you.

They could call, email, text, or drop in on your Service Centre as they’ll be in town anyway.

At a big picture level, we have to honor them and help get the job done.”

Astute journey mapping experts will recognize that in the case of self-service (website, online FAQs and the like), some ratio of the voice calls received in the Centre will have placed by Customers after trying self-service first – and where the self-service option failed to deliver the desired information (or might have required too much effort to find).

So to be Tai Chi’ed on a voice call – right back to the self-service channel that had failed in the first place – is clearly not an award winning strategy.

 

There is a danger when measuring service through a compliance lens

We had been working with a large institution on their Mystery Shopper program.

To allow for trending period over period (in this case years) – compliance standards for measurement had not been refreshed or updated for years.

So – when tabulations were done, the scores were (as expected) good.

All the greetings, closings and using Customer names ‘two times’ needed to generate and show off great results to senior management.

But during analysis of the conversations, we had picked up on the extensive use of the Tai Chi approach (that’s what good Mystery Shopper providers do).

Unfortunately, that finding was considered incidental at best.

If this had been a real Customer experience-based Mystery Shopper program, the measurements would have been different – and learnings around the use of Tai Chi to handle Customers would have been embraced and action-ed differently.

 

As I say widely and often these days – if your Mystery Shopper program delivers the results you had hoped for or expected you’re probably doing it wrong.

 

How about a version like this?

Good morning this is Andrew, how may I help you?

Hi Andrew, Siti here. Can I ask how to apply for the scholarship?

Sure Siti. Happy to help with that!

(A bit of to and fro to address Siti’s needs)

Ok Siti – had you viewed our website before? 

Ah ok – no worries – let me show you where, in future, you can easily reference what we’ve been taking about on this call.

Of course, if the website does not provide an easy reference – this becomes business intelligence that can be aggregated and funneled to the CX Team for review and enhancement of that touchpoint.

And it’s the reduction of a future call that makes the big difference in your Contact Centre productivity.

Thank you for reading – and please – no more Tai Chi!

Daniel

[email protected] / www.omnitouchinternational.com

How to conduct (and not conduct) a Customer experience Mystery Shopper

by OmniTouch International OmniTouch International No Comments

Adding the phrase ‘Customer Experience’ in front of something doesn’t make it so.  And this applies to Mystery Shopper research.   A Customer Experience Mystery Shopper is something very particular and special.

Some time back in Singapore, one of the local Awards Clubs introduced a Customer Experience Mystery Shopper Award into its portfolio.

Cool I thought – it will be interesting to see what a Customer experience-based Mystery Shopper Award looks like as per a global vendor.

A short time later, I was helping a hospitality Client set up the Quality Assurance program for their Contact Centre.

A group of 20 senior folks were gathered together in the conference room, and we were in the midst of selecting & defining quality standards when one of them stopped and asked –

Hey Dan – did you know that we entered the Customer Experience Mystery Shopper Award this past year?

No I said – how did it go?

Well – we aren’t so sure. Because in this workshop I’m getting a sense of the complexity that goes into setting & measuring quality – but I’m not so sure it was this rigorous in our Awards entry.

He continued…

I have the final report from the Mystery Shopper vendor here on my laptop – can we flash it up and talk about it?

But of course!

The cover slide whirred up on the screen.

Opening slide – very formal – The Customer Experience Mystery Shopper Report etc. etc.

We were all ready. And then, next slide…

THE GREETING – score 98%

What? The Greeting? Oh – ok. Anyway 98%

Then the next slide…

GET THE CUSTOMER NAME – score 97%

Oh…really?

And it carried on from there.

Slide after slide after slide reported on a compliance measurement – even the Hold Technique was featured.

As we hit slide 20+ something someone in the room turned to me and asked – So Dan — you look a bit pale – what do you think?

Well it was an easy question to answer.

Well guys – I said– what you have here is a wonderfully presented compliance report – but I haven’t seen anything yet that even remotely measures or talks about the Customer experience.

And the room agreed.

Interestingly – in this report there was a final measurement slide that said CUSTOMER EXPERIENCE (yes – that’s right – a single slide).

Score – 58%.

But the legend was unclear as to how the score was derived.

After some discussion around the table we guessed that perhaps this was the personal score or viewpoint of the Mystery Shopper.

So let’s put it on the table right now.

If the first thing that comes to your mind when planning a Customer Experience-based Mystery Shopper program is THE GREETING – then you are on the wrong track.

 

There’s a lot of compliance-based Mystery Shopper work going on out there

There are certainly some valid reasons for having a solid compliance-based Mystery Shopper program.

They’re used extensively in the banking & finance industry – especially for ensuring regulatory compliance.

In the Public Sector, compliance-based programs provide a basic ‘minimum-standard’ dipstick – that provides high level assurance that when an email gets sent – it receives a reply.

Or when a telephone call is placed, some kind of basic response – along with basic courtesy – is provided.

While it’s rare to see a Public Sector program skew heavily to the Customer experience (now there’s an opportunity I’d love to be part of!) compliance based programs ensure a level of essential service is provided.

Another example of a smart compliance program is ensuring that things work the way they are supposed to work.

That when a certain telephone number is dialed at a certain time of day – that the call goes to the right place (you’d be surprised how many times it doesn’t).

Or when a certain set of IVR options or digital instructions are followed, that the Customer ends up where they were supposed to and got what they were supposed to get.

As channels proliferate and overlap, it’s really important to ensure that channel mechanisms work the way they are supposed to – the omni-channel Mystery Shopper program.

It’s not right to say that compliance-based programs are ‘bad’ while CX-based programs are ‘better’.

It’s always about defining what you want to learn and then figuring out the best way to learn it.

But there’s a big opportunity in putting together a solid Customer Experience-based Mystery Shopper program.

And no – adding the word Customer experience in front of something doesn’t make it so.

 

If you are considering a compliance-based Mystery Shopper program ask yourself – what am I going to learn from this program that I couldn’t learn from my own Team Members?

Assuming that you’re not conducting Mystery Shopper because you have to (such as described earlier for the finance or Public Sectors), I’d ask myself a very simple question before starting.

What am I going to learn from this Mystery Shopper program that my own Team Members – both Management & Frontline – wouldn’t have already picked up on? 

Once, when I met with a fancy hotel chain, the resident Trainer told me (in a very proud tone) that their Mystery Shopper – apparently a hotel expert who traveled the world – had picked up that the wheels on the room service trolley were squeaky.

And before I could think (and perhaps keep my mouth shut) I blurted out – why would you pay someone for that? 

Shouldn’t your Room Service People & Supervisor pick up on that? 

What kind of culture exists around here if your own Team Members wouldn’t find and fix such matters on their own?

At the end of the day, if the Mystery Shopper program looks and feels ‘police-based’ it will be wildly unpopular – and that makes improvement efforts very difficult.

Because when a program has limited credibility – it automatically has limited impact.

The key is always to define the purpose – a set of objectives for the program – that will resonate with Stakeholders and set the Organization up for future success.

 

So what does a real Customer experience-based Mystery Shopper program look like?

There is no one single model – that’s the beauty of deep dive research – and we share a few models here from our work with innovative Clients.

Let’s start this discussion with the brand

Colin Shaw of Beyond Philosophy says that a brand is perception – nothing more, nothing less. It is what you think and feel about that company: an opinion, a viewpoint, an expectation.

So the Customer experience is the journey the Customer has with your brand.

When you look at it like that – then opening the Mystery Shopper design conversation with a discussion of the brand makes a lot of sense.

If your brand proposition incorporates things like trust, or accuracy, or ownership – then these values can be codified and studied during the Mystery Shopper journey.

The gaps between the ‘brand’ and the ‘Customer experience’ can be identified for further action.

One of favorite Customer experience-based Mystery Shopper programs was with a high end hotel where the GM & Team wanted to focus exclusively on brand values. So we designed everything to effectively measure the success (or gap) in bringing brand values to life.

It was a privilege to work with such forward thinking management and I share this example in many of my talks and workshops on Customer experience.

 

And what about emotion? If you’re executing a real Customer experience-based Mystery Shopperprogram then studying the emotion is a must

One of my favorite things about the rise of Customer Experience is the inclusion of emotion in business discussions.

For too long, Customers (and Employees as well) have been treated as batches of numbers, or ‘segments’ that are expected to behave and perform in certain ways.

If they follow ‘your rules’ – then they can get what they want or what they need.

But if you read any established Customer experience authority you’ll note how quickly (and powerfully) the topic of emotion comes up – in fact Bruce Temkin argues that more than 50% of the Customer experience is driven by emotion – so how can that be ignored?

So in our work designing Customer-experience based Mystery Shopper programs we always talk about emotions.

During the course of booking a dining reservation what emotion do we want to ‘bring out’? It’s definitely not using the Customer’s name 2 times!

Let’s be frank – if you don’t know what emotions you are trying to evoke – how will your Team Members know?

Testing emotion is one of the best things you can do in a Customer Experience-based Mystery Shopper program.

 

We also like The Diary approach to recording thoughts & feelings over the course of the journey

For a well known theme park, we conducted a series of lengthy (6 – 8 hour) Mystery Shopper visits that incorporated thoughts & feelings.

Structured in a diary format and supported by photographs, each final report was quite lengthy.

But after each visit was done, we were able to boil down observations across the journey into a number of themes.

We then cross referenced all the themes across all the visits.

Mystery Shopper research – like Focus Groups – is a deep dive qualitative research methodology – and lends itself beautifully to this kind of study.

What was also great about this program was that no score was assigned.

It was about designing an observation through the eyes of a ‘stand-in’ Customer to document the Customer journey.

The report became legendary and we still have company management write to us now and then on how useful (and revolutionary) the approach had been for them.

At the very minimum

At the very minimum – if you are ready to use Mystery Shopper as a Customer experience tool, consider upgrading your measurements beyond simple compliance standards.

Sure – compliance standards are easy to measure.

But they have very little to tell you with regard to the thoughts, emotions, feelings & Customer journey.

With compliance standards, you can get excellent marks and still deliver a lousy experience.

And just adding the phrase ‘Customer Experience’ to something doesn’t make it so.

Thank you for reading!

Daniel

 

[email protected] / www.omnitouchinternational.com / (65) 9838 2353

 Daniel Ord