If a Customer Comment Falls in a Forest and No One is Around to Hear it, Does it Make a Sound?

Note: This blog post was originally published on the Clarabridge Blog - you can find the original post here.

I visit customers, prospects and partners fairly regularly, and during those visits a number of common topics come up – updates on our product releases, new partner activity, best practices and project reviews. We also discuss the business impacts of the Clarabridge solution - what customer insights they find, and how our customers use text analytics insights to improve customer value, customer experiences, and customer loyalty.


The conversations have veered into provocative territory on two recent occasions, where we have considered some interesting ethical implications of mining customer experience data.


Obligations as a listener


In one meeting at a healthcare company (attended by business, technical, and legal representatives of the company), they wanted to know how much they could mine, sort, and even “filter” text content before it was delivered to business analysts. The company has strict protocols for identifying and communicating safety and quality issues that originate from patients and providers, but they weren’t sure what to do if they found unsubstantiated information on a social media site. And they weren’t sure if they wanted all insights to go to all analysts. Generally, healthcare companies are obligated to reach out to patients and doctors to provide guidance and support if there’s a problem, and depending on the problem, they need to also report findings to an appropriate federal agency.


When it came to text mining of customer content; however, the company had questions. What is the reporting requirement, if the feedback comes from an anonymous survey? Or from a consumer posting on twitter? How much obligation does a company have to monitor, mine, and intervene in the social media world? At present there are no rules, federal guidelines, or even well defined best practices for using social media monitoring, to identify and counsel customers if they identify safety or quality issues. Should companies get out ahead of the government to develop progressive practices? Or should they wait?


Executive Compass for Customer Experience Management (CEM)


More important, once you start “listening” to the voice of your customers using text analytics and monitoring technology, are you now obligated to act on the insight? Or if you don’t listen, are you not obligated to act? What’s the moral or ethical imperative of using text mining technology to understand your customers better if they’re talking about product quality, safety, side effects, and even morbidity? Should a company be seeking to quantify this feedback?


These questions occur in many industries, not just in the healthcare sector. If a client in the financial services sector believes a contract has been broken and threatens to litigate in a blog post, an angry call to a call center, or in a survey rant – how seriously should a company take the threats? If a patron walks into a retail franchise location and spies what they believe to be a violent felon, or sex offender working behind the cash register, what is the obligation of the parent company - to notify, or enforce an HR action on the franchise? These insights are often latent in “voice of the customer” feedback, and have been found by our customers using Clarabridge – prior to using Clarabridge the information was latent and largely invisible.


Ultimately, text mining helps a customer listen, analyze and measure the extent of a problem or the outcome produced by an action. But taking action – whether it’s a sales, marketing, support, or (in the cases outlined above) safety, risk management, or criminal prosecution decisions, to me, ultimately depends on the will, and commitment of the organization to act on the insight. What do you think? What policies has your company incorporated?

Sid

Moving my Customer Experience/Text Analytics blogging to the Clarabridge Blog

As of early this week, Clarabridge has launched its own company blog at www.clarabridge.com/blog.aspx

I will continue to keep up this blog, as a vehicle for non-text analytics/customer experience blogging, and also as a personal forum for cross-posting my blogs on the Clarabridge site, but I encourage you to check out that site for interesting posts from me and other Clarabridge executives, customers, and guest posters.

Sid.

What does Google Voice have to do with Customer Experience Intelligence?

Transcript of message (feel free to listen to the message above by clicking the "play" triangle)

hello mister panic managing my name is scott B Q and i'm calling from think london and thinklondon is official economic development agencies funded unsupported by the mayor's office of london england we provide free confidential assistance to cos planning a considering a physical presence in london i'm calling to ask all clear bridges firm plans for your extension what you're considering opening office or facility in london also calling to let you know that if you have plans for a physical presence in europe within the next 2 to 3 years this is upcoming opportunity to meet directly with the special deli kitchen that will be in washington D C from september 10th the 11th of 2009 mister locations compostible by system a refunded advisorsto this is london 2012 summer olympic games and executives from think ones in we're currently scheduling individual meetings with this delegate chin for cos of the defined interest in the physical presence in europe with the next 2 to 3 acres once more my name is scott Q and i'm calling from that sink one's an exam with the bowman agency you may contact me at(703) 770-8052 again scott Q think london (703) 770-8052 and also be sending you a followup email i thank you for your time and i hope you have a wonderful weekend

I got a Google Voice account shortly after the service opened up to new subscribers.

The service allows a user to establish a phone number that can follow you and ring your work, cell, home office, etc., according to rules as simple or as complex as you like, and the service lets you pick up, transfer, conference, or send a call to voicemail (even letting you "listen" to a call as a message is being recorded and cut in if you like, just like you used to do with the old fashioned tape answering machines).

The most useful feature, in theory, is the free transcription service - once a message is recorded, Google does a speech-to-text transcription and forwards the message to your gmail account (or Google Voice phone client) for perusing so you can read it without listening.

Here's the connection to customer experience intelligence - vendors like Clarabridge ingest customer feedback (from call centers, surveys, web sites, blogs, social media, etc) and the nirvana of customer experience applications for a while has been being able to quickly, seamlessly ingest voice recordings as a data source). Most customer experience vendors prefer to operate on feedback that is already in text form, for a number of reasons:
- text is the predominant media (in surveys, web sites, emails, and call center notes)
- voice files are notoriously tricky to transcribe with accuracy due to noise, sound quality, accents, speaking styles, etc.
- to truly assess sentiment, category of feedback, and to perform real meaningful analytics on text-sourced voice of the customer information, you need to apply Natural Language Processing (NLP) somewhere (either in the transcription or in the analysis of text) to accurately determine the words in the recording, to assess the meaning (part of speech, use case) of the words, to categorize conversation topics accurately, and to accurately map customer sentiment to the specific feedback contained in a conversation.

Applying NLP to mis-transcribed voice calls can produce humourous, and incorrect results, and so far I believe that no speech-to-text vendors have yet integrated NLP into the speech processing stage (being able to correctly determine what a word is, and what part of speech a word is, for instance).

I was excited to try Google Voice to see if the world's biggest name in search had cracked the code on good transcription. If they'd done it, then perhaps we'd all be closer to being able to talk to our computers and analyze the spoken word with robust text mining technology.

My conclusion for now - the technology is STILL not ready for prime time.

Some findings:

• Even though Google knows my name (it's in my username, in my address book, etc) it transcribes voice messages with every possible name EXCEPT mine.

"Hello Steve"
"Hello slid" (a personal favorite).
"Hello said..."
"Hello Mister Panic" (another personal favorite)

Luckily, Google Voice didn't deem it necessary to call me by my college nickname
"Hello squid" (though I wouldn't have minded it as much as I did back then...)

Google needs to match up its transcription to a list of all my names in my address book, and to my user name.

• It is DEFINITELY affected by cellphone quality (lower). Calls from my home phone (a high quality VOIP line) transcribe with much more accuracy than calls from my cell (which sound fuzzier, contain road noise).

• The service does not do a good job finding beginning and end of sentences. Most messages come across as a big run-on sentence. If NLP were applied to the fragment - it should be able to estimate ends of sentences, clauses, and connected words/concepts, but lacking basic punctuation the job of processing ideas is harder to do right now particularly with longer messages. You can see the message transcription and listen to the source at the top of this post.
  

At Clarabridge we HAVE actually run text transcriptions through our customer experience solution. And it works. But accuracy of categorization, and therefore precision of analysis (how well does a category or sentiment get mapped to customer calls) and recall (how many messages are recalled when analyzing a specific type of customer feedback) can suffer.

Sometimes "good" is good enough, and your mileage will vary when you try to directly connect voice calls into text analytics solutions.

By comparison, poorly formed sentences, misspellings associated with type-written feedback work fare better than auto-transcriptions of voice calls - because the NLP algorithm in the Clarabridge engine can very accurately and consistently decipher the intent even with typewritten errors. Speech transcriptions just are more erroneous, at least today, and harder to accurately decipher.

We'll keep hoping for improvements in the space -- if Google isn't there yet, likely most other vendors aren't either.

In the meantime, if you want to ingest speech content, we generally recommend using a speech to text vendor who also runs a sample of transcribed recordings through a human assisted "error correction" stage - it costs more, but it raises the accuracy of the text to the 80+% range, and at that level of accuracy it can sail through text analytics solutions and produce very high quality customer experience insights.

Twitter and all that - Part II - Customer Support vs Customer Experience Intelligence

Don't know if you've all seen this article from USA Today last week. It had some interesting insights:

http://www.usatoday.com/tech/news/2009-06-25-twitter-businesses-consumers_N.htm

Basically the article profiled some very creative ways businesses are tapping into social media to get closer to their customers. The article recounted some well known stories (ie when Tweeters learned about a power outage during a Stanley Cup Playoffs game, or about how Dell, Comcast, and others are using Twitter to respond to customer complaints or advertise special sales).

I think all these creative uses of Twitter are fascinating, and can create real value for customers (who now have a means of communicating directly to each other and to companies via a medium they prefer and are increasingly flocking to).

From my vantage point looking at Twitter as a source for Customer Experience Intelligence, I still remain convinced that it is only one data point, and perhaps not the best data point, for comprehensive, actionable intelligence, for a number of reasons:

1) The average "tweeter" posts one tweet. Ever. See this study from Harvard, reported by the BBC. http://news.bbc.co.uk/2/hi/technology/8089508.stm While Twitter growth has been explosive, basically a very few people are tweeting a very lot. And a lot of what they have to say is not particularly insightful to companies.

2) Companies ARE getting real value in monitoring tweets, and in reaching out to customers to find out what problems they are having, or what suggestions they have. Most companies can count the number of tweets that require an action in the 10s to 100s per day - not much more than a customer support rep covers in a day, and companies who monitor, are finding that the tweets from users are not bad ways to provide support, or an ear for feedback. But looked at this way - this is really just another channel for customer support. Good, but not necessarily transformational. To the extent a support rep has a conversation with a tweeter - the rep is most likely to want to archive the conversation in a CRM system so it can be analyzed alongside all the other channels of support and feedback that are being captured.

3) Call Centers are still where all the action is, and will remain for some time. From the article:

"For perspective, consider the size of call-center operations for major brands. Comcast says it is unlikely to uproot its operations, which employ 25,000 — most of them in the U.S. — in favor of Twitter. "A majority of our customers prefer to contact us by phone," Eliason says."

"Twitter is for basic troubleshooting," says Zsolt Katona, a marketing professor at the University of California-Berkeley's Haas School of Business. "Be careful not to ignore those who rely on the phone for customer support."

4) Every member of the twitter generation is also quite likely to go to review sites, fill out point of sale surveys (online, of course), and even get tech support via online forums (or even use the phone if they have to) - and the information in these sources is richer (contains detailed insight about the experiences, problems, troubleshooting, and problem resolution). These sources also tend to allow companies to "link in" information about the transaction, the reviewer or poster, etc) permitting far more correlation of the user, the product, the experience that generated the feedback, than a twitter post will ever do.


So what am I really saying? In conclusion - Twitter is a good tool for short insights from a random and small community of fervent users. I use it myself, and find it very useful for scanning trends, seeing what people are saying about products and events, and even for communicating with people who have questions about my company or product (a few months ago I even provided some tech support via Twitter to a person who had a question about Clarabridge's offerings).

But it's not a great source of actionable intelligence from a representative segment of a customer base, and it's not a great source for "text mining" insight that can be cross referenced against specific customers, specific experiences, and specific products. The better sources for rich, qualitative/unstructured high volume customer experience insights are (and will remain for some time) call center verbatims, survey feedback (structured and unstructured feedback), and web forums, web discussion groups, web review sites, and community web sites.

Twitter and all that - Customer PERCEPTION or Customer Experience Intelligence?

There's been a lot of discussion lately about the value that Twitter, Facebook, etc brings to companies looking for customer "insights" or customer "experiences" - the thought process is that if only companies could have a live feed to Twitter or Facebook data that they could keep a finger on the pulse of customer experiences, suggestions, issues, fix problems, and ultimately create a happier, more loyal, more profitable customer base.

While there's value in social media tracking, I'm going to take a contrarian position. I believe that web/social media helps identify customer "perceptions" - but it does a poor job helping companies track real customer "experiences" - and thus the social media content is not a good place to track, measure improvements, and ultimately monitor customer experiences.

Why is that, you might ask?
1) the web is largely anonymous. If a person tweets "I'm sitting in Starbucks, my latte sucks" - you don't really know enough to fix the problem. Where is the customer? What store? Who served it? Is the shopper a frequent customer? How often does he shop there? Is the problem endemic at the store or just a transitory problem? You can't determine ANY of those insights from a 'tweet.'

2) the web doesn't generally contain a 'closed loop.' If a customer complains that he/she is having a problem with a product or service, and they get some insight that helps them fix the problem, is the case "closed?" Who closed it? What was the resolution? You can't tell that from a web forum, by and large.

In short - the web does not contain 'actionable intelligence' - it only contains - 'perceived intelligence' - to get to actionability you need more information from the person, details on the problems, and a closed loop from the resolution process that identifies that issues are tracked to completion.

You can track perceived issues, but you can't really use the insights from web content to identify, fix, and ultimately measure the impact of your changes on your customers.

Far better to track the insights, interactions, and free form conversations, chats, and verbatims from:
- customer calls to a call center
- survey feedback
- online chats
- company moderated (or at least participating) forums where you can reach out to a customer directly and work with them to identify, resolve, and track issues and resolutions.

Tracking your top 10 Dissatisfaction Drivers

A few days ago Bruce Temkin, from Forrester research, dissected a report frequently deployed by Clarabridge customers called the "Negative Influence" report.

As stated in his blog: "The Clarabridge Negative Influence report correlates the negative experiences described in the open ended feedback (based on the specific categories of experiences that are described with accompanying negative sentiment) with a low score, and also weights the ranking of most negative influences based on frequency – how often a specific experience is most often associated with negative assessments."

Rather than reposting his blog - here's a link to the piece. There's some good commentary from readers following the blog that's also worth reading.

http://experiencematters.wordpress.com/2009/04/29/tracking-your-top-10-dissatisfaction-drivers/

My worst customer experience ever

I'll never forget my worst customer experience ever. I was 1 year into my first job after college, working as a management consultant for Ernst & Young. After a nice lunch with some co-workers I stopped in at a local 7-11 convenience store to pick up a pack of Reese's Peanut Butter Cups for dessert.

I got back to my desk, ripped open the package, and upon biting into the first cup noticed that the filling wasn't as creamy as I expected. The consistency of the peanut butter was mealy and granular.

Upon review of the cup, I was HORRIFIED to discover little granular maggots embedded in the half eaten peanut butter cup. Upon further review, I was further horrified to discover that the maggots were actually alive and squirming around in the cup, and that the half eaten portion I'd quickly spit out of my mouth contained yet MORE maggots!!!!

I was, you can imagine, disgusted and livid.

After rinsing my mouth with several cups of water, brushing my mouth 3 or 4 times, and chewing through a pack and a half of gum to freshen my mouth - I proceeded to write a letter to Hershey's (manufacturer of the cups), and to delicately wrap up the half eaten portion of the remaining cup. I packaged up the whole disgusting mess and sent it, FedEx, to the customer support address on the package.

Within 2 weeks I received a sincerely worded response from a nice woman in Customer Relations. She apologized for my experience, assured me that Hershey's has manufacturing, distribution, and retail quality controls and the experience I had is both rare and unacceptable.

I was sent coupons for over $20 of Hershey's products, and a box of Hershey's coffee cups, for my trouble, and was thanked for my business.

I have to admit I was impressed by the relatively rapid response (keep in mind the experience happened in the late 80s, before the advent of internet email and the world wide web) - and in spite of the disgusting nature of my experience, I was willing to give my favorite candy another chance.

I remain a loyal Reese's Peanut Butter Cups eater. But I always break open the cups before I eat them now....

Emotion and Trust and Transparency and Text Mining

Gartner just came out with a listing of the Seven Great Concerns for CEOs in 2009 - and it listed one interesting area that touches on issues of importance to the Clarablog (note highlighted area below).

CEO Issue Three: Loss of Business and Governmental Trust
The institutions that were once counted on to safeguard the economy seem to have failed, and the lack of transparency in the economic system has been exposed. There has been a subsequent loss of trust, as well, amid fears that other unknowns are awaiting. Trust is an intangible element in business but is crucial to transact business. IT can help improve transparency in the way business is done through reputation management, e-discovery and business intelligence. Gartner also expects a strengthening of "data driven" management culture as the risks of moving forward with insufficient data become far less acceptable.

They're right, of course. Everyone wants transparency. People are less tolerant of obfuscation, misleading, mishandling, and mis-marketing from the corporations they do business with.

It's in a company's best interest to "come clean" with the facts, and get messages out to the market. It's also in a company's best interest to LISTEN to customers - hear out their issues, concerns, suggestions, and even EMOTIONS.

Text Analytics, driving business intelligence, identifies, issues, emotions, sentiments, and concerns from customers and cost- and time-effectively helps raise corporate awareness of customers' concerns, perceptions, and needs.

Customer Experience Intelligence is primarily an ROI driver to help improve services and products (see earlier blog posts), but it can also serve an important role in helping companies attain and maintain public trust during times when public faith in the institutions of business and government is weakened.

- Sid

Smart Response - a Smart Way to use Text Analytics to Improve Customer Experience

Companies need to analyze customer experience feedback 3 different ways:

1) "What are the 10 things my most profitable customers are most upset about THIS MONTH?" - Answering this question requires the ability to mine through all customer feedback, segment the data by business-definable dimensions like time and profitability, and establish an easy to view "dashboard" that quickly distills the answer from millions of pieces of text-based information to a concise answer to the question.

2) "What's new with my customers TODAY?" - Answering this question requires near real-time analysis of incoming customer feedback to see what's spiking over normal levels of feedback, so that an operational manager can quickly respond to a adverse event, service problem, or other customer experience issue that's quickly developed, to avoid the issue growing into a full-fledged problem.

3) "What should I do for the customer who just complained to me about a bad experience RIGHT NOW?" Unlike the first two examples - which involve analyzing customers in aggregate or by segments or groupings, this last type of analysis is about smart determination of an individual customer's issue, and smart response to that customer in real-time. If a customer had a bad experience, and took the time to complain - smart response is about using text mining to quickly ascertain, and adjudicate the customer experience with a suggested resolution that can be communicated directly back to the customer.

The first two categories depend on text mining large volumes of customer feedback. The last category depends on the ability to use text analytics in real-time. We announced Clarabridge SmartResponse (tm) http://www.clarabridge.com/default.aspx?tabid=136&PressReleaseID=609 as a response to our client demands for the ability to harvest and analyze feedback in real time as well as respond to individual customer communications.

Our clients have been using our Content Mining Platform to answer the first two types of questions for a while now - they've recognized that it's important to keep tabs on their customers' experiences, needs, and suggestions, and realize that ongoing analysis of trends, problems and suggestions can lead to better decisions, better product offerings, and better experiences - all of which lead to happier, more loyal, more profitable customers:

• Hotels learn what products and amenities customers want, and conversely what they can cut.
• Airlines can track the preferences and needs of most valuable travelers.
• Retailers can evaluate the response to new product lines, and track feedback on product quality, safety, and passion.

Smart Response solves an entirely different type of customer challenge. If your customer is tweeting about a bad experience, you want to respond to them immediately. If they send you an email or fill out a survey to describe a particularly bad flight experience, or stay at a hotel, or treatment at a store, you want to quickly assess the problem, determine the best possible response, and communicate back to the customer with thanks for the feedback, a sincere apology, and if warranted an offer of compensation for their troubles. Smart response is about real-time mining of feedback, real-time assessment of the problem, and real-time determination of possible responses for the customer.

Customers want not just to be heard, but to be addressed. Smart Response truly closes the customer experience loop with the customer.

RE: In2Clouds theory about text analytics and Government Transparency

MM in his blog In2Clouds referenced my recent blog on the business benefits and ROI of Text Analytics in the commercial domain, and suggested that it might be useful to be able to use Clarabridge to mash up government information with constituent feedback, suggestions, complaints about programs, producing better transparency and accountability.

I agree - It would be VERY INTERESTING -- and in fact solutions such as MM is envisioning will become reality in 2009, I'm sure.

I see a solution that incorporates:

1) Databases containing funding information ($ granted, per program, over time, over geography) coming from Federal Agencies
2) Databases containing spending information ($ spent, per region/state/jurisdiction, over time) coming from the state agencies
3) Performance Management information (job growth, roads created, educational metrics, energy capacity improvements) coming from a variety of public, private, and watchdog sources.
4) Constituency feedback (anecdotes from citizens on program qualitative benefits, outcomes, problems, observed fraud/waste/abuse) coming from a variety of social media sites on federal, state, local and third party sites)

Put all that information together, and you have truly comprehensive view of the life cycle of the government recovery/stimulus effort, and you have a living, breathing, always on, actionable view of the stimulus in action (or not, depending).

These kinds of data fusion/mash up solution visions highlight the real potential of integrating unstructured data containing qualitative feedback with structured data containing dollars and cents and performance metrics, to truly track the what, where, and WHY of complex programs and systems, and use the information to track, analyze, and improve programs.

In short - text analytics are not just for business. Government can and should also get into the game.

Achieving ROI with Customer Experience/Text Analytics Solutions

Welcome to 2009 – we’re in a recession.

Perhaps you are
-a retailer
-a travel/hospitality company
-a consumer goods company
-a high tech products company (hardware or software)
-a financial services company
-a media company

In short, you’re like one of the many companies that have decided to use customer experience intelligence solutions, like the Clarabridge Content Management Platform, to establish a text analytics-based voice of the customer initiative.

Why, you might ask, should you be looking at such an initiative in 2009 – when companies supposed to be cutting costs, getting efficient, and doing whatever they can to stay profitable, solvent, and alive? Is customer experience a “nice to have,” or a “need to have” solution?

Anecdotal insight from existing Clarabridge customers would suggest that it is very much a “need to have” solution. Text analytics, applied to customer experience management, is an important way to weatherproof your company against the economic storm we’re currently experiencing. A few big reasons:

1) ONLY THE EFFICIENT SURVIVE – and Customer Experience Analytics solutions create immediate efficiency. Most businesses collect and process customer feedback (survey content, call center content, email content) manually, and spend too much money, and too much time manually reading, coding, and analyzing content, particularly unstructured, text-based content. One of our customers reduced a staff function from 25 analysts to fewer than 5 after adopting Clarabridge, and increased its ability interpret feedback from a fraction of feedback to 100% of feedback. Another customer cut market research spending year over year by over 25%, more than covering the first 2 years of cost for the solution, and is now getting actionable feedback from customers within 24 hours of capture, down from process that used to take 30 days from capture to analysis.

2) ALERT DRIVEN INSIGHTS CATCH PROBLEMS BEFORE THEY COST YOU MONEY. Text analytics solutions do a great job of tracking experiences quickly, and quantitative metrics can measure the magnitude of a new problem before the problem causes cost, customer satisfaction, and resource impacts to your business. A leading software manufacturer uses alert driven analytics from Clarabridge to catch the “fast-growing” issues associated with new software releases so they can quickly kill bugs and errors BEFORE they affect the entire customer base – averting millions of dollars of support costs.

3) USE A SCALPEL, NOT A CHAINSAW TO CUT COSTS WHILE RETAINING DESIRABLE CUSTOMERS. Many customer-oriented companies engage in a technique called market segmentation to identify and analyze the buying patterns and preferences of discrete segments of their customer base. Before cutting a product or service expense, our clients use Clarabridge to assess how valuable the product or service is perceived to be. Others run trial cost cutting programs and evaluate the impact of the cost cutting on their “desirable” customers – the most loyal, the most profitable, and the highest net worth – to make sure they’re not engaging in an action that will drive away the desirable segment. Hotels are using Clarabridge to assess which amenities can be cut without adversely affecting business travelers. Retailers are assessing which product lines can be cut without impacting frequent customer loyalty.

4) SAVE BIG BUCKS WITH DATA-DRIVEN, INSIGHTFUL BIG DECISIONS. Over time, companies inevitably have a moment when they have to make a BIG decision. Perhaps it’s to roll out an expensive new product. Or kill a high support cost product. Or tear down a property that seems unfixable in some way. Before making the decision that can impact the company by millions of dollars, our customers use Clarabridge to validate the decision. One customer considered whether to tear down or modernize an old property due to customer complaints about noise levels and the age of the building. Using Clarabridge they were able to determine quickly that customer feedback from the property was not statistically more or less negative than feedback from newer properties, and the company subsequently decided NOT to tear down the property, averting millions of dollars of unnecessary expense.

In short – return on investments (ROI) can be clearly tied to operational, analytical and strategic use of Customer Experience Analytics, and solutions from companies like Clarabridge don’t just generate cost savings in the short term, they allow better decisions to preserve loyalty in the long term – preserving customer relationships, loyalty, and corporate profitability