ClaraNext - Part I

 It's July 2024.  About 6 months ago a trajectory that extended back over 30 years has come to an end. A career that started with small strategies, grew into big data, and evolved into artificial intelligence powered conversational analytics.  

Chapter 1: Microstrategy 

In January 1993 I had the privilege to join a ragtag group of consultants and analytics geeks at a small, under 20 (at the time) employee startup called MicroStrategy.  MicroStrategy was the brainchild of a twentysomething MIT graduate, Michael Saylor, who believed that "micro"computers, or PCs, could be harnessed using modelling and analytical tools to help drive better corporate strategy decisions in the Fortune 500 - and thus "MicroStrategy" was born. 

By the time I'd joined MicroStrategy the company had migrated away from pure PC-based strategy modelling and towards using databases containing corporate operational information to create analytics and visualizations.  The company was a pioneering vendor offering "business intelligence" (or BI) software, and over 10 years the company (and I) learned how to model and build very large databases, and we created software that could interrogate arbitrarily large data sets and create rich analytics.  

Along the way I also learned how to build a company, manage a P&L, sell and compete against smaller and bigger competitors, partner, IPO, and (after the dot com bubble tore a huge hole in half of the business), restructure, reorganize, and rebuild after crisis and business challenge.  In short, I learned how to an entrepreneur.  

In 2001, after a 9 year odyssey at MicroStrategy, I'd finally decided I need to take a break, recharge, and rethink the next chapters of my life.  I'd recently gotten married, work was a grind, and I found myself thinking about all the good and bad decisions we'd made the past few years, what we'd learned from it all, and wondering if I might be foolish enough to start a business informed by all that knowledge.  

Chapter 2: Claraview 

By October 2001 - I and two partners decided to start a company, Claraview.  The premise was simple -  we were three smart (we thought) guys with a ton of ideas that would change the world of big data analytics.  All we needed to do was build, sell, and be successful.   It took us a nearly a year to land our first big accounts, but we had three key assets - 1) smart, customer savvy, agile founders, 2) access to a network of really talented professionals who'd either left or been let go from MicroStrategy during its several rounds of restructuring, and 3) knowledge of many customers and partners who desperately needed the knowledge we had had to continue building, maintaining, and innovating big data solutions. 

We conceived Claraview as a company that would build solutions that helped organizations achieve clarity of insight - powered by big data.  In 2001 and 2002, however, most of the tech world was still licking its wounds from the bursting of the dot-com tech bubble, and we reverted to the more predicable business model of providing consulting and integration services to customers.  But we founders still harbored a desire to build a big-idea powered software company and during our spare time noodled over and ideated a few ideas.  

By 2006 we'd grown Claraview to nearly 100 employees, were generating millions of dollars of revenue, an had toyed with a few product ideas we considered bringing to market.  One idea was based on applying analytics to the intelligence community.  The other was based on automating business processes with workflow based forms technology.  The third idea - to build a technology platform that ingested unstructured data (think sentences, documents, conversations, etc) and turn it into robust, quantitatively rich insights and analytics - was the charm.  That idea, to build a platform that created clarity of insights by bridging the fields of unstructured data and structured analytics, was ultimately the idea that led to the creation of Clarabridge.  Clarabridge was prototyped  and nurtured by Claraview in 2005, spun into a separate company in 2006, and, once the founders sold off the Claraview business in 2008, became the sole focus for me and our team for nearly 15 years.  

Next: Clarabridge - and the rise of experience intelligence. 

What things may come… Text Analytics, Customer Experience Predictions for 2011

http://www.clarabridge.com/ClarabridgeBlog/tabid/189/EntryId/60/What-things-may-come-2011.aspx

2010 has been a banner year, not only because as a company, Clarabridge has passed many significant milestones, but because sentiment and text analytics have passed from infancy to adulthood over a very short, intense period of time. Market forces have converged to provide us with the opportunity and the means to push sentiment and text analytics into many mainstream applications across a variety of industries. You can see the tangible results in our year end press release, and the rush of interest from the analyst community, with folks like Gartner, Hurwitz’s Fern Halper, Constellation’s Ray Wang, Bruce Temkin, Paul Greenberg all talking about the key role multichannel text and sentiment analytics will play in 2011.

2010 has been a banner year, not only because as a company, Clarabridge has passed many significant milestones, but because sentiment and text analytics have passed from infancy to adulthood over a very short, intense period of time. Market forces have converged to provide us with the opportunity and the means to push sentiment and text analytics into many mainstream applications across a variety of industries. You can see the tangible results in our year end press release, and the rush of interest from the analyst community, with folks like Gartner, Hurwitz’s Fern Halper, Constellation’s Ray Wang, Bruce Temkin, Paul Greenberg all talking about the key role multichannel text and sentiment analytics will play in 2011.

So, that all said, I’ve decided to put out my own predictions for 2011 in the sentiment and text analytics space.

• In 2011 expect to see the beginnings of the merging of text analytics and predictive analytics technology stacks. Vendors of text analytics solutions will begin to move beyond looking back, or analyzing the moment, and move into the realm of identifying the predictors of positive outcomes in a campaign, identifying and tracking the drivers of loyalty using advanced predictive models, and using predictive techniques to help prioritize and manage changes to customer experiences. This will happen in a variety of industries including financial services, hospitality, retail, and healthcare industries.

• The value of text analytics, especially related to customer experience monitoring, analysis, and action, is amplified when the content being analyzed is sourced from a broad, representative data set of customers. In 2010, while pure play social media monitoring and analysis was exploding across the scene, a small but rapidly growing set of Fortune 1000 organizations began to recognize that multichannel customer analytics, from a wide variety of text based sources (online, offline, call center, and community/survey), provided a much fuller, much more accurate view of customer emotion, customer intent, and loyalty than single source solutions. In 2011 expect to see solutions that fuse multiple channels of customer interaction, engagement, and support to be a dominant growth area for semantic analysis.

• Text mining and sentiment analysis is evolving as analysts become more sophisticated and demanding of insights. Over the past few years text analytics solutions focused on “structuring” the unstructured, fusing it with structured data, then on applying industry or domain specific classification models for further refined analysis and investigation, and most recently on instrumenting the data for robust monitoring, analysis, and alerting of past or current conditions. In 2011 expect use cases to continue to be analyst driven and facilitated through a business intelligence (BI) reporting framework, but expect the latency of solutions to decrease, and the automation of insight assessment, delivery, and action to increase. For leading edge users expect to see text analytics solutions assess customer problems and automatically suggest communications to customers, or to automate the recommendations of customer support representatives. In short, look for text analytics to become more “real time” and to become more automated in distilling and disseminating insights.

• Vendors of text analytics solutions have typically focused on the 80/20 rule in the past few years – supporting the languages and markets closest to their home bases that provide 80% value for 20% of the effort. But enterprises collect and analyze content from around the world, in many languages, and over time they are demanding that more content from more sources be integrated into their solutions, and that insights be made available to more users across the globe. In 2011 expect to see North American vendors to add European, Latin American and Asian language support to their offerings to expand their business footprints internationally through direct and indirect sales and marketing efforts, and expect to see enterprise applications be deployed across multiple user communities worldwide using localized application interfaces and analytics.

• Over the past few years customer experience analytics solutions have grown – bigger data volumes, more content sources, more end users across an organization. Vendors who have stepped up to the challenges of enterprises have responded by creating more scalable, more powerful solutions and underlying solution architectures. Expect the enterprise trend to continue in 2011, but just as the web has allowed consumers to create instantly personalized, tailored online experiences, expect in 2011 text analytics applications will also be offered to the individual analyst and information worker though more customizable, more self-service interfaces and functionality. In short, expect the solution space to scale the enterprise while also being designed and deployed to the individual knowledge worker on a one to one basis.

• Over the past few years text mining and analytics solutions have not been seamlessly integrated into operational systems or processes. In 2010 a number of vendors started offering robust APIs and frameworks for integrating information into operational systems – CRM, Call Center, Campaign Automation, and Social Media engagement spaces. In 2011 expect to see these operationally integrated solutions come on line from many pioneering solution providers, and to see text insights automatically driving recommendations, actions, or process improvements into operational systems and platforms. Enabling this trend, expect to see partnerships and collaborations between historically analytic vendors and historically transactional vendors in the customer support, customer experience, and customer intelligence spaces. Clarabridge and Verint’s integrated speech/text offering is a leading indicator of creative collaborations to come.

With this all said, what can businesses do to prepare? Invest in voice of the customer pilot programs if you don’t have one, and if you do, invest in bringing in more employees with analytics backgrounds. Look forward, and spend time and money now to understand how all of your systems are going to interact with each other. Put together an action plan. Make your organization customer centric, and the commitment has to be from the top down. Leaders set the tone for how people act and react. Your company culture will define your ability to succeed, so don’t underestimate the value of making sure everyone in the company is focused on listening to the customer.

Data is the present and the future, those who know how to use it, will benefit the most, Knowledge is power is a mantra for a reason.

If a Customer Comment Falls in a Forest and No One is Around to Hear it, Does it Make a Sound?

Note: This blog post was originally published on the Clarabridge Blog - you can find the original post here.

I visit customers, prospects and partners fairly regularly, and during those visits a number of common topics come up – updates on our product releases, new partner activity, best practices and project reviews. We also discuss the business impacts of the Clarabridge solution - what customer insights they find, and how our customers use text analytics insights to improve customer value, customer experiences, and customer loyalty.


The conversations have veered into provocative territory on two recent occasions, where we have considered some interesting ethical implications of mining customer experience data.


Obligations as a listener


In one meeting at a healthcare company (attended by business, technical, and legal representatives of the company), they wanted to know how much they could mine, sort, and even “filter” text content before it was delivered to business analysts. The company has strict protocols for identifying and communicating safety and quality issues that originate from patients and providers, but they weren’t sure what to do if they found unsubstantiated information on a social media site. And they weren’t sure if they wanted all insights to go to all analysts. Generally, healthcare companies are obligated to reach out to patients and doctors to provide guidance and support if there’s a problem, and depending on the problem, they need to also report findings to an appropriate federal agency.


When it came to text mining of customer content; however, the company had questions. What is the reporting requirement, if the feedback comes from an anonymous survey? Or from a consumer posting on twitter? How much obligation does a company have to monitor, mine, and intervene in the social media world? At present there are no rules, federal guidelines, or even well defined best practices for using social media monitoring, to identify and counsel customers if they identify safety or quality issues. Should companies get out ahead of the government to develop progressive practices? Or should they wait?


Executive Compass for Customer Experience Management (CEM)


More important, once you start “listening” to the voice of your customers using text analytics and monitoring technology, are you now obligated to act on the insight? Or if you don’t listen, are you not obligated to act? What’s the moral or ethical imperative of using text mining technology to understand your customers better if they’re talking about product quality, safety, side effects, and even morbidity? Should a company be seeking to quantify this feedback?


These questions occur in many industries, not just in the healthcare sector. If a client in the financial services sector believes a contract has been broken and threatens to litigate in a blog post, an angry call to a call center, or in a survey rant – how seriously should a company take the threats? If a patron walks into a retail franchise location and spies what they believe to be a violent felon, or sex offender working behind the cash register, what is the obligation of the parent company - to notify, or enforce an HR action on the franchise? These insights are often latent in “voice of the customer” feedback, and have been found by our customers using Clarabridge – prior to using Clarabridge the information was latent and largely invisible.


Ultimately, text mining helps a customer listen, analyze and measure the extent of a problem or the outcome produced by an action. But taking action – whether it’s a sales, marketing, support, or (in the cases outlined above) safety, risk management, or criminal prosecution decisions, to me, ultimately depends on the will, and commitment of the organization to act on the insight. What do you think? What policies has your company incorporated?

Sid

Moving my Customer Experience/Text Analytics blogging to the Clarabridge Blog

As of early this week, Clarabridge has launched its own company blog at www.clarabridge.com/blog.aspx

I will continue to keep up this blog, as a vehicle for non-text analytics/customer experience blogging, and also as a personal forum for cross-posting my blogs on the Clarabridge site, but I encourage you to check out that site for interesting posts from me and other Clarabridge executives, customers, and guest posters.

Sid.

What does Google Voice have to do with Customer Experience Intelligence?

Transcript of message (feel free to listen to the message above by clicking the "play" triangle)

hello mister panic managing my name is scott B Q and i'm calling from think london and thinklondon is official economic development agencies funded unsupported by the mayor's office of london england we provide free confidential assistance to cos planning a considering a physical presence in london i'm calling to ask all clear bridges firm plans for your extension what you're considering opening office or facility in london also calling to let you know that if you have plans for a physical presence in europe within the next 2 to 3 years this is upcoming opportunity to meet directly with the special deli kitchen that will be in washington D C from september 10th the 11th of 2009 mister locations compostible by system a refunded advisorsto this is london 2012 summer olympic games and executives from think ones in we're currently scheduling individual meetings with this delegate chin for cos of the defined interest in the physical presence in europe with the next 2 to 3 acres once more my name is scott Q and i'm calling from that sink one's an exam with the bowman agency you may contact me at(703) 770-8052 again scott Q think london (703) 770-8052 and also be sending you a followup email i thank you for your time and i hope you have a wonderful weekend

I got a Google Voice account shortly after the service opened up to new subscribers.

The service allows a user to establish a phone number that can follow you and ring your work, cell, home office, etc., according to rules as simple or as complex as you like, and the service lets you pick up, transfer, conference, or send a call to voicemail (even letting you "listen" to a call as a message is being recorded and cut in if you like, just like you used to do with the old fashioned tape answering machines).

The most useful feature, in theory, is the free transcription service - once a message is recorded, Google does a speech-to-text transcription and forwards the message to your gmail account (or Google Voice phone client) for perusing so you can read it without listening.

Here's the connection to customer experience intelligence - vendors like Clarabridge ingest customer feedback (from call centers, surveys, web sites, blogs, social media, etc) and the nirvana of customer experience applications for a while has been being able to quickly, seamlessly ingest voice recordings as a data source). Most customer experience vendors prefer to operate on feedback that is already in text form, for a number of reasons:
- text is the predominant media (in surveys, web sites, emails, and call center notes)
- voice files are notoriously tricky to transcribe with accuracy due to noise, sound quality, accents, speaking styles, etc.
- to truly assess sentiment, category of feedback, and to perform real meaningful analytics on text-sourced voice of the customer information, you need to apply Natural Language Processing (NLP) somewhere (either in the transcription or in the analysis of text) to accurately determine the words in the recording, to assess the meaning (part of speech, use case) of the words, to categorize conversation topics accurately, and to accurately map customer sentiment to the specific feedback contained in a conversation.

Applying NLP to mis-transcribed voice calls can produce humourous, and incorrect results, and so far I believe that no speech-to-text vendors have yet integrated NLP into the speech processing stage (being able to correctly determine what a word is, and what part of speech a word is, for instance).

I was excited to try Google Voice to see if the world's biggest name in search had cracked the code on good transcription. If they'd done it, then perhaps we'd all be closer to being able to talk to our computers and analyze the spoken word with robust text mining technology.

My conclusion for now - the technology is STILL not ready for prime time.

Some findings:

• Even though Google knows my name (it's in my username, in my address book, etc) it transcribes voice messages with every possible name EXCEPT mine.

"Hello Steve"
"Hello slid" (a personal favorite).
"Hello said..."
"Hello Mister Panic" (another personal favorite)

Luckily, Google Voice didn't deem it necessary to call me by my college nickname
"Hello squid" (though I wouldn't have minded it as much as I did back then...)

Google needs to match up its transcription to a list of all my names in my address book, and to my user name.

• It is DEFINITELY affected by cellphone quality (lower). Calls from my home phone (a high quality VOIP line) transcribe with much more accuracy than calls from my cell (which sound fuzzier, contain road noise).

• The service does not do a good job finding beginning and end of sentences. Most messages come across as a big run-on sentence. If NLP were applied to the fragment - it should be able to estimate ends of sentences, clauses, and connected words/concepts, but lacking basic punctuation the job of processing ideas is harder to do right now particularly with longer messages. You can see the message transcription and listen to the source at the top of this post.
  

At Clarabridge we HAVE actually run text transcriptions through our customer experience solution. And it works. But accuracy of categorization, and therefore precision of analysis (how well does a category or sentiment get mapped to customer calls) and recall (how many messages are recalled when analyzing a specific type of customer feedback) can suffer.

Sometimes "good" is good enough, and your mileage will vary when you try to directly connect voice calls into text analytics solutions.

By comparison, poorly formed sentences, misspellings associated with type-written feedback work fare better than auto-transcriptions of voice calls - because the NLP algorithm in the Clarabridge engine can very accurately and consistently decipher the intent even with typewritten errors. Speech transcriptions just are more erroneous, at least today, and harder to accurately decipher.

We'll keep hoping for improvements in the space -- if Google isn't there yet, likely most other vendors aren't either.

In the meantime, if you want to ingest speech content, we generally recommend using a speech to text vendor who also runs a sample of transcribed recordings through a human assisted "error correction" stage - it costs more, but it raises the accuracy of the text to the 80+% range, and at that level of accuracy it can sail through text analytics solutions and produce very high quality customer experience insights.

Twitter and all that - Part II - Customer Support vs Customer Experience Intelligence

Don't know if you've all seen this article from USA Today last week. It had some interesting insights:

http://www.usatoday.com/tech/news/2009-06-25-twitter-businesses-consumers_N.htm

Basically the article profiled some very creative ways businesses are tapping into social media to get closer to their customers. The article recounted some well known stories (ie when Tweeters learned about a power outage during a Stanley Cup Playoffs game, or about how Dell, Comcast, and others are using Twitter to respond to customer complaints or advertise special sales).

I think all these creative uses of Twitter are fascinating, and can create real value for customers (who now have a means of communicating directly to each other and to companies via a medium they prefer and are increasingly flocking to).

From my vantage point looking at Twitter as a source for Customer Experience Intelligence, I still remain convinced that it is only one data point, and perhaps not the best data point, for comprehensive, actionable intelligence, for a number of reasons:

1) The average "tweeter" posts one tweet. Ever. See this study from Harvard, reported by the BBC. http://news.bbc.co.uk/2/hi/technology/8089508.stm While Twitter growth has been explosive, basically a very few people are tweeting a very lot. And a lot of what they have to say is not particularly insightful to companies.

2) Companies ARE getting real value in monitoring tweets, and in reaching out to customers to find out what problems they are having, or what suggestions they have. Most companies can count the number of tweets that require an action in the 10s to 100s per day - not much more than a customer support rep covers in a day, and companies who monitor, are finding that the tweets from users are not bad ways to provide support, or an ear for feedback. But looked at this way - this is really just another channel for customer support. Good, but not necessarily transformational. To the extent a support rep has a conversation with a tweeter - the rep is most likely to want to archive the conversation in a CRM system so it can be analyzed alongside all the other channels of support and feedback that are being captured.

3) Call Centers are still where all the action is, and will remain for some time. From the article:

"For perspective, consider the size of call-center operations for major brands. Comcast says it is unlikely to uproot its operations, which employ 25,000 — most of them in the U.S. — in favor of Twitter. "A majority of our customers prefer to contact us by phone," Eliason says."

"Twitter is for basic troubleshooting," says Zsolt Katona, a marketing professor at the University of California-Berkeley's Haas School of Business. "Be careful not to ignore those who rely on the phone for customer support."

4) Every member of the twitter generation is also quite likely to go to review sites, fill out point of sale surveys (online, of course), and even get tech support via online forums (or even use the phone if they have to) - and the information in these sources is richer (contains detailed insight about the experiences, problems, troubleshooting, and problem resolution). These sources also tend to allow companies to "link in" information about the transaction, the reviewer or poster, etc) permitting far more correlation of the user, the product, the experience that generated the feedback, than a twitter post will ever do.


So what am I really saying? In conclusion - Twitter is a good tool for short insights from a random and small community of fervent users. I use it myself, and find it very useful for scanning trends, seeing what people are saying about products and events, and even for communicating with people who have questions about my company or product (a few months ago I even provided some tech support via Twitter to a person who had a question about Clarabridge's offerings).

But it's not a great source of actionable intelligence from a representative segment of a customer base, and it's not a great source for "text mining" insight that can be cross referenced against specific customers, specific experiences, and specific products. The better sources for rich, qualitative/unstructured high volume customer experience insights are (and will remain for some time) call center verbatims, survey feedback (structured and unstructured feedback), and web forums, web discussion groups, web review sites, and community web sites.

Twitter and all that - Customer PERCEPTION or Customer Experience Intelligence?

There's been a lot of discussion lately about the value that Twitter, Facebook, etc brings to companies looking for customer "insights" or customer "experiences" - the thought process is that if only companies could have a live feed to Twitter or Facebook data that they could keep a finger on the pulse of customer experiences, suggestions, issues, fix problems, and ultimately create a happier, more loyal, more profitable customer base.

While there's value in social media tracking, I'm going to take a contrarian position. I believe that web/social media helps identify customer "perceptions" - but it does a poor job helping companies track real customer "experiences" - and thus the social media content is not a good place to track, measure improvements, and ultimately monitor customer experiences.

Why is that, you might ask?
1) the web is largely anonymous. If a person tweets "I'm sitting in Starbucks, my latte sucks" - you don't really know enough to fix the problem. Where is the customer? What store? Who served it? Is the shopper a frequent customer? How often does he shop there? Is the problem endemic at the store or just a transitory problem? You can't determine ANY of those insights from a 'tweet.'

2) the web doesn't generally contain a 'closed loop.' If a customer complains that he/she is having a problem with a product or service, and they get some insight that helps them fix the problem, is the case "closed?" Who closed it? What was the resolution? You can't tell that from a web forum, by and large.

In short - the web does not contain 'actionable intelligence' - it only contains - 'perceived intelligence' - to get to actionability you need more information from the person, details on the problems, and a closed loop from the resolution process that identifies that issues are tracked to completion.

You can track perceived issues, but you can't really use the insights from web content to identify, fix, and ultimately measure the impact of your changes on your customers.

Far better to track the insights, interactions, and free form conversations, chats, and verbatims from:
- customer calls to a call center
- survey feedback
- online chats
- company moderated (or at least participating) forums where you can reach out to a customer directly and work with them to identify, resolve, and track issues and resolutions.

Tracking your top 10 Dissatisfaction Drivers

A few days ago Bruce Temkin, from Forrester research, dissected a report frequently deployed by Clarabridge customers called the "Negative Influence" report.

As stated in his blog: "The Clarabridge Negative Influence report correlates the negative experiences described in the open ended feedback (based on the specific categories of experiences that are described with accompanying negative sentiment) with a low score, and also weights the ranking of most negative influences based on frequency – how often a specific experience is most often associated with negative assessments."

Rather than reposting his blog - here's a link to the piece. There's some good commentary from readers following the blog that's also worth reading.

http://experiencematters.wordpress.com/2009/04/29/tracking-your-top-10-dissatisfaction-drivers/

My worst customer experience ever

I'll never forget my worst customer experience ever. I was 1 year into my first job after college, working as a management consultant for Ernst & Young. After a nice lunch with some co-workers I stopped in at a local 7-11 convenience store to pick up a pack of Reese's Peanut Butter Cups for dessert.

I got back to my desk, ripped open the package, and upon biting into the first cup noticed that the filling wasn't as creamy as I expected. The consistency of the peanut butter was mealy and granular.

Upon review of the cup, I was HORRIFIED to discover little granular maggots embedded in the half eaten peanut butter cup. Upon further review, I was further horrified to discover that the maggots were actually alive and squirming around in the cup, and that the half eaten portion I'd quickly spit out of my mouth contained yet MORE maggots!!!!

I was, you can imagine, disgusted and livid.

After rinsing my mouth with several cups of water, brushing my mouth 3 or 4 times, and chewing through a pack and a half of gum to freshen my mouth - I proceeded to write a letter to Hershey's (manufacturer of the cups), and to delicately wrap up the half eaten portion of the remaining cup. I packaged up the whole disgusting mess and sent it, FedEx, to the customer support address on the package.

Within 2 weeks I received a sincerely worded response from a nice woman in Customer Relations. She apologized for my experience, assured me that Hershey's has manufacturing, distribution, and retail quality controls and the experience I had is both rare and unacceptable.

I was sent coupons for over $20 of Hershey's products, and a box of Hershey's coffee cups, for my trouble, and was thanked for my business.

I have to admit I was impressed by the relatively rapid response (keep in mind the experience happened in the late 80s, before the advent of internet email and the world wide web) - and in spite of the disgusting nature of my experience, I was willing to give my favorite candy another chance.

I remain a loyal Reese's Peanut Butter Cups eater. But I always break open the cups before I eat them now....

Emotion and Trust and Transparency and Text Mining

Gartner just came out with a listing of the Seven Great Concerns for CEOs in 2009 - and it listed one interesting area that touches on issues of importance to the Clarablog (note highlighted area below).

CEO Issue Three: Loss of Business and Governmental Trust
The institutions that were once counted on to safeguard the economy seem to have failed, and the lack of transparency in the economic system has been exposed. There has been a subsequent loss of trust, as well, amid fears that other unknowns are awaiting. Trust is an intangible element in business but is crucial to transact business. IT can help improve transparency in the way business is done through reputation management, e-discovery and business intelligence. Gartner also expects a strengthening of "data driven" management culture as the risks of moving forward with insufficient data become far less acceptable.

They're right, of course. Everyone wants transparency. People are less tolerant of obfuscation, misleading, mishandling, and mis-marketing from the corporations they do business with.

It's in a company's best interest to "come clean" with the facts, and get messages out to the market. It's also in a company's best interest to LISTEN to customers - hear out their issues, concerns, suggestions, and even EMOTIONS.

Text Analytics, driving business intelligence, identifies, issues, emotions, sentiments, and concerns from customers and cost- and time-effectively helps raise corporate awareness of customers' concerns, perceptions, and needs.

Customer Experience Intelligence is primarily an ROI driver to help improve services and products (see earlier blog posts), but it can also serve an important role in helping companies attain and maintain public trust during times when public faith in the institutions of business and government is weakened.

- Sid

Smart Response - a Smart Way to use Text Analytics to Improve Customer Experience

Companies need to analyze customer experience feedback 3 different ways:

1) "What are the 10 things my most profitable customers are most upset about THIS MONTH?" - Answering this question requires the ability to mine through all customer feedback, segment the data by business-definable dimensions like time and profitability, and establish an easy to view "dashboard" that quickly distills the answer from millions of pieces of text-based information to a concise answer to the question.

2) "What's new with my customers TODAY?" - Answering this question requires near real-time analysis of incoming customer feedback to see what's spiking over normal levels of feedback, so that an operational manager can quickly respond to a adverse event, service problem, or other customer experience issue that's quickly developed, to avoid the issue growing into a full-fledged problem.

3) "What should I do for the customer who just complained to me about a bad experience RIGHT NOW?" Unlike the first two examples - which involve analyzing customers in aggregate or by segments or groupings, this last type of analysis is about smart determination of an individual customer's issue, and smart response to that customer in real-time. If a customer had a bad experience, and took the time to complain - smart response is about using text mining to quickly ascertain, and adjudicate the customer experience with a suggested resolution that can be communicated directly back to the customer.

The first two categories depend on text mining large volumes of customer feedback. The last category depends on the ability to use text analytics in real-time. We announced Clarabridge SmartResponse (tm) http://www.clarabridge.com/default.aspx?tabid=136&PressReleaseID=609 as a response to our client demands for the ability to harvest and analyze feedback in real time as well as respond to individual customer communications.

Our clients have been using our Content Mining Platform to answer the first two types of questions for a while now - they've recognized that it's important to keep tabs on their customers' experiences, needs, and suggestions, and realize that ongoing analysis of trends, problems and suggestions can lead to better decisions, better product offerings, and better experiences - all of which lead to happier, more loyal, more profitable customers:

• Hotels learn what products and amenities customers want, and conversely what they can cut.
• Airlines can track the preferences and needs of most valuable travelers.
• Retailers can evaluate the response to new product lines, and track feedback on product quality, safety, and passion.

Smart Response solves an entirely different type of customer challenge. If your customer is tweeting about a bad experience, you want to respond to them immediately. If they send you an email or fill out a survey to describe a particularly bad flight experience, or stay at a hotel, or treatment at a store, you want to quickly assess the problem, determine the best possible response, and communicate back to the customer with thanks for the feedback, a sincere apology, and if warranted an offer of compensation for their troubles. Smart response is about real-time mining of feedback, real-time assessment of the problem, and real-time determination of possible responses for the customer.

Customers want not just to be heard, but to be addressed. Smart Response truly closes the customer experience loop with the customer.

RE: In2Clouds theory about text analytics and Government Transparency

MM in his blog In2Clouds referenced my recent blog on the business benefits and ROI of Text Analytics in the commercial domain, and suggested that it might be useful to be able to use Clarabridge to mash up government information with constituent feedback, suggestions, complaints about programs, producing better transparency and accountability.

I agree - It would be VERY INTERESTING -- and in fact solutions such as MM is envisioning will become reality in 2009, I'm sure.

I see a solution that incorporates:

1) Databases containing funding information ($ granted, per program, over time, over geography) coming from Federal Agencies
2) Databases containing spending information ($ spent, per region/state/jurisdiction, over time) coming from the state agencies
3) Performance Management information (job growth, roads created, educational metrics, energy capacity improvements) coming from a variety of public, private, and watchdog sources.
4) Constituency feedback (anecdotes from citizens on program qualitative benefits, outcomes, problems, observed fraud/waste/abuse) coming from a variety of social media sites on federal, state, local and third party sites)

Put all that information together, and you have truly comprehensive view of the life cycle of the government recovery/stimulus effort, and you have a living, breathing, always on, actionable view of the stimulus in action (or not, depending).

These kinds of data fusion/mash up solution visions highlight the real potential of integrating unstructured data containing qualitative feedback with structured data containing dollars and cents and performance metrics, to truly track the what, where, and WHY of complex programs and systems, and use the information to track, analyze, and improve programs.

In short - text analytics are not just for business. Government can and should also get into the game.

Achieving ROI with Customer Experience/Text Analytics Solutions

Welcome to 2009 – we’re in a recession.

Perhaps you are
-a retailer
-a travel/hospitality company
-a consumer goods company
-a high tech products company (hardware or software)
-a financial services company
-a media company

In short, you’re like one of the many companies that have decided to use customer experience intelligence solutions, like the Clarabridge Content Management Platform, to establish a text analytics-based voice of the customer initiative.

Why, you might ask, should you be looking at such an initiative in 2009 – when companies supposed to be cutting costs, getting efficient, and doing whatever they can to stay profitable, solvent, and alive? Is customer experience a “nice to have,” or a “need to have” solution?

Anecdotal insight from existing Clarabridge customers would suggest that it is very much a “need to have” solution. Text analytics, applied to customer experience management, is an important way to weatherproof your company against the economic storm we’re currently experiencing. A few big reasons:

1) ONLY THE EFFICIENT SURVIVE – and Customer Experience Analytics solutions create immediate efficiency. Most businesses collect and process customer feedback (survey content, call center content, email content) manually, and spend too much money, and too much time manually reading, coding, and analyzing content, particularly unstructured, text-based content. One of our customers reduced a staff function from 25 analysts to fewer than 5 after adopting Clarabridge, and increased its ability interpret feedback from a fraction of feedback to 100% of feedback. Another customer cut market research spending year over year by over 25%, more than covering the first 2 years of cost for the solution, and is now getting actionable feedback from customers within 24 hours of capture, down from process that used to take 30 days from capture to analysis.

2) ALERT DRIVEN INSIGHTS CATCH PROBLEMS BEFORE THEY COST YOU MONEY. Text analytics solutions do a great job of tracking experiences quickly, and quantitative metrics can measure the magnitude of a new problem before the problem causes cost, customer satisfaction, and resource impacts to your business. A leading software manufacturer uses alert driven analytics from Clarabridge to catch the “fast-growing” issues associated with new software releases so they can quickly kill bugs and errors BEFORE they affect the entire customer base – averting millions of dollars of support costs.

3) USE A SCALPEL, NOT A CHAINSAW TO CUT COSTS WHILE RETAINING DESIRABLE CUSTOMERS. Many customer-oriented companies engage in a technique called market segmentation to identify and analyze the buying patterns and preferences of discrete segments of their customer base. Before cutting a product or service expense, our clients use Clarabridge to assess how valuable the product or service is perceived to be. Others run trial cost cutting programs and evaluate the impact of the cost cutting on their “desirable” customers – the most loyal, the most profitable, and the highest net worth – to make sure they’re not engaging in an action that will drive away the desirable segment. Hotels are using Clarabridge to assess which amenities can be cut without adversely affecting business travelers. Retailers are assessing which product lines can be cut without impacting frequent customer loyalty.

4) SAVE BIG BUCKS WITH DATA-DRIVEN, INSIGHTFUL BIG DECISIONS. Over time, companies inevitably have a moment when they have to make a BIG decision. Perhaps it’s to roll out an expensive new product. Or kill a high support cost product. Or tear down a property that seems unfixable in some way. Before making the decision that can impact the company by millions of dollars, our customers use Clarabridge to validate the decision. One customer considered whether to tear down or modernize an old property due to customer complaints about noise levels and the age of the building. Using Clarabridge they were able to determine quickly that customer feedback from the property was not statistically more or less negative than feedback from newer properties, and the company subsequently decided NOT to tear down the property, averting millions of dollars of unnecessary expense.

In short – return on investments (ROI) can be clearly tied to operational, analytical and strategic use of Customer Experience Analytics, and solutions from companies like Clarabridge don’t just generate cost savings in the short term, they allow better decisions to preserve loyalty in the long term – preserving customer relationships, loyalty, and corporate profitability

Text Analytics / Customer Experience Intelligence 2009 - Predictions

I was recently asked what I "see as the 3 most important text-analytics technology, solution, or market challenges in 2009?"

Good question. 2008 has been an expansionary year for the text analytics space along many dimensions - expanding appreciation of the business value of text analytics, expanding deployments across corporate enterprises, and a general expansion and evolution of the market segment away from a 'tech' sell to more of a solution sell.

2008 also seemed to validate that text analytics, and specifically text analytics applied to voice of the customer analytics, (tapping into and transforming customer feedback and experience content into rich actionable analytics) are largely "recession-proof" if sold and deployed effectively.

My company, Clarabridge, is having a great year - we continue to grow, achieve a fantastic level of deployment success, and continue to add value to customers in many ways:

- we help customers do more analysis, with less cost
- we let our customers spot problems more quickly than before (within a day or two, vs within a week or month), so
- our customers can take action on customer feedback and opinion quickly and decisively (i.e. if they spot a service or product issue they can address it and fix it quickly and decisively)

As a result of the time and cost efficiencies inherent in text analytics-based customer experience solutions, we enable decisions and customer experience improvements far more cost effectively than manual reading, coding, analysis and decision making would allow.

Partly due to the successes of our customers, and willingness of our customers to share their success stories of cost savings and decision making efficiencies with other customers, partners, and prospects, we've continued to grow our customer base -- even in a slowing economy this year demand for our customer experience/text analytics solutions remained strong and grew solidly.

2008 proved that customer experience-based text analytics solutions are needed in both good times and bad - in good times firms want to maintain their reputation, fix issues quickly, and outcompete their rivals, and in bad times firms want to do all they can to be responsive to their customers to maintain their loyalty and profitability.

I think 2009 will largely see a further evolution of the space as we've seen in 2008, a bit more 'mainstreaming' of the solution, and the development of interesting partnerships and technology integrations between vendors and solution providers.

Specifically, then, my 3 predictions:

1) Text Analytics solutions will expand from a functional/departmental initiative to an enterprise imperative. In 2006-2008 - text analytics was deployed to enable the analysis of marketing, call center, and survey/market research departments, and the results of the text analysis were used primarily within a single department.

In 2008 we saw at Clarabridge a few of our more progressive customers looking to evolve and expand from departmental approaches to an approach that supports multiple organizations. We see a few prospects (who will likely become customers in 2009) who actually want, on DAY ONE of deployment to have a solution that can that meets the needs of many groups and departments in an enterprise.

As a result of having a cross-functional, cross organizational goal, I expect in 2009 that for a class of "enterprise buyer" selling cycles will become more complex, the solutions will need to show more a priori business value and relevance to many stakeholders, and because the solutions are not just deployed in one area, IT requirements and architecture criteria will become more important as organizations will want to know more how the solution is going to fit with "enterprise" standards.

As a result of this trend - in 2009 selling and deploying will become more complex, more constituencies will become involved in the decision, and at the same time articulating and demonstrating business value will be even more important to more business stakeholders.

2) Text Analytics will evolve from being an isolated to an integrated solution. Towards the end of 2008 we started seeing more interest from companies and partners looking to seamlessly integrate the results of text analytics back into operational systems. -- i.e. they wanted to process customer verbatims and merge the categorized, scored results back into call center applications, or moderated web forums.

This new requirement - not just for text analytics, but for operational integration of text insights and scores into downstream applications and solutions, provides an interesting opportunity for text analytics vendors to consider - whether to be a standalone application, or to be a more tightly integrated with partner products and offerings. I think the progressive text analytics companies will be wise to look for strategic partners with whom to integrate their solutions.

3) Text Analytics solutions in the customer experience space will quickly get bigger and faster. 2008 taught Clarabridge that to succeed, we needed to support ever bigger data sets, with ever faster processing requirements, and as a result we quickly learned to create our solution for scale and growth. Companies with small volumes of text data are keeping their history for perspective and trending. Companies with large customer bases and diverse lines of business are creating huge data sets - with 10s to 100s of millions of customer interactions being processed through the Clarabridge text analytics engines. Data volumes will continue to grow in 2009 and usage requirements will also go up as more users seek to explore, analyze and act on the insights in customer experience solutions.

There's no reason to expect scalability requirements won't continue to grow in 2009 and beyond. Successful vendors will be those who can see beyond today's data and user volumes and design for an order or two more magnitude in their offerings.

As long as vendors grow with the market and deliver value and efficiency to customers, they will succeed as this market evolves. Clarabridge will be one of those successful vendors.

- Sid Banerjee
CEO, Clarabridge

Insight by Design: Another pitch for text analytics gone awry

Insight by Design: Another pitch for text analytics gone awry

New Study: "Exploring the Link Between Customer Care and Brand Reputation in the Age of Social Media "

Just came across press releases on this study, including this one:

http://sncr.org/?p=110

http://www.brandweek.com/bw/news/recent_display.jsp?vnu_content_id=1003793178

A few salient quotes from a Brandweek article on the study:

"Forget focus groups. Consumers are giving it straight to brands, and each other, via online social media in big numbers, according to a recent study by the Society for New Communications Research, Palo Alto, Calif."

...The study found that 74% of respondents choose companies or brands based on customer service experiences shared by other Web users on the Internet. Eighty-one percent of those polled said they believe blogs, online rating systems and discussion forums give consumers “a greater voice” in customer service. However, only 33% of respondents felt that companies take customers’ opinions seriously."

...Of those industries judged to be doing the best job in using social media to respond to customer service issues, technology, retail and travel companies took top honors. Dell and Amazon were noted most often as those companies doing the best job handling customer care problems via social media.

Utilities, healthcare and insurance firms fared the worst. "

Well - it's good to see that others are seeing what we're seeing at Clarabridge. In no particular order - my thoughts:

1) desirable, savvy, computer-literate, affluent customers value the customer experience, and don't want to waste time with companies that waste their time with sub-par experiences

2) those customers are not shy about making their opinions known on, and offline.

3) When they get upset - the words, the emotions, and the passions are captured in all kinds of places. In the notes in a customer support interaction. In the passionate responses of a customer survey (increasingly if not exclusively on-line surveys nowadays). And of course (as this study notes) in the forums, and groups online where like minded consumers congregate.

If you're one of those companies in travel, hospitality retail, and technology, you're probably interested to learn more. Not surprisingly, when you look at the customer rosters for firms like Clarabridge - they are full of those kinds of companies who want to learn more about customer feedback, and use the feedback to improve their customers experiences. Customers like Gaylord Hotels, Marriott, Gap Stores, United Airlines, Intuit, H&R Block, etc. They're using text mining platforms like Clarabridge's CMP to crawl web content, or text mine survey and call center verbatim content, and they're tapping into customer feedback, learning how they fare, fixing problems, benchmarking against their competitors. In short - they're GETTING IT.

As a consumer of health care, utility, and insurance firms, I'm not surprised that they're not. I'm also not surprised because I mostly don't see many of those companies using customer experience analytics solutions very much.

It's not enough to recognize that customers are posting their rants, passions, and feedback on the Internet and in surveys and in their conversations with companies. Companies need to:

- Connect to, and Collect the content from wherever it is.
- Mine and Refine the content, to translate text (in all its clumsy, wordy verbosity) into ideas, concepts, experiences, and sentiments that can be quantified,
- Analyze and Discover themes, issues, problems, opportunities, and passions of customers about the products and services they consume.

It's all about Customer Experience Intelligence. It's all about what Clarabridge does. It's not a fad. It's a trend.

---Sid.

Text Mining and Customer Experience - the blog I intend to write...

Since this is my first entry, I'll use this first posting to organize my thoughts and think about what I want to write over the next few days and weeks.

A bit of background:

Who am I - I'm a technology executive with more than a few years working in and around the business intelligence, data warehousing, and more recently the text mining and customer experience management and analysis space. A few years ago I started a text analytics company that's really taken off in 2007. I decided to start the blog to have a place to document my thoughts on the business we're in, the creative ways people are starting to analyze the terabytes and petabytes of unstructured customer-originated data that are increasingly available to business decision makers, and to speculate on issues that are at the intersection of business, information technology, customer experience, marketing, maybe politics, and the pace of change in all of the above...

Over the next few days and weeks ideas I want to write down include:

1) Why is there a market for text analytics, and specifically the business application subset that my company's involved in, which we call Customer Experience Management and Analysis. How did it evolve to be a viable space for a company like the one I'm working for?

2) What the heck should we call the space in 2008? Should the company's space be oriented around a technology (such as text mining, text analytics)? Or should we define our business so that it's more aligned to a specific class of business problems that it solves (i.e. customer experience management and analysis)? A good label for a company should support a nice acronym - should the acronym factor into the naming of the space?

3) What are the benefits our customers get from our solutions? Specifically,
- why do they buy our stuff
- what are the initial benefits they get when they deploy our solutions?
- what's the long term benefit of using our solutions? How do you define ROI up front, and more importantly with sustained use over time? Are our solutions "transformative?" How?

4) When you are building a solution that is grounded in technology components that are packaged and developed into a business solution, (as our company's offerings are), how much do you build, vs, license, vs partner with others in the surrounding ecosystem that supports your business area? What parts of your solution are critical, and differentiating, vs what parts are best accessed via partners? Why did we make some of the technology development decisions that we made when we built our platform? How do we expect our technology to evolve over time?

5) A brief history of Text Mining applied to customer market research -- How did the space evolve? What preceded Customer Experience Management and Analysis? What was the ancient history of the space? I have a view that text mining is a logical evolution of technology and analysis that dates back to the era of mass production, leading through the eras of mass distribution, mass retailing, and now mass consumption, and that as every era has developed over the past century, the market has required "instrumentation" - and that text analytics is the latest instrumentation that is now being applied to business (more on this as I write this post).

6) as an adjunct to posting topic #1, - is the market for text analytics and customer experience management and alalysis "wide open" or are there barriers to new entrants that are making it harder to just enter into the space if you aren't already in the thick of it? What are the critical business and technology factors that any potential entrant needs to consider before diving into this space?

7) Case Studies, Case Studies, Case Studies. If customers won't go on record (if they do you can read about it on the corporate web site), what are the interesting stories that have emerged from our deployments over the past couple of years? Changing the names and perhaps industries to protect the unauthorized, what real insights have customers gleaned through mining through customer experience data? How have they reduced the time, and cost intensity of predecessor approaches to seeking out, examining, and acting on customer experience insights? What "quick wins" have they found? Why are our customers continuing to use our solutions year after year?

8) Case Studies/Lessons learned. What SHOULDN'T Text analytics be used for? What have we learned through trials, tribulations, and failed efforts, and what did we need to do to turn lemons into lemonade? How can you avoid making mistakes by learning from our experiences?

9) What is the Customer Experience Maturity Model, and where do you sit on it? Why is it important to move up the curve from customer experience "novice" to custome experience "guru?"

10) Political Experience Management and Analysis - what can we learn from the presidential campaigns about mining customer sentiment, experience, and using it to sharpen your "customer experiences" and relationships?

11) Our Partner Ecosystem - what is it now? What do I want it to be? How can you be a partner of our company if you're a prospect, consultant, marketing services company, systems integrator, data provider, or just an interested party?

If I can get all these topics written, I should have a resonable blog. If anybody has additional ideas for posts, feel free to comment.

Happy 2008. Challenge me....