Two birds with one stone?
I was recently contacted by two clients to discuss how we could help them use a computer to read the text from their incoming customer feedback.
These requests are particularly interesting because, despite the two clients’ resulting solutions being incredibly similar in terms of the technical requirement, their problems were actually very different.
The first client is an Own Brand Quality Manager and wanted to save time by not having to read the verbatim of call centre complaints for products.
The second is an Own Brand Product Developer whom wanted to read through hundreds of consumer panel tests and understand the themes that were prevalent and if/how trends would emerge.
Where the first client’s issue was predominantly operational the second’s was far more strategic. The solution, however, was equally as valuable to each customer.
What my text analysis journey can teach you
Because of this dichotomy, I spent a great deal of time investigating text analytics and trying to understand the application of text analysis to the two problems.
The output from my pilgrimage through the world of lexicons, stemming and sentiment is as follows.
There are 3 broad ways to analyse and understand text using a machine, each has its strengths and weaknesses, and all are applicable for the right situation.
Rule based analysis (or triggers):
The simplest of these to understand is trigger words. Trigger words examine the text and apply a set of rules. For example, the rules may apply a label of “packaging” whenever it sees the word “box”, “carton” or “bottle”.
Rules are good for:
- Focussing on words that have been flagged as important.
- Grouping linked terms into buckets (box, packet, bottle -> Packaging).
Rules struggle with:
- Multiple meanings and complex logic.
- Categorising records into actionable buckets
(e.g. “zip lock” - is it: won’t open? won’t close? leaks? broken?).
Natural Language Processing (NLP):
By far the most prevalent “advanced technique” applied to customer feedback, NLP evolved from a set of rules designed to understand the keywords and sentiment of the phrases as spoken or written naturally.
The technique typically involves using complex statistical models inside of a machine learning environment. Thankfully, there are multiple “out-the-box” NLP solutions that allow users to simply send hundreds of pieces of text out and receive the key words and sentiment back in return.
NLP is good for:
- Condensing a lot of text into a simple visualisation.
- Seeing generic themes in a block of text
(provided it is context/already sufficiently categorized or filtered).
NLP struggles with:
- Understanding context around the use of words inside a phrase
(e.g. “didn’t look right, but tasted fine” – misattribute taste?).
- Associating conjugated words
(e.g. taste, tastes, tasted, tasting…).
Finally, the 3rd option currently available is the next evolution in the field of machine linguistics.
AI machines can be trained by an expert human to understand context, experience and nuance. This is particularly useful in the world of grocery where a lot of the language is not “natural”.I.e. it is collected by third parties in the form of notes and/or is layered with context in the form of products and the various synonyms that can be used to describe both good and bad experiences. E.g. crunchy carrots are a good thing, crunchy humous, not so much.
AI Engines are good for:
- Understanding context and experience within text and assigning into actionable buckets.
- Interpreting related phrases and groups of words in a similar and consistent way, consistently over time, channels and categories.
- Recognising all key themes and building up trends over time outside of the most frequent (to allow KPI trend and analysis).
- Categorization of not only subject but context and intent.
AI Engines struggle with:
- Very granular identification (not enough examples to learn from).
- Unsupervised operation (will always need a human in the loop to manage the exceptions; constant tuning to understand things it’s never seen before).
Returning to our Own Brand Quality Manager and Product Developer: the conclusions were that a combination of NLP Keywords and Rule based trigger words were the best option to quickly identify the underlying issues, whereas the AI engine was the best bet to consistently understand the consumer panels and help identify the themes.