The Feature Our Happiest Customers Love Most
By Gemma JoyceFeb 28
Social media provides a treasure trove of insights for brands looking to understand consumers.
But due to the vast amount of social media data, powerful analysis is required to uncover insights. In addition to image analysis, one of the primary methods of uncovering insights from social media data is text analysis.
Text analytics is the process of deriving information from text sources (Gartner). Text analysis can be applied to any text-based dataset, including social media, surveys, forum posts, support tickets, call transcripts, and more.
Computers have historically had trouble understanding natural human language due to its nuance, subjectivity and idiosyncrasies. But new technology and techniques have greatly increased the accuracy of text analytics.
While humans are still better at understanding language, the vast amount of text data makes automated analysis solutions particularly useful for processing data at scale.
Text analysis can be applied to social media data to answer a wide variety of questions about consumers, brands, products or any other topic.
Here are some of the top use cases:
Understand the general sentiment or specific emotions expressed about a brand, product, or topic.
Use text analysis to understand what percentage of a conversation is about a particular brand, product, or topic.
Drill down within any conversation to understand what driving it and how the content of the conversation has changed over time.
Identify intent to purchase and any other stages of the consumer buying cycle that your brand cares about.
Want to measure something that’s specific to your brand or product? Machine learning-based text analysis allows you to create your own categories and train a platform to categorize social posts accordingly.
Now that you know what you can do with text analytics, let’s look at the two primary approaches to it.
There are two main approaches to text analytics:
Each method has specific strengths and weaknesses, depending on your analysis goals. Choosing the right approach for your use case is important in order to maximize efficiency and the relevance value of the insights.
Rule-based pattern matching can be based on simple boolean keywords or more complex models compiled over time by language experts. The linguistic rules can range from identifying parts of speech, syntax, and inflections to rules about different topics, regions, and stylistic variations. This rule-based method can be quickly applied to a set of documents for fast analysis.
The analysis runs quickly (after the rules have been created).
Mistakes are easy to spot
Easy to understand where rules are successful and where they return irrelevant data.
Text can be broken into smaller chunks for analysis.
Results closely match expectations
Rules-based analysis will find what you’re looking for, but often serves to reinforce initial assumptions instead of challenging them with a broader perspective.
There are always exceptions to rules
Language is variable, constantly changing, and often informal. It is impossible for rules to account for all the ways meaning can be expressed in text. Text analysis based on linguistic rules often misses information that is relevant due to the rigidity of the rules.
Building complex rules can take years
Complex rules based on expert knowledge sometimes require years of research to compile the necessary resources to perform the analysis.
Detailed development for each language
Certain languages that have not been widely studied may not be easily analyzed before extensive research on the unique features of the grammar and vocabulary.
Rules are created by humans with inherent biases, and will only match patterns which were expected to be found. Discovering trends and new ways of expressing ideas is hampered by the reliance on static resources.
Machine learning-based analysis discovers patterns naturally from text examples. Using statistical methods, documents are compared to one another to determine the most important and useful patterns in the corpus for the desired behavior.
Machine learning analysis methods are diverse and can range from simple to complex, but they all share the same fundamental goal of learning the most valuable and distinctive patterns based on examples provided by a human.
Train with examples
Requires less complex linguistic resources, but learns patterns that are useful for the task under consideration.
Customizable and adjustable
Models can be altered and adjusted to adapt to new conditions that weren’t anticipated.
Machine learning models capture important context missed by rules based approaches because they rely on applying patterns using probability and statistics.
Machine learning models reveal changes in the way ideas are expressed that human experts would not have expected.
Analyze any language
Analyzing a new language requires less linguistic expertise because research and development requires fewer custom resources.
Must provide training data
Machine learning requires extensive training, but that training allows for more relevant insights.
Slight decrease in precision
The lack of strict rules leads to a slight dip in precision as a trade off for uncovering more hidden insights. Uncover more contextual insights in the conversation.
Document length affects approach
Analyzing short documents (like Tweets) versus long documents (like blogs), requires different considerations and approaches.
Both types of text analysis have their strengths and weaknesses. Ultimately, having the flexibility to switch between linguistic rules or machine learning models depending on the goals or your analysis will provide the best results.
See how text analytics can be applied to social media in our US Consumer Trends Report.