Understanding Sentiment Analysis: What It Is & Why It’s Used Research

Research By Kristian Bannister on January 26th 2015

Sentiment analysis – otherwise known as opinion mining – is a much bandied about but often misunderstood term.

In essence, it is the process of determining the emotional tone behind a series of words, used to gain an understanding of the the attitudes, opinions and emotions expressed within an online mention.

Understanding Sentiment Analysis

Sentiment analysis uses

Sentiment analysis is extremely useful in social media monitoring as it allows us to gain an overview of the wider public opinion behind certain topics. Social media monitoring tools like Brandwatch Analytics make that process quicker and easier than ever before, thanks to real-time monitoring capabilities.

The applications of sentiment analysis are broad and powerful. The ability to extract insights from social data is a practice that is being widely adopted by organisations across the world.

Shifts in sentiment on social media have been shown to correlate with shifts in the stock market.

The Obama administration used sentiment analysis to gauge public opinion to policy announcements and campaign messages ahead of 2012 presidential election.

The ability to quickly understand consumer attitudes and react accordingly is something that Expedia Canada took advantage of when they noticed that there was a steady increase in negative feedback to the music used in one of their television adverts.

Understanding Sentiment: Tweet

Sentiment analysis conducted by the brand revealed that the music played on the commercial had become incredibly irritating after multiple airings, and consumers were flocking to social media to vent their frustrations.

A couple of weeks after the advert first aired, over half of online conversation about the campaign was negative.

Rather than chalking up the advert as a failure, Expedia was able to address the negative sentiment in a playful and self-knowing way by airing a new version of the advert which featured the offending violin being smashed.

Contextual understanding and tone

But that is not to say that sentiment analysis is a perfect science at all.

The human language is complex. Teaching a machine to analyse the various grammatical nuances, cultural variations, slang and misspellings that occur in online mentions is a difficult process. Teaching a machine to understand how context can affect tone is even more difficult.

Social Media Sentiment Analysis

Humans are fairly intuitive when it comes to interpreting the tone of a piece of writing.

Consider the following sentence: “My flight’s been delayed. Brilliant!”

Most humans would be able to quickly interpret that the person was being sarcastic. We know that for most people having a delayed flight is not a good experience (unless there’s a free bar as recompense involved). By applying this contextual understanding to the sentence, we can easily identify the sentiment as negative.

Without contextual understanding, a machine looking at the sentence above might see the word “brilliant” and categorise it as positive.

How we do sentiment analysis at Brandwatch

Brandwatch's Sentiment Analysis

Remember the scene in Terminator 2, when a young Jon Connor teaches the T-800 hip 90s phrases like “no problemo”, “eat me” and “hasta la vista, baby”?

That’s not entirely dissimilar to how a linguist expert would teach a machine how to conduct basic sentiment analysis.

As language evolves, the dictionary that machines use to comprehend sentiment will continue to expand.

With the use of social media, language is evolving faster than ever before. 140 character limits, the need to be succinct and other prevailing memes have transformed the ways we talk to each other online. This of course brings with it many challenges.

At Brandwatch, we employ a rules-based process to help our software better understand the ways context can affect sentiment.

We take all the words and phrases that imply positive or negative sentiment and apply rules that consider how context might affect the tone of the content. Carefully crafted rules help our software know the first sentence below is positive and the second is negative.

“I want a burrito so bad”

“I just had a burrito. It was so bad.”

The caveats of sentiment analysis

The above examples show how sentiment analysis has its limitations and is not to be used as a 100% accurate marker.

As with any automated process, it is prone to error and often needs a human eye to watch over it. At Brandwatch, we give users the opportunity to redefine sentiment if they believe that it has been wrongfully categorised.

Sentiment monitoring

Beyond reliability, it’s important to acknowledge that human’s expression doesn’t fit into just three buckets; not all sentiment can be categorised as simply as positive, negative or neutral.

Predictions for the future of sentiment analysis

While it’s difficult to speculate how a relatively immature system might evolve in the the future, there is a general assumption that sentiment analysis needs to move beyond a one-dimensional positive to negative scale.

The Future of Sentiment Analysis

In the same way that politics cannot always be reduced to a position on a left to right scale, there are other kinds of sentiment that cannot be placed on a simple barometer.

For the future, to truly understand and capture the broad range of emotions that humans express as written word, we need a more sophisticated multidimensional scale.

Can you measure skepticism, hope, anxiety, excitement or lack thereof? Until this happens, sentiment analysis is (literally) one-dimensional!

Organisations will certainly become more aware of the applications of sentiment analysis within their marketplace, fueling the growth of sector specific services and technology delivering sentiment specific use cases – for example, intelligence tools that aid decision-making for financial traders and analysts.

We will see a shift in perception of the reliability of sentiment analysis. Users will become more comfortable with the idea that the automatic analysis of individual text material is hard to match human performance.

The insight that can be gained from large datasets (millions of Tweets) will overshadow the concerns about reliability at a granular level (a single Tweet).

Instead, the focus will be on how to make results interpretable and actionable. In the meantime, we’ll be ensuring we are working at making sentiment analysis as accurate and easy to understand as possible.

Advanced Sentiment Analysis

Find meaning in the conversations that matter.

Find out more

Kristian Bannister

Kristian is a Digital Strategist here at Brandwatch. He helps our marketing and website teams find best opportunities for growth. He writes about using social data to support marketing, market research, content strategy and SEO.

  • ITGuy

    What emotions would be the ones to be included in a multi-emotional system? As in what emotions form the building blocks of expression?

  • Kristian Bannister

    Sorry for the late reply. The next logical step would be in addition to positive and negative (or ‘valence’), there is a spectrum of ‘arousal’.

    While boredom and annoyance are both negative, they are different: https://en.wikipedia.org/wiki/Emotional_granularity#mediaviewer/File:Valence-Arousal_Circumplex.jpg

    However this may even fall short as there are some emotions that are positive and indicative of a high level of arousal but also very different, such as ecstasy, amazement and admiration.

    The closet thing I can think of to a multi-emotional scale is the Plutchik-wheel: http://en.wikipedia.org/wiki/Contrasting_and_categorization_of_emotions#mediaviewer/File:Plutchik-wheel.svg

  • Game On

    I have also create some algo to check sentiments in social media that i have implement in this tool


  • Sudha Sompura

    How is research in cross-domain sentiment analysis?
    And, what are some recent developments in the field of sentiment analysis?

  • Nasir Mahmood

    In the modern era of technology world has become a small village. Social media has played a vital role in shaping the social norms and values in an innovative way. It has become most popular mode of communication, people uses various social sites like YouTube, Twitter, LinkedIn and Facebook for information sharing. They use these sites to get connected with world. Social networks has emerged a new source of information sharing around the globe. This network is so smart that it is available to everyone round the clock on their PCs, laptops and on mobile phones so people can access everything any time when they want. Every day when you get up you see news and stories of friends and of the people you follow on social media. News agencies and broadcasters actively update their profiles, blogs and news feeds on social media so that people get instant updates about what is happening around them. People can discuss social and political issues while sitting apart by using this facility. It has great influence on politics, political workers realize the importance of social media and they effectively use it.
    Social media also provide great opportunity to traders and to businessmen. They have enormous opportunities to sell their goods and products online. It has provided speedy growth to business, as people can share and sell products worldwide. It has provided considerable large prospects to customers sitting worldwide. Due to its fastest communication service people more relay on social media for information sharing.
    Sentiment analysis identifies emotions, feelings and opinion of a person or group of people, by analyzing written statement of that person or group. After identifying it categorizes these sates of emotions of subjected population. It is most powerful tool to classify the attitude of writer and polarity of text. It finds the hidden emotional state of a person who has written the text.
    Sentiment analysis basically finds the polarity of a document or paragraph which is written by someone. After finding polarity it identifies the positivity or negativity of that particular scenario in which that statement was written and which has to be analyzed. This analysis can be used in various fields of life like politics, judging the reliability of product, and in analyzing the users on social media like twitter. Many companies use micro blogging to asses about the public opinion about their products. As sentiment analysis is one of the most accurate and widely used technique around the world. So it helps companies to produce more reliable products to their customers.

  • Nasir Mahmood

    Scientists have used different linguistic structures, to discover public opinion on multiple social sites and in daily life. In this research, author used multiple features to carry out experiments. This research has used n-grams, lexicon, parts of speech and micro blogs. These features identifies the unigrams, bigrams and polarity to count respective parts of speech. [3] In experimentation two folds were created, in first fold data was trained while in second, validation of features was checked. Then they used Sentiment analysis to discover the confident ideology of a specific group of people. [4]
    Many researchers used movie reviews to classify the emotions that is either it is positive or negative. [5] This research has discovered sentiment analysis which is widely used practice in social media, which recognize the hidden emotions in a written paragraph. Sentiment analysis identifies the hidden opinion and secret emotional background of a paragraph written on social site like twitter. This technique identifies the majority’s opinion. This analysis is widely used while making governmental and political policies. It is used to test the opinion of a certain population about a specific policy. [6]
    Turkish language is among the most morphological languages. A very limited work has been done on this language regarding morphology. So to apply sentiment analysis on Turkish language is a challenging task. [7] Twitter is one of the most advanced and widely used social media platform. Status or massage posted by a twitter user gets spread to his/her followers, it’s also been used for politics. To perform sentiment analysis on Turkish social media a sample of tweets was collected. These tweets were than mapped on news which were circulating on social media, than their sentiment analysis was carried out. This was the basic methodology which was adopted. Sentiment cataloguing shows that it is very complex and problematic than customary classification which was adopted in the past. [8]
    Turkish websites promote their brands in a very innovative way, they used sentiment analysis to promote their brands. This polarity detection research has mainly focused on brands on different websites in Turkey. Different filters were used to carry out the analysis. In this research punctuation and exclamation sign was removed as they don’t add value. To improve vocabulary they used lexicon and open domain solution as it was open source library. They also removed the Turkish as these words were causing duplication. Comments from different local movies were used to develop a systematic approach to detect the polarity. Tweets were collected and opinion finder was used to classify these tweets, then, type of tweet was tabulated, that it was objective tweets or it was subjective tweet. In order to carry out sentiment analysis they used Support Vector Machine (SVM). The use of SVM resulted an advantage that it was the best model among all others. [9]

  • Nasir Mahmood

    Sentiment analysis has been studied in National Football League and in English Premier League. The researcher predicted the outcome by making categories with some specified conditions. Than the researcher used statistical and sentimental analysis. A system of control support was designed, then by using this system tweets were collected and classified. After examining a conditional format was used to make decision. Best $2704 and odds only $1887 conditions were used but this was less accurate as payout was $3011. Then it was found that positive sentiment and cultural expectance dominated as difference in goals was 0.42 for positive sentiment while it was 0.90 goals for negative sentiment. By the analysis of researcher it was found that positive tweets created a supremacy which shows that twitter was accurate. [10]
    Twitter lexicon was the main focus of this research work was. [11] This was developed and was further used in sentiment analysis. To carryout different requirements n-gram was used to in order to reduce the features. This research of “brand sentiment analysis of twitter” proved that with less number of lexicon and features, increased the efficiency of sentiment. The SVM was used to carry out the comparison. This comparison exposed that new lexicon give good results than traditional old version. Then again a comparison was carried out between DAN2 and new lexicon. This analysis provided that DAN2 is far better than SVM with same new lexicon. [12]
    Naïve Bayes methodology to distinguish between different tweets was the main focus of author in this research. Two specific word were selected for a team of 27 trainers. Then they were asked to classify the tweets, than results were drawn by using Naive Bayes methodology. For Naïve Bayes analysis total 50 tweets were selected, among these 50 tweets two groups were made. One group was for training and one for classification. After classifying by trainer and Naïve Bayes, sentimental validation was run. A very good result was originated that was 90% ± 14% accurate. [13]
    Sentiment analysis of FIFA World Cup was carried out during football matches in 2014. In this “sentimental analysis” twitter API has been used to cry out the research. Research on different matches was carried out to examine the feelings of football fans. After carrying out analysis we found that US fans’ emotions were changing with time and with the change in situation of the matches. [14] To investigate the emotions of US fans, all the tweets posted within US were analyzed. Among all the tweets, 1007 tweets of first match, 1295 of the second and 2135 of third match were from US. After collection and tabulation of distinct emotions was made, then Natural Language Toolkit (NLTK) processed the data. [15] During post analysis NRC lexicon and lexicon word frequency was used to evaluate the basic emotions of spectators during matches. At the end it was evaluated that US fans were positively emotional when their team was scoring the goals and were aggressive when opponent team was scoring goals. This analysis also showed that reaction of US supporters was full of joy when other teams were playing and was scoring goals. At the end of this research it was concluded that spectators expressed a valid predictive response. [16]

    After related work we concluded that Sentiment analysis has been used most of the areas where we have to analyze the emotions, feelings and opinion. It has been used in social media, sports, marketing, business, news, in blogging and other fields of life. After doing related work we found that we should model a hybrid system which could perform sentimental analysis on Turkish news. In this model integration of different features and categorization of different sentiment will be done. As Turkish language is one of the most morphological languages so we have to develop a new model to analyze Turkish news.

  • Gabriel Ruiz

    Great article on sentiment analysis. As a marketing student just beginning to study this kind of stuff I’m far away from using it in a professional sense, but I certainly enjoy learning about the nitty gritty details that go into successful marketing ( or correcting it in the case of expedia). I have to do sentiment analysis for my own school project so thank you for the basics!

  • mj

    can u please tell which api u used for making this software

  • Game On

    i have used twitter api, youtube api, and reddit api.

  • JR

    May thanks for your comments here, Nasir.

    Where can I read more about sentiment analysis with regards to it proving great opportunity to businessmen, producers (product creators) to sell their goods and products online? I find this simply fascinating.

    Also, is there a list of specialized applications (perhaps additional examples) where sentiment analysis is finding success?