On the Tough Question of Contradictory Data in Market Research Methodologies

Posing some tough questions to our Research Services Project Manager

Our Research Services team Project Manager Evelyn Castillo is no stranger to in-depth, demographic specific (or not), market research projects. She shared her perspective as to why social listening could be contradictory to other research and was generous enough to divulge some intriguing and useful examples in several different industries.

There may be many reasons and examples of why social research insights could be contradictory to other research. Here are two key ones that come to mind.

Traditional research methods, for example focus groups, provide a limited sample.

When you broaden your research to a wider audience via social, you’re not necessarily getting contradictory data but rather you’re getting new data.

In a focus group you’re asking specific, pre-determined questions, which can limit the responders’ answers. You’re setting them up to answer one way or another. The goal is to avoid that in focus groups, but alas bias creeps in despite all efforts to the contrary. You introduce bias simply by being in the room during a focus group.

Social data might seem contradictory, but it’s a new view on what the data might be and can take you down roads and surface topics you didn’t expect to encounter when you embark on a research project. It reveals new areas you’re not usually exposed to with traditional research in the form of surveys or focus groups. Social research allows you to select a very general topic without having to determine the parameters of how they may be discussing that topic – be it product, industry, trend, or even a specific spokesperson. Social then does the work of gathering and showing you what everyone is saying on the topic which brings up many different ways they’re discussing it.

Traditional research or syndicated data is inherently outdated.

We use syndicated data such as that from MRI, Simmons or Nielsen, as truth because it’s what we readily have access to in our research teams. And in fact, traditional and syndicated data is a good source of data to extract insights, but we have to keep in mind that by the time it gets into our research mix and our collective systems, it can be a year old. A lot changes and quickly in our world, especially given the socially-driven consumer environment.

One thing I’ve discovered is that social is not a solution to this problem of stale data, but another way of looking at it. When you find something contradictory by comparing social to other research, it’s an impetus to dig deeper. It sparks curiosity and questions about why the discrepancy exists to begin with and what other factors may be in play.

Applying social in the real world (of market research)

I asked Evelyn if she could share with us some more tangible examples to help bring to life how social data isn’t necessarily contradictory, but revelatory for market research teams.

For one alcohol brand I was working with, we found through syndicated data that younger people were not drinking as much beer and the consumption volume had decreased.

We had initially based our insights on syndicated data which limits the options to select to respond. Frankly, this type of research is always a little bit behind the trend. With social listening you have the luxury of simply searching for the term “beer” and seeing what’s trending, how people are discussing this topic, what other tangential topics and phrases might be coming through in this conversation.

So why was consumption of “beer” down? It’s possible that it wasn’t down. Social introduced the possibility that it’s not that they were drinking less beer, it’s that the brands and types of beer they are drinking are changing.

When you’re doing social research you also can’t see how much they’re consuming – that’s a sales data question. But we were able to see on social that they were still talking about drinking beer. The brands in the discussion were simply changing as newcomers in the beer manufacturing industry prevailed in social conversations.

We could use social to pinpoint specific demographics discussing beer, and then more specifically craft beer. These demographics could be as general as simply looking at “women” or as specific as Hispanic millennial women under 30, or men in finance under 35 in the 10 most populated cities in the U.S.

Social data gives you the liberty to broaden the questions that you ask from your research and gives you the ability to intake the data in a new and fast way.

The important thing when you find contradictory data is to determine the why.

So what are the key differences between the two forms of research? Using both sets combinedly to inform your research allows you to understand why one set of research has a specific outcome. In the alcohol brand’s case, you can learn from one piece of research that the volume is decreasing and then use social insights to determine the why.

Here’s another example from the cosmetics industry.

Let’s say a cosmetic brand is interested in learning how African Americans use cosmetics.

If you were to begin your research by looking at sales data you could assume the use of cosmetics by this demographic is less than the general market if sales volumes are low. However, if you were to broaden the scope of your research to social, you would learn that this specific demographic discusses using cosmetics extensively online. But then why are sales low? Where is the discrepancy between sales data and social discussion?

In a particular research project I worked on to dig deeper into this topic, syndicated research didn’t tell us what social revealed, that in fact limitations of color matching was a major issue this demographic encounters. And they were taking to the web to discuss. Unless a brand specifically asks a question about color matching limitations in a traditional research survey or focus group, they would not hear about this product limitation that was affecting sales for a large demographic.

When you see people are talking about makeup on social media, but the numbers are saying they’re not consuming this CPG (Consumer Product Good) product, you then have a starting point to address this through product R&D.

What you’re likely to find through social research is that this specific demographic of consumers is discussing the limitations of the product online, and simultaneously praising the brands and products that they are in fact using to fulfill their product needs by offering a wider range of shades to match skin tones.

All of these findings hold valuable insights. These insights can be the catalyst to product feature change (such as offering a wider variety of skin tone matching options in various cosmetics) increasing sales, building customer loyalty, and addressing a consumer need that will elevate your brand in the eyes of the consumer.