Let’s Debate: Traditional Market Research versus Social Insights

Will McInnes opened up a debate at the Now You Know Conference back in May with a thought that really resonated with me.

“With social data you’ve got two apparent hemispheres, a yin and a yang – a paradox. One is messy and unstructured; one is hygienic and linear, known and reliable.

In our industry, in the world that we live in, I’d say there’s a very high level of healthy skepticism about the reliability of social data. And yet, at the same time, we have industries underpinned by – when you scratch the surface – very flawed methods of sampling.”

We wanted to explore this idea of traditional market research versus insights from social, and so we assembled smart domain area experts to discuss, on stage, in front of hundreds of conference-goers.

Christel Quek, our VP of Asia Pacific hosted the conversation and was joined by Jim Tobin, President of Ignite Social Media, and VP Customer Experience Strategy and Insight at Metia, Misia Tramp.

“I started a company called Ignite Social Media in 2007,” explains Jim.

“We’re a social media marketing agency, and we typically work with large household name brands in technology, CPG, retail.

We map cases of services. We run Facebook pages, do blogger outreach, do the analytics of all of our programs, we do the paid media buying for social, sort of all those things with a specialty on social media particularly.”

Misia comes from a market research background, as she explains.

“I started out life as a traditional researcher looking at insight and innovation around customer experience, and then over the years as digital data started becoming fashionable, started using that for ethnographic purposes.

My business got acquired in 2009 by software companies, and since then I’ve been working on the intersection of technology and insight, as well as strategy.”

Without further ado, let’s get to the debate.

Up now: debate panel w/ @jtobin & @misiatramp re: skeptical market research v. optimistic social. #NYKCONF

— candacecorner (@candacecorner) May 11, 2016

Is traditional market research still relevant?

CHRISTEL: We’re going to start off with a little curveball here. Why do you think traditional market research is still relevant today?

JIM: Well, I think it depends on what you’re trying to do and how much you have invested in doing that, and whether traditional market research makes sense. 

When I was doing focus groups back in 2000, we’d budgeted about $5,000 per group – that’s pretty expensive stuff. When I worked for the State of Michigan we were in the midst of changing the slogan for the State from “Say Yes To Michigan” to something new.

All the research suggested that people in Illinois and Indiana and Ohio came to Michigan for the Great Lakes and for the family time.

And so we thought that a great slogan would be ‘The Great Life’, playing on the Great Lakes. Fortunately, we focus group tested that before we spent millions of dollars rolling it out and advertising it – because the reaction was almost universally anti that slogan.

People said, ‘Wait, you’re saying I don’t have a great life in Chicago? I don’t have a great life in Cleveland?’ to which we said ‘No’, and so we ended up with the slogan Great Lakes, Great Times, which resonated much, much better. 

Now, in the social space, I do almost no market research like that. If you’re going to invest millions of dollars in something and you haven’t sort of vetted it, I think you’re rolling the dice.

MISIA: You see, I get quite cross when people say what role traditional market research has because I would argue it all is. 

You know, I mean social data is just simply the world’s best ethnography. I think one of the problems that people have in adopting social data is the fact that they distinguish between this idea of what is traditional versus social, and I think that actually impinges its adoption. It makes it sound like social’s something different, like it’s a dataset not to be trusted. It means it has different rule sets.

Where we’ve been successful is just talking about the research mix and referring to data, be that linguistic, numeric or visual. That’s all it is. So, I actually don’t like it when people talk about this idea of traditional versus social, because fundamentally as a researcher we have to be data-agnostic.

If you really want to get this three-sixty degree view of the customer that everyone goes on about all the time, you have to understand them through many lenses. And that’s all it is. 

If you demystify it and if you refer to it just as another data source that’s linguistic, just like an open-ended question or a focus group transcript that is liable to the same set of rules, I find the objections disappear. 

JIM: But is it the same set of rules? We’re supposed to debate, so here we go. 

Hillary and Bernie had their first debate and two phone surveys found 55% and 56% that Hilary won that debate. Slade ran an online survey and that found 77% said Bernie won the debate and only 17% said Hilary won the debate. 

13% said Jim Webb won the debate – anyone remember Jim Webb? He was the guy who seemed to be paralyzed on the stage. Anyway, so there is a potential for selection bias when you look at social data that at least you try to control for with traditional.

MISIA: And how do you do that in traditional?

JIM: Screens.

MISIA: Yes, but you’re still screening against a group – you have a base to begin with and you could argue that people who were sitting on a panel were being paid to participate in the surveys. We understand what those biases are. And the rules are no different with social data. 

Actually, I would just argue that one of the things that I find very frustrating is that the poor research design that gets applied to social data. I mean, I think it’s fabulous that tools like Brandwatch are making it easy for us to access the data, but there’s an argument that says, ‘Well, actually, if it’s that easy, some of the basic social rules go flying out the window’. 

We create questionnaires based on measuring things we want to understand, or how well we’re doing things to people, or eliciting an opinion about a thing that we care very deeply about, and we’re not gaining the context around those questions, which means that ultimately the research ends up being very unpredictive.

I would argue the same thing about social. We’ve got one client who is in the life insurance space, and obviously, people aren’t online going on and on about their life insurance policies, right, or getting very excited about consuming it when they’re dead. 

But what people are talking about are their lifestyles, their concerns, and their needs around their families. So what is that higher order problem you’re trying to solve? So this idea that social has a selection bias, I totally disagree with. It’s about the question you’re asking rather than what the bias is.