Interview: Carnegie Mellon Professor Ari Lightman On How Students Are Empowered By Learning To Use Brandwatch Consumer Research
By Kara FinnertyJun 10
Published July 13th 2016
Will McInnes opened up a debate at the Now You Know Conference back in May with a thought that really resonated with me.
“With social data you’ve got two apparent hemispheres, a yin and a yang – a paradox. One is messy and unstructured; one is hygienic and linear, known and reliable.
In our industry, in the world that we live in, I’d say there’s a very high level of healthy skepticism about the reliability of social data. And yet, at the same time, we have industries underpinned by – when you scratch the surface – very flawed methods of sampling.”
We wanted to explore this idea of traditional market research versus insights from social, and so we assembled smart domain area experts to discuss, on stage, in front of hundreds of conference-goers.
Christel Quek, our VP of Asia Pacific hosted the conversation and was joined by Jim Tobin, President of Ignite Social Media, and VP Customer Experience Strategy and Insight at Metia, Misia Tramp.
“I started a company called Ignite Social Media in 2007,” explains Jim.
“We’re a social media marketing agency, and we typically work with large household name brands in technology, CPG, retail.
We map cases of services. We run Facebook pages, do blogger outreach, do the analytics of all of our programs, we do the paid media buying for social, sort of all those things with a specialty on social media particularly.”
Misia comes from a market research background, as she explains.
“I started out life as a traditional researcher looking at insight and innovation around customer experience, and then over the years as digital data started becoming fashionable, started using that for ethnographic purposes.
My business got acquired in 2009 by software companies, and since then I’ve been working on the intersection of technology and insight, as well as strategy.”
Without further ado, let’s get to the debate.
— candacecorner (@candacecorner) May 11, 2016
CHRISTEL: We’re going to start off with a little curveball here. Why do you think traditional market research is still relevant today?
JIM: Well, I think it depends on what you’re trying to do and how much you have invested in doing that, and whether traditional market research makes sense.
When I was doing focus groups back in 2000, we’d budgeted about $5,000 per group – that’s pretty expensive stuff. When I worked for the State of Michigan we were in the midst of changing the slogan for the State from “Say Yes To Michigan” to something new.
All the research suggested that people in Illinois and Indiana and Ohio came to Michigan for the Great Lakes and for the family time.
And so we thought that a great slogan would be ‘The Great Life’, playing on the Great Lakes. Fortunately, we focus group tested that before we spent millions of dollars rolling it out and advertising it – because the reaction was almost universally anti that slogan.
People said, ‘Wait, you’re saying I don’t have a great life in Chicago? I don’t have a great life in Cleveland?’ to which we said ‘No’, and so we ended up with the slogan Great Lakes, Great Times, which resonated much, much better.
Now, in the social space, I do almost no market research like that. If you’re going to invest millions of dollars in something and you haven’t sort of vetted it, I think you’re rolling the dice.
MISIA: You see, I get quite cross when people say what role traditional market research has because I would argue it all is.
You know, I mean social data is just simply the world’s best ethnography. I think one of the problems that people have in adopting social data is the fact that they distinguish between this idea of what is traditional versus social, and I think that actually impinges its adoption. It makes it sound like social’s something different, like it’s a dataset not to be trusted. It means it has different rule sets.
Where we’ve been successful is just talking about the research mix and referring to data, be that linguistic, numeric or visual. That’s all it is. So, I actually don’t like it when people talk about this idea of traditional versus social, because fundamentally as a researcher we have to be data-agnostic.
If you really want to get this three-sixty degree view of the customer that everyone goes on about all the time, you have to understand them through many lenses. And that’s all it is.
If you demystify it and if you refer to it just as another data source that’s linguistic, just like an open-ended question or a focus group transcript that is liable to the same set of rules, I find the objections disappear.
JIM: But is it the same set of rules? We’re supposed to debate, so here we go.
Hillary and Bernie had their first debate and two phone surveys found 55% and 56% that Hilary won that debate. Slade ran an online survey and that found 77% said Bernie won the debate and only 17% said Hilary won the debate.
13% said Jim Webb won the debate – anyone remember Jim Webb? He was the guy who seemed to be paralyzed on the stage. Anyway, so there is a potential for selection bias when you look at social data that at least you try to control for with traditional.
MISIA: And how do you do that in traditional?
MISIA: Yes, but you’re still screening against a group – you have a base to begin with and you could argue that people who were sitting on a panel were being paid to participate in the surveys. We understand what those biases are. And the rules are no different with social data.
Actually, I would just argue that one of the things that I find very frustrating is that the poor research design that gets applied to social data. I mean, I think it’s fabulous that tools like Brandwatch are making it easy for us to access the data, but there’s an argument that says, ‘Well, actually, if it’s that easy, some of the basic social rules go flying out the window’.
We create questionnaires based on measuring things we want to understand, or how well we’re doing things to people, or eliciting an opinion about a thing that we care very deeply about, and we’re not gaining the context around those questions, which means that ultimately the research ends up being very unpredictive.
I would argue the same thing about social. We’ve got one client who is in the life insurance space, and obviously, people aren’t online going on and on about their life insurance policies, right, or getting very excited about consuming it when they’re dead.
But what people are talking about are their lifestyles, their concerns, and their needs around their families. So what is that higher order problem you’re trying to solve? So this idea that social has a selection bias, I totally disagree with. It’s about the question you’re asking rather than what the bias is.
CHRISTEL: So the next question is, why use insights from social data?
JIM: I think volume, cost, speed. If it’s well designed and you’re using it to ask the right questions, then you can do very well.
MISIA: One of the things we do a lot of, and it still stuns me that people don’t really, is socially connect your research. If you’re dealing with a very cynical set of stakeholders who’ve been tracking NPS for 35,000 years and are unwilling to lose their tracking metrics, socially connect it. Ask your customers at the end of the survey if you could get their Twitter handle or and find out if they really are promoting you.
We did this for Microsoft. We had a survey, we ran a segmentation, we asked for their Twitter handles, we asked them, ‘Were you recommending Microsoft over competitors?’
‘Of course, absolutely every time I possibly can. That’s what I’m recommending.’ You socially connect the survey to what they’re publishing and you find they’re completely lying, and that actually the net promoter score was overestimated by something like 350%.
CHRISTEL: Oh, my.
MISIA: In terms of the social data set, what it’s good for, what it’s not good for, we don’t even talk about social data anymore.
We talk about linguistic data, and everyone thinks it’s much cleverer as well when you say that. But it’s true. And then you start mixing in open-endeds, focus groups, ethnographies, and reviews if that’s relevant, in-product feedback.
Suddenly, when you demystify it and you just bring it down to data, people stop thinking about it as this sort of wild west that’s being run by a bunch of crazy millennials who are overcharging them left, right and center.
CHRISTEL: That’s true. Moving on to the current election. There’s a lot of different polling methods for TV advertising and there are a lot of different ways to try and figure out is Bernie winning, is Hilary winning. So what do you think, in your view, is one critical way this market can be disrupted?
JIM: We create 100, 150, 200 pieces of content around a brand and a product, put it out on the social web, see which 15, 20 pieces do really well and then syndicate it.
It’s driving sales over and over again which is kind of stunning to me, frankly, but as we grow and as we see this we’re kind of stunned that over here there’s still $10 billion being spent on freestanding inserts in the Sunday newspapers.
There is nothing out there in social video. Facebook Live is the next thing, and Periscope…for ten minutes Meerkat was the next thing, you know.
MISIA: We’re now in a convergence-based culture where consumers and audiences are demanding the means of production, rather than being in spectator culture where they’re passive recipients of the messages that we’re sending to them.
I would argue that our metrics are still stuck in a spectator-based culture, a vanity-based culture. For me it’s being focused on outcomes, not outputs, and making sure that you’re predicting the right metric.
Rather than this sort of inquisitorial asking questions that are around favorability or looking at numbers of eyeballs, we’re finding those metrics predict commercial behaviors less and less and less over time.
That’s what matters. It’s the outcome. The measures that go into that are actually irrelevant.
CHRISTEL: Misia, I’d like to ask you this. In your opinion, what do you think is a really outstanding client outcome that you’ve seen actually from all of this that we’ve talked about so far?
MISIA: Well obviously revenue is always one that counts the most, isn’t it? I think one of my biggest personal successes with clients is when I can actually get them all in the room speaking the same language.
People are always going to naturally marry in their tribes, right? So a successful outcome for me is to at least get everyone using the same platform of data, whether it’s the same segmentation, whether it’s the same social metrics – data can be that common language.
Another is people being comfortable self-serving around linguistic data and actually using the real voice of the customer. Using the voice that’s sitting right in front of us within decision making.
One other thing that we use social data to do a lot is customer journey mapping. We’ve been pretty successful at doing that and finding out where those friction points are that are causing people to stop the buying journey. Being able to say, ‘By the way, your buying journey is failing, 25% of the time,’ for me that’s a critical success in terms of making it truly actionable.
It’s in the language of the customer, people know exactly what to do with it.
JIM: So if we can’t do sales, I’d look to drive conversation about brands. I know particularly in a room of data scientists that has the potential to sound superficial.
But if you think about the basis of marketing and the basis of advertising, it is to generate impressions, favorable impressions in front of those with the propensity to buy. And if I can do that and if social naturally segments itself by interest group and I can get quality content around those interests, I should be doing the same thing that advertising has done for decades.
CHRISTEL: What is really exciting for you in the near future in terms of the intersection of social data, traditional research?
MISIA: Visual data. That’s one area I think we’ve just not even scratched the surface with yet. But really for me, I’m excited to see what we learn when we just put all these data sets, wire them together, and see what decisions and innovations we can drive from there. It’s as simple as that for me.
JIM: I hope we get to the point where we can share helpful content at the point where somebody needs it. My fear is that the technology to share content at the buying experience will accelerate beyond the content production, and so what we’ll share will be crap. Everyone will shut off their phones before we get to the good stuff.
CHRISTEL: Like how you just kind of mute all your irritating notifications on your phone, right?
CHRISTEL: Thank you very much Jim and Misia. Really nice to talk to you guys.
Thanks to Jim and Misia for taking part.
We’ll be sharing more insights from this year’s Now You Know Conference in the coming days and weeks. Keep us bookmarked, and follow us on Twitter: @Brandwatch.