Game of Thrones in Data: The Character Connections Drawn by Tweeters
By Gemma JoyceMay 17
Published February 15th 2019
“Part of me thought that debating is an incredible challenge for AI, and that I should be able to beat it,” says Harish, as we discuss his pre-debate nerves.
“But another part of me thought surely before the game of chess against Deep Blue every single grand master would have said a chess machine cannot compete with Gary Kasparov.”
Harish is about to take on the intimidating IBM Project Debater machine, a project that’s long been in the making. The machine has an intimate understanding of him – in fact, it has modelled his voice from hours of recordings. But, aside from a brief glimpse in rehearsal the previous day, Harish’s first interaction with the machine will be on stage in front of a live audience and many expectant online viewers.
“They didn’t want me seeing it or interacting with it before. The first time I’d seen the machine in any form was the rehearsal the day before the main debate. Even then I didn’t see it debate, I just heard it’s voice and saw the way it looked. So there was some degree of trepidation on the day of the debate itself,” he explains.
Some degree of trepidation seems like an understatement to me, but I’m not a world-renowned debating champion with an undergraduate degree from Oxford and a postgraduate degree from Cambridge. Nor do I have the title of Winner of the European Debating Championship, or Grand Finalist in the World Universities Debating Championships under my belt.
If anyone is feeling trepidation in this interview, it is me.
Harish and I are chatting just after his return to London after his triumphant debate against IBM’s Project Debater.
The event has been covered extensively by the press, I imagine because it has quelled some anxieties around whether machines are able to beat humans at literally everything.
That said, it’s not like IBM’s machine, which Harish describes as a “technological marvel”, totally failed during the competition. In fact, the differences between debating the machine and debating a human competitor weren’t enormous for Harish.
“After the first minute of getting used to the shock of it not being a human being, or the surprise of what that actually meant, it became much like debating against a human being.
“Once you’re on that stage and you’re hearing arguments, it doesn’t really matter if an argument is coming from a machine or from a human. You treat it the same way, which is: This is a claim which is made and substantiated, how do I want to deal with it?”
Meanwhile, there were plenty of things he was impressed by. Both he and IBM’s machine were given the motion at the same time – 15 minutes before the debate. The computer was able to summon evidence and examples from a vast database, while Harish had to rely on his own knowledge and rhetorical skills.
“What the machine is better at than any human could ever be is finding relevant evidence – studies, examples, cases – and get that context. Another thing I think is very impressive was not only its ability to present evidence, but also its ability to explain why it mattered in the context of the debate.”
Given Harish’s debating credentials, I’m keen to hear how he ranks this win among his many triumphs.
“I think as an achievement it’s obviously been really cool, the media coverage has been intense, and certainly it was a high pressure event. In terms of the actual debate itself, it was good but it doesn’t quite compare to beating the very best competitors worldwide.”
IBM Project Debater may not have had Harish shaking in his polished debating shoes, but he’s got some great notes for how it might improve.
“While I think it picked up a lot on what I was saying and had some reasonable responses to it, it struggled responding to the more subtle claims which I made. It may be in part that it didn’t understand them given the reality of natural language, or it might just be that it isn’t quite ready to respond to complicated arguments. That would probably be the area it needs to improve the most.”
Meanwhile, IBM Project Debater’s delivery could do with some tweaking. It was able to use some pre-programmed humour, but being able to play with the mood of the room could help it become more convincing, he says.
“In terms of the arguments, it is very much on the right path with just a few changes needed, with the exception that it needs to do a little better at response. On delivery, it is currently not as good as a very good human being, and there is potential that this could improve dramatically over the next few years.”
The most convincing speakers are those that can make you feel as well as think. As mentioned above, this could be what IBM’s Debater is lacking when it comes to winning around an audience.
Of course, it doesn’t need to have emotions to evoke them.
“It started talking about human emotions which in some ways was the most interesting part about it. There was a section of the speech when it talked about the experiences of individuals who are poor and how that creates an obligation on us,” Harish explains. “Obviously it doesn’t have those emotions.”
He gives the example of political speakers who are able to evoke strong emotions within crowds of people, but aren’t necessarily feeling them themselves.
“It has a lot of the tools and is in the right direction, but particularly when it comes to response and the way it plays with the emotions of an audience and tries to convince them through ways other than logic still requires a little more if it were to be able to beat the best humans.”
Harish admitted he is far more impressed than afraid at this stage.
“Unlike Go, where you need to get more area than your opponent, or chess, where you need to checkmate the king, the goal in debating is to be more convincing than your opponent. There aren’t simple rules to tell you whether or not you’ve met that. I think it’s quite impressive that it’s able to debate reasonably well at all based on the fact that there aren’t those clear goals.”
Harish says that when you break down what would be needed for IBM Project Debater to win into constituent parts, there’s nothing to suggest it couldn’t beat a human champion.
“I imagine in even two years it will be very competitive and definitely in 10 years if the project continues with the focus it’s had recently it will be very, very difficult to beat. But that’s purely speculative on my part.”
When discussing this debate, and AI generally, it’s hard not to address more wide-reaching implications. Harish is excited about what the technology could be used for, and gives the example of his own work. (He might be a champion debater, but he also has a day job.)
Harish is currently head of economic risk at AKE International, a consultancy that helps organizations with issues around political and economic risk. He’ll spend a day researching something in depth but, he says, he won’t be able to read a fraction of what IBM’s computer would be able to process in 30 seconds.
He’s convinced that AI will be able to help with human decision-making in the near future.
“There are so many points of decision-making constantly, and we often make decisions based on a lack of credible information, biased information, but almost always incomplete information. With the ability of AI to find what is relevant and to explain it to us, to understand and answer the questions we ask it, there are very few fields where that will not be transformative for individuals to make good decisions.”
My last question to Harish feels a little personal. I want to know how embarrassing it’ll be if he’s ever beaten by IBM Project Debater, or a machine like it.
“Is it that at that debate I wasn’t particularly good or is it that this machine has worked out how to do this activity better than anyone else can? If it’s the latter, after the initial shock and the ‘damnit, I lost to the machine’, there will be an understanding that there are certain things machines can do better than human beings.”
Championship-level debating is probably one of the most difficult things human minds can go through. We don’t know when machines will be able to do it better, but just like the great chess and Go champions before us, we probably won’t expect it.
Thanks so much to Harish for sharing his experience with us. You can find him on LinkedIn here.