Interview: Harish Natarajan on Beating IBM Project Debater and Restoring Everyone’s Faith in Humanity…For Now

“A technological marvel”

Harish and I are chatting just after his return to London after his triumphant debate against IBM’s Project Debater.

The event has been covered extensively by the press, I imagine because it has quelled some anxieties around whether machines are able to beat humans at literally everything.

That said, it’s not like IBM’s machine, which Harish describes as a “technological marvel”, totally failed during the competition. In fact, the differences between debating the machine and debating a human competitor weren’t enormous for Harish.

“After the first minute of getting used to the shock of it not being a human being, or the surprise of what that actually meant, it became much like debating against a human being.

“Once you’re on that stage and you’re hearing arguments, it doesn’t really matter if an argument is coming from a machine or from a human. You treat it the same way, which is: This is a claim which is made and substantiated, how do I want to deal with it?”

Meanwhile, there were plenty of things he was impressed by. Both he and IBM’s machine were given the motion at the same time – 15 minutes before the debate. The computer was able to summon evidence and examples from a vast database, while Harish had to rely on his own knowledge and rhetorical skills.

“What the machine is better at than any human could ever be is finding relevant evidence – studies, examples, cases – and get that context. Another thing I think is very impressive was not only its ability to present evidence, but also its ability to explain why it mattered in the context of the debate.”

“It doesn’t quite compare”

Given Harish’s debating credentials, I’m keen to hear how he ranks this win among his many triumphs.

“I think as an achievement it’s obviously been really cool, the media coverage has been intense, and certainly it was a high pressure event. In terms of the actual debate itself, it was good but it doesn’t quite compare to beating the very best competitors worldwide.”

IBM Project Debater may not have had Harish shaking in his polished debating shoes, but he’s got some great notes for how it might improve.

“While I think it picked up a lot on what I was saying and had some reasonable responses to it, it struggled responding to the more subtle claims which I made. It may be in part that it didn’t understand them given the reality of natural language, or it might just be that it isn’t quite ready to respond to complicated arguments. That would probably be the area it needs to improve the most.”

Meanwhile, IBM Project Debater’s delivery could do with some tweaking. It was able to use some pre-programmed humour, but being able to play with the mood of the room could help it become more convincing, he says.

“In terms of the arguments, it is very much on the right path with just a few changes needed, with the exception that it needs to do a little better at response. On delivery, it is currently not as good as a very good human being, and there is potential that this could improve dramatically over the next few years.”

Playing with emotions

The most convincing speakers are those that can make you feel as well as think. As mentioned above, this could be what IBM’s  Debater is lacking when it comes to winning around an audience.

Of course, it doesn’t need to have emotions to evoke them.

“It started talking about human emotions which in some ways was the most interesting part about it. There was a section of the speech when it talked about the experiences of individuals who are poor and how that creates an obligation on us,” Harish explains. “Obviously it doesn’t have those emotions.”

He gives the example of political speakers who are able to evoke strong emotions within crowds of people, but aren’t necessarily feeling them themselves.

“It has a lot of the tools and is in the right direction, but particularly when it comes to response and the way it plays with the emotions of an audience and tries to convince them through ways other than logic still requires a little more if it were to be able to beat the best humans.”