I’ve been busy lately, combined with a long commute and it means I’m not really watching Jeopardy! these days. So, I missed seeing this whole Watson hoopla.
But, I did catch this op-ed from CNN:
There is no doubt the IBM supercomputer Watson gave an impressive performance on “Jeopardy!” this week. But I was worried by the computer’s biggest fluff Tuesday night. In answer to the question about naming a U.S. city whose first airport is named after a World War II hero and its second after a World War II battle, it gave Toronto, Ontario…
Both the humans on the program knew the correct answer: Chicago. Even a famously geographically challenged person like me (I come from the UK and know there is something called the Midwest beyond Chicago) knew the answer.
Why did I know it? Because I have spent enough time stranded at O’Hare to have visited the monument to Butch O’Hare in the terminal. Watson, who has not, came up with the wrong answer. This reveals precisely what Watson lacks — embodiment.[full article]
I’d change “embodiment” to “context” and we’d be at the problem faced by the semantic web (the sort of AI searching magics touted by Watson’s masters at IBM). All merit due to the Watson-box for it’s win, but it’s still essentially a search engine.
The ability to retrieve and process static information, then buzz in at beyond human speeds is novel. Still, if you put Watson on a line between Google and the average human, it’s not far beyond Google.
People do love these John Henry type battles. Man vs. progress. It’s easy to see it as a clear win for Watson, given the strict rule structure of Jeopardy. Back in the day, Deep Blue did well at chess, but that’s an even more strict rule based environment.
Computers will always do well where the playing field is resonably and consistently structured.
And, Watson’s shown that he’s not good at guessing below a certain confidence threhold, and I guantantee that confidence goes down the less rules there are. Intuition is a hard thing to fake and this is what drives a lot of people’s information seeking behaviour.
Anyways, I think the greater challenge will be to see a computer win at Family Feud.
