What's Wrong with the Turing Test?

If I could answer one question through the course of my research it would be "what is intelligence?" This question like no other drives my studies. I wrote this post a while ago but did not post it. I did not post it because I intended to refine it. But the reality is I will always be refining my thoughts on this topic. Tonight I went out to the pub with several of my colleagues at Université de Montréal and this topic came up reminding me that I need to just put this out there. As such I am posting it now, with some small changes. I look forward to your responses.

The question "what is intelligence?" is non-trivial. We have been seeking an answer for millennia. While many definitions have been offered [1] no single definition has really ever dominated. Even in the sixty or so years that we have been seriously studying how to create an artificial intelligence we have never actually formally defined intelligence. Ask 100 people to define it and you will likely receive 100 different definitions [1], [2]. The de facto standard is the Turing test developed by Alan Turing [3]. History tells us that he was worried about getting mired in a long drawn out philosophical debate which would likely prevent any progress on actually creating an artificially intelligent being.

The Turing Test as it has come to be known is a variation on a game known as the imitation game [2] wherein a participant, the interrogator, attempts to identify which of the other two participants, one male and one female, was in fact male. Of course the female's objective was to fool the interrogator. The crux of the game was that the decision had to be made solely from communication patterns. In other words, the interrogator did not meet nor could they see the other participants. Additionally, to avoid cues from writing styles, messages would be passed between them in an anonymous way, such as through a computer terminal.

In the Turing Test the objective is to identify the artificial being [4], the computer, as opposed to the male. The hypothesis being that if the interrogator cannot differentiate the artificial being and the human, the artificial being must necessarily be intelligent if we accept that humans are intelligent. This test is clever because it does not require a definition of intelligence nor a measure of intelligence other than agreement that humans are intelligent. However the Turing Test is a behavioral test. Not everyone accepts the test. One of the more well known opponents is John Searle, a professor of philosophy at the University of California, Berkeley. Dr. Searle offers the Chinese Room argument in counter to the Turing Test.

According to the Chinese Room argument we can construct a process that appears intelligent but in fact is not. We do so by placing a person in a room, particularly one that does not speak Chinese. The resident will receive messages from outside the room written in Chinese. The resident must then consult a set of instructions that, when followed, dictate the appropriate response. One, that to any outside observer, would have to have been written by someone that knows Chinese. Since the resident does not know Chinese it supposedly follows that intelligence had only been imitated by the process.

There are a number of counter arguments to the Chinese Room argument. Some argue that the analogy breaks down as a result of the fact that a human performing the processing would simply take too long. Others argue that the room itself is intelligent. But I digress.

While I don't personally accept the Chinese Room argument I do agree there is a flaw in the Turing Test. Specifically, by the nature of its construction, it will permit us to classify a being as intelligent if it behaves like a human. From this we have to conclude that everything else is either not intelligent or at least not classifiable as intelligent without some added criterion.

This not only applies to the animals but to all other beings. Consider the scenario wherein we are visited by aliens that can talk to us, that can do incredibly complicated mathematics, even teach us a few things, and possess technologies way beyond our understanding such as a clearly engineered means of interstellar travel which they used to come to Earth. Would we consider these beings intelligent? We can all think of scenarios wherein the answer is "not necessarily" but in all likelihood we would agree that they are in fact intelligent. But how likely is it that the Turing test will apply?

Of course this problem applies to artificial beings as well, i.e. our computer programs. Have we already created an artificial intelligence? Some might argue we have with Cleverbot garnering 59.3% positive votes from 1,334 participants at the Techniche 2011 festival. Others would likely respond that the real turing test involves physical interaction, i.e. shaking hands with the being, and still not being able to discern a difference. This again highlights the problem.

A precise definition of intelligence would address this problem. However it would not only allow us to differentiate between intelligent and not and answer the question of whether we have already created an artificial intelligence. But it could allow for the development of metrics for comparing intelligences and even help us understand why our existing creations are not intelligent, if that is truly the case.

[1] Legg & Hutter, 2006, A Collection of Definitions of Intelligence, http://www.vetta.org/documents/A-Collection-of-Definitions-of-Intelligence.pdf
[2] Pfeifer, 1999, Understanding Intelligence
[3] Turing, 1950, Computing Machinery and Intelligence http://www.csee.umbc.edu/courses/471/papers/turing.pdf
[4] Russel and Norvig, 2009, Artificial Intelligence: A Modern Approach Third Edition

No comments:

Post a Comment