Back in 1950 Alan Turing wrote a paper entitled "Computing Machinery and Intelligence" in which he proposed a test (which has been since named the "Turing Test") to determine if an artificial intelligence had been successfully created. The basic idea is that if an artificial intelligence is created such that person A can have a conversation (via typed text) with persons B and C, one of whom is another human sitting at a keyboard and one of whom is a computer program, such that A is not able to tell which of the two entities he's talking to is a human and which is a computer, then this would be evidence that the computer is an artificial intelligence to the extent that it can functionally behave like a human.
Of course, it's a bit flattering to think that the way to determine if a machine is a "thinking machine" is by seeing if it can interact with us humans in a social setting. On the flip side, some levels of human interaction are so low level that one can imagine a fairly simple compluter program succeeding at them pretty well. For instance, I would imagine you could probaby be moderately successful in creating a computer program that could present a reasonable facsimile of human communication via Facebook statuses, links and comments. But that's in part because communication on Facebook is pretty simplistic and you can always not respond or throw out non sequitors without seeming like anything other than a fairly normal Facebook user.
Building a computer program capable of producing a passable simulation of a more wide-ranging conversation is, however, a lot trickier, and such a feat is clearly some ways off in the future.
Thinking about this, however, in relation to the question of whole brain emulation which I wrote about last week it occurs to me that there are two questions here:
1) Is it possible to create a computer program which could fool a human into thinking that he is having a conversation with another human.
2) Is it possible to create a computer program which is actually capable of having a conversation for its own sake.
The first of these is achievable if one can build a good enough set of algorithms around how humans respond to questions, common knowledge for conversational grist, etc. The second, however, is much trickier. When we converse with someone we generally do so because we want to communicate something to that person and/or we want to know something about that person. In other words, conversation is essentially relational. You want to know about the person you are talking with, and you want them to know about you. You want to establish areas of common interest, experience and belief. You want to bring that person to share certain ideas about you or about the world.
I'm not sure that it would ever be possible to build a computer program which had these feelings and desires. Oh, sure, you could give it a basic like/not-like function where it tries to achieve commonalities or confidences and if it is rebuffed puts its interlocutor in the "not like" category and relates to it differently. But this is very different from actually wanting to know about someone and wanting to like and be liked by them. How our own human emotions work in this regard is far from clear to us, so I can't imagine that we're in any prosition to understand them so well that we can create copies in others.
Of course, in real conversation you often can't tell if the other person you're talking with, even face to face, is actually interested in you or actually likes you. This question is a source of considerable concern in human interactions. So perhaps the question of whether a computer can care about who you are or what you talk about is irrelevant to the question of whether a computer could be designed that could pass the Turing Test. But I do think that the question is probably quite relevant to whether a computer could ever be a person.