Hidden interlocutor misidentification in practical Turing Tests

Based on insufficient evidence, and inadequate research, Floridi and his students report inaccuracies and draw false conclusions in their Minds and Machines evaluation, which this paper aims to clarify. Acting as invited judges, Floridi et al. participated in nine, of the ninety-six, Turing tests staged in the finals of the 18th Loebner Prize for Artificial Intelligence in October 2008. From the transcripts it appears that they used power over solidarity as an interrogation technique. As a result, they were fooled on several occasions into believing that a machine was a human and that a human was a machine. Worse still, they did not realise their mistake. This resulted in a combined correct identification rate of less than 56%. In their paper they assumed that they had made correct identifications when they in fact had been incorrect.

Social interaction deficits and conversational inadequacy in Williams syndrome

Research on social communication skills in individuals with Williams syndrome has been inconclusive, with some arguing that these skills are a relative strength and others that they are a weakness. The aim of the present study was to investigate social interaction abilities in a group of children with WS, and to compare them to a group of children with specific language impairment and a group of typically developing children. Semi-structured conversations were conducted and 100-150 utterances were selected for analysis in terms of exchange structure, turn taking, information transfer and conversational inadequacy. The statistical analyses showed that the children with WS had difficulties with exchange structure and responding appropriately to the interlocutor's requests for information and clarification. They also had significant difficulties with interpreting meaning and providing enough information for the conversational partner. Despite similar language abilities with a group of children with specific language impairment, the children with WS had different social interaction skills, which suggests that they follow an atypical trajectory of development and their neurolinguistic profile does not directly support innate modularity. (c) 2005 Elsevier Ltd. All rights reserved.

Deception-detection and machine intelligence in practical Turing tests

Deception-detection is the crux of Turing’s experiment to examine machine thinking conveyed through a capacity to respond with sustained and satisfactory answers to unrestricted questions put by a human interrogator. However, in 60 years to the month since the publication of Computing Machinery and Intelligence little agreement exists for a canonical format for Turing’s textual game of imitation, deception and machine intelligence. This research raises from the trapped mine of philosophical claims, counter-claims and rebuttals Turing’s own distinct five minutes question-answer imitation game, which he envisioned practicalised in two different ways: a) A two-participant, interrogator-witness viva voce, b) A three-participant, comparison of a machine with a human both questioned simultaneously by a human interrogator. Using Loebner’s 18th Prize for Artificial Intelligence contest, and Colby et al.’s 1972 transcript analysis paradigm, this research practicalised Turing’s imitation game with over 400 human participants and 13 machines across three original experiments. Results show that, at the current state of technology, a deception rate of 8.33% was achieved by machines in 60 human-machine simultaneous comparison tests. Results also show more than 1 in 3 Reviewers succumbed to hidden interlocutor misidentification after reading transcripts from experiment 2. Deception-detection is essential to uncover the increasing number of malfeasant programmes, such as CyberLover, developed to steal identity and financially defraud users in chatrooms across the Internet. Practicalising Turing’s two tests can assist in understanding natural dialogue and mitigate the risk from cybercrime.

Outwitted by the hidden: unsure emotions

In this paper the authors consider natural, feigned or absence of emotions in text-based dialogues. The dialogues occurred during interactions between human Judges/Interrogators and hidden entities in practical Turing tests implemented at Bletchley Park in June 2012. The authors focus on the interactions that left the Interrogator unable to say whether they were talking to a human or a machine after five minutes of questioning; the hidden interlocutor received an ‘unsure’ classification. In cases where the Judge has provided post-event feedback the authors present their rationale from three viva voce one-to-one Turing tests. The authors find that emoticons and other visual devices used to express feelings in text-based interaction were missing in the conversations between the Interrogators and hidden interlocutors.