Virtual characters designed for forensic assessment and rehabilitation of sex offenders: standardized and made-to-measure

This paper presents two studies pertaining to the use of virtual characters applied in clinical forensic rehabilitation of sex offenders. The first study is about the validation of the perceived age of virtual characters designed to simulate primary and secondary sexual character of typical adult and child individuals. The second study puts to use these virtual characters in comparing a group of sex offenders and a group of non deviant individuals on their sexual arousal responses as recorded in virtual immersion. Finally, two clinical vignettes illustrating the use of made-to-measure virtual characters to more closely fit sexual preferences are presented in Discussion.

XSAMPL3D: An Action Description Language for the Animation of Virtual Characters

In this paper we present XSAMPL3D, a novel language for the high-level representation of actions performed on objects by (virtual) humans. XSAMPL3D was designed to serve as action representation language in an imitation-based approach to character animation: First, a human demonstrates a sequence of object manipulations in an immersive Virtual Reality (VR) environment. From this demonstration, an XSAMPL3D description is automatically derived that represents the actions in terms of high-level action types and involved objects. The XSAMPL3D action description can then be used for the synthesis of animations where virtual humans of different body sizes and proportions reproduce the demonstrated action. Actions are encoded in a compact and human-readable XML-format. Thus, XSAMPL3D describtions are also amenable to manual authoring, e.g. for rapid prototyping of animations when no immersive VR environment is at the animator's disposal. However, when XSAMPL3D descriptions are derived from VR interactions, they can accomodate many details of the demonstrated action, such as motion trajectiories,hand shapes and other hand-object relations during grasping. Such detail would be hard to specify with manual motion authoring techniques only. Through the inclusion of language features that allow the representation of all relevant aspects of demonstrated object manipulations, XSAMPL3D is a suitable action representation language for the imitation-based approach to character animation.

Knowledgeable Feedback via a Cast of Virtual Characters with Different Competences

DynaLearn (http://www.DynaLearn.eu) develops a cognitive artefact that engages learners in an active learning by modelling process to develop conceptual system knowledge. Learners create external representations using diagrams. The diagrams capture conceptual knowledge using the Garp3 Qualitative Reasoning (QR) formalism [2]. The expressions can be simulated, confronting learners with the logical consequences thereof. To further aid learners, DynaLearn employs a sequence of knowledge representations (Learning Spaces, LS), with increasing complexity in terms of the modelling ingredients a learner can use [1]. An online repository contains QR models created by experts/teachers and learners. The server runs semantic services [4] to generate feedback at the request of learners via the workbench. The feedback is communicated to the learner via a set of virtual characters, each having its own competence [3]. A specific feedback thus incorporates three aspects: content, character appearance, and a didactic setting (e.g. Quiz mode). In the interactive event we will demonstrate the latest achievements of the DynaLearn project. First, the 6 learning spaces for learners to work with. Second, the generation of feedback relevant to the individual needs of a learner using Semantic Web technology. Third, the verbalization of the feedback via different animated virtual characters, notably: Basic help, Critic, Recommender, Quizmaster & Teachable agen

Intelligent system for interaction with virtual characters based on volumetric sensors

Dissertação de Mestrado, Engenharia Elétrica e Eletrónica, Instituto Superior de Engenharia, Universidade do Algarve, 2015

Proxemics with multiple dynamic characters in an immersive virtual environment

An experiment was carried out to examine the impact on electrodermal activity of people when approached by groups of one or four virtual characters at varying distances. It was premised on the basis of proxemics theory that the closer the approach of the virtual characters to the participant, the greater the level of physiological arousal. Physiological arousal was measured by the number of skin conductance responses within a short time period after the approach, and the maximum change in skin conductance level 5 s after the approach. The virtual characters were each either female or a cylinder of human size, and one or four characters approached each subject a total of 12 times. Twelve male subjects were recruited for the experiment. The results suggest that the number of skin conductance responses after the approach and the change in skin conductance level increased the closer the virtual characters approached toward the participants. Moreover, these response variables were inversely correlated with the number of visits, showing a typical adaptation effect. There was some evidence to suggest that the number of characters who simultaneously approached (one or four) was positively associated with the responses. Surprisingly there was no evidence of a difference in response between the humanoid characters and cylinders on the basis of this physiological data. It is suggested that the similarity in this quantitative arousal response to virtual characters and virtual objects might mask a profound difference in qualitative response, an interpretation supported by questionnaire and interview results. Overall the experiment supported the premise that people exhibit heightened physiological arousal the closer they are approached by virtual characters.

Proxemics with multiple dynamic characters in an immersive virtual environment

An experiment was carried out to examine the impact on electrodermal activity of people when approached by groups of one or four virtual characters at varying distances. It was premised on the basis of proxemics theory that the closer the approach of the virtual characters to the participant, the greater the level of physiological arousal. Physiological arousal was measured by the number of skin conductance responses within a short time period after the approach, and the maximum change in skin conductance level 5 s after the approach. The virtual characters were each either female or a cylinder of human size, and one or four characters approached each subject a total of 12 times. Twelve male subjects were recruited for the experiment. The results suggest that the number of skin conductance responses after the approach and the change in skin conductance level increased the closer the virtual characters approached toward the participants. Moreover, these response variables were inversely correlated with the number of visits, showing a typical adaptation effect. There was some evidence to suggest that the number of characters who simultaneously approached (one or four) was positively associated with the responses. Surprisingly there was no evidence of a difference in response between the humanoid characters and cylinders on the basis of this physiological data. It is suggested that the similarity in this quantitative arousal response to virtual characters and virtual objects might mask a profound difference in qualitative response, an interpretation supported by questionnaire and interview results. Overall the experiment supported the premise that people exhibit heightened physiological arousal the closer they are approached by virtual characters.

Proxemics with multiple dynamic characters in an immersive virtual environment

An experiment was carried out to examine the impact on electrodermal activity of people when approached by groups of one or four virtual characters at varying distances. It was premised on the basis of proxemics theory that the closer the approach of the virtual characters to the participant, the greater the level of physiological arousal. Physiological arousal was measured by the number of skin conductance responses within a short time period after the approach, and the maximum change in skin conductance level 5 s after the approach. The virtual characters were each either female or a cylinder of human size, and one or four characters approached each subject a total of 12 times. Twelve male subjects were recruited for the experiment. The results suggest that the number of skin conductance responses after the approach and the change in skin conductance level increased the closer the virtual characters approached toward the participants. Moreover, these response variables were inversely correlated with the number of visits, showing a typical adaptation effect. There was some evidence to suggest that the number of characters who simultaneously approached (one or four) was positively associated with the responses. Surprisingly there was no evidence of a difference in response between the humanoid characters and cylinders on the basis of this physiological data. It is suggested that the similarity in this quantitative arousal response to virtual characters and virtual objects might mask a profound difference in qualitative response, an interpretation supported by questionnaire and interview results. Overall the experiment supported the premise that people exhibit heightened physiological arousal the closer they are approached by virtual characters.

The Impact of a Character Posture Model on the Communication of Affect in an Immersive Virtual Environment

This paper presents the quantitative and qualitative findings from an experiment designed to evaluate a developing model of affective postures for full-body virtual characters in immersive virtual environments (IVEs). Forty-nine participants were each requested to explore a virtual environment by asking two virtual characters for instructions. The participants used a CAVE-like system to explore the environment. Participant responses and their impression of the virtual characters were evaluated through a wide variety of both quantitative and qualitative methods. Combining a controlled experimental approach with various data-collection methods provided a number of advantages such as providing a reason to the quantitative results. The quantitative results indicate that posture plays an important role in the communication of affect by virtual characters. The qualitative findings indicated that participants attribute a variety of psychological states to the behavioral cues displayed by virtual characters. In addition, participants tended to interpret the social context portrayed by the virtual characters in a holistic manner. This suggests that one aspect of the virtual scene colors the perception of the whole social context portrayed by the virtual characters. We conclude by discussing the importance of designing holistically congruent virtual characters especially in immersive settings.

Gaze behavior nonlinear dynamics assessed in virtual immersion as a diagnostic index of sexual deviancy: preliminary results

This paper presents preliminary results about the use of virtual characters, penile plethysmography and gaze behaviour dynamics to assess deviant sexual preferences. Pedophile patients’ responses are compared to those of non-deviant subjects while they were immersed with virtual characters depicting relevant sexual features.

Learning Data-Driven Models of Non-Verbal Behaviors for Building Rapport Using an Intelligent Virtual Agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness. Evidence-based patient-centered Brief Motivational Interviewing (BMI) interven- tions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary. Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems. To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].

Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness.^ Evidence-based patient-centered Brief Motivational Interviewing (BMI) interventions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary.^ Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems.^ To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].^

PERFORMANCE CAPTURE: À PROCURA DAS MELHORES TÉCNICAS E ESTÉTICAS NO CONTEXTO DA ANIMAÇÃO DIGITAL

This paper investigates realism in character computer animation, which triggered the development of new techniques and aesthetic in spectacular cinema and contemporary culture. With the advent of motion or performing capture, animation has made possible that virtual characters or digital creatures reach higher levels in emotional acting, taking place in virtual cinematic worlds or even special effects movies. This technology, when placed at the service of imagination and fantasy can provide new dimensions in character motion and communication. In this context, projects like Peter Jackson’s (2001) The Lord of the Rings, James Cameron’s Avatar (2009) and more recently Steven Spielberg’s Tintin (2011) demonstrate that motion technology is constantly evolving, and it represents a credible option to explore new techniques and aesthetic in contemporary animation.

A Representação de Personagens Virtuais: Como a Tecnologia Obriga à Criação de Estilos e Estéticas Interdisciplinares na Animação Contemporânea

The evolution of computer animation represents one of the most relevant andrevolutionary aspects in the rise of contemporary digital visual culture (Darlew,2000), in particular, phenomena such as cinema “spectacular “ (Ibidem) and videogames. This article analyzes the characteristics of this “culture of simulation” (Turkle, 1995:20) relating the multidisciplinary and spectrum of technical and stylistic choices to the dimension of virtual characters acting. The result of these hybrid mixtures and computerized human motion capture techniques - called virtual cinema, universal capture, motion capture, etc. - cosists mainly on the sophistication of “rotoscoping”, as a new interpretation and appropriation of the captured image. This human motion capture technology, used largely by cinema and digital games, is one of the reasons why the authenticity of the animation is sometimes questioned. It is in the fi eld of 3D computer animation visual that this change is more signifi cant, appearing regularly innovative techniques of image manipulation and “hyper-cinema” (Lamarre, 2006: 31) character’s control with deeper sense of emotions. This shift in the culture that Manovich (2006: 27) calls “photo-GRAPHICS” - and Mulvey (2007) argue that creates a new form of possessive relationship with the viewer, in that it can analyze in detail the image, it can acquire it and modify it - is one of the most important aspects in the rise of Cubbit’s (2007) “cinema of attraction”. This article delves intrinsically into the analyze of virtual character animation — particularly in the fi eld of 3D computer animation and human digital acting.

Sistema de animação de personagens virtuais para comunicação não verbal

O principal objectivo da animação de personagens virtuais é o de contar uma história através da utilização de personagens virtuais emocionalmente expressivos. Os personagens têm personalidades distintas, e transmitem as suas emoções e processos de pensamento através dos seus comportamentos (comunicação não verbal). As suas acções muitas das vezes constituem a geração de movimentos corporais complexos. Existem diversas questões a considerar quando se anima uma entidade complexa, tais como, a posição das zonas móveis e as suas velocidades. Os personagens virtuais são um exemplo de entidades complexas e estão entre os elementos mais utilizados em animação computacional. O foco desta dissertação consistiu na criação de uma proposta de sistema de animação de personagens virtuais, cujos movimentos e expressões faciais são capazes de transmitir emoções e estados de espírito. Os movimentos primários, ou seja os movimentos que definem o comportamento dos personagens, são provenientes da captura de movimentos humanos (Motion Capture). As animações secundárias, tais como as expressões faciais, são criadas em Autodesk Maya recorrendo à técnica BlendShapes. Os dados obtidos pela captura de movimentos, são organizados numa biblioteca de comportamentos através de um grafo de movimentos, conhecido por Move Tree. Esta estrutura permite o controlo em tempo real dos personagens através da gestão do estado dos personagens. O sistema possibilita também a transição eficaz entre movimentos semelhantes e entre diferentes velocidades de locomoção, minimizando o efeito de arrastamento de pés conhecido como footskate. Torna-se assim possível definir um trajecto que o personagem poderá seguir com movimentos suaves. Estão também disponíveis os resultados obtidos nas sessões de avaliação realizadas, que visaram a determinação da qualidade das transições entre animações. Propõem-se ainda o melhoramento do sistema através da implementação da construção automática do grafo de movimentos.

Avatars and the «Imitation Game» – can machines smile?

Considering Alan Turing’s challenge in «Computing Machinery and Intelligence» (1950) – can machines play the «imitation game»? – it is proposed that the requirements of the Turing test are already implicitly being used for checking the credibility of virtual characters and avatars. Like characters, Avatars aim to visually express emotions (the exterior signs of the existence of feeling) and its creators have to resort to emotion codes. Traditional arts have profusely contributed for this field and, together with the science of anatomy, shaped the grounds for current Facial Action Coding System (FACS) and their databases. However, FACS researchers have to improve their «instruction tables» so that the machines will be able, in a near future, to be programmed to carry out the operation of recognizing human expressions (face and body) and classify them adequately. For the moment, the reproductions have to resort to the copy of real life expressions, and the presente smile of avatars comes from mirroring their human users.