Impact Study of Nonverbal Facial Cues on Spontaneous Chatting with Virtual Humans

Non-verbal communication (NVC) is considered to represent more than 90 percent of everyday communication. In virtual world, this important aspect of interaction between virtual humans (VH) is strongly neglected. This paper presents a user-test study to demonstrate the impact of automatically generated graphics-based NVC expression on the dialog quality: first, we wanted to compare impassive and emotion facial expression simulation for impact on the chatting. Second, we wanted to see whether people like chatting within a 3D graphical environment. Our model only proposes facial expressions and head movements induced from spontaneous chatting between VHs. Only subtle facial expressions are being used as nonverbal cues - i.e. related to the emotional model. Motion capture animations related to hand gestures, such as cleaning glasses, were randomly used to make the virtual human lively. After briefly introducing the technical architecture of the 3D-chatting system, we focus on two aspects of chatting through VHs. First, what is the influence of facial expressions that are induced from text dialog? For this purpose, we exploited an emotion engine extracting an emotional content from a text and depicting it into a virtual character developed previously [GAS11]. Second, as our goal was not addressing automatic generation of text, we compared the impact of nonverbal cues in conversation with a chatbot or with a human operator with a wizard of oz approach. Among main results, the within group study -involving 40 subjects- suggests that subtle facial expressions impact significantly not only on the quality of experience but also on dialog understanding.

Face Lift Surgery for Reconstructed Virtual Humans

We introduce an innovative, semi-automatic method to transform low resolution facial meshes into high definition ones, based on the tailoring of a generic, neutral human head model, designed by an artist, to fit the facial features of a specific person. To determine these facial features we need to select a set of "control points" (corners of eyes, lips, etc.) in at least two photographs of the subject's face. The neutral head mesh is then automatically reshaped according to the relation between the control points in the original subject's mesh through a set of transformation pyramids. The last step consists in merging both meshes and filling the gaps that appear in the previous process. This algorithm avoids the use of expensive and complicated technologies to obtain depth maps, which also need to be meshed later.

Intelligent Virtual Patients for Training Clinical Skills

The article presents the design process of intelligent virtual human patients that are used for the enhancement of clinical skills. The description covers the development from conceptualization and character creation to technical components and the application in clinical research and training. The aim is to create believable social interactions with virtual agents that help the clinician to develop skills in symptom and ability assessment, diagnosis, interview techniques and interpersonal communication. The virtual patient fulfills the requirements of a standardized patient producing consistent, reliable and valid interactions in portraying symptoms and behaviour related to a specific clinical condition.

XSAMPL3D: An Action Description Language for the Animation of Virtual Characters

In this paper we present XSAMPL3D, a novel language for the high-level representation of actions performed on objects by (virtual) humans. XSAMPL3D was designed to serve as action representation language in an imitation-based approach to character animation: First, a human demonstrates a sequence of object manipulations in an immersive Virtual Reality (VR) environment. From this demonstration, an XSAMPL3D description is automatically derived that represents the actions in terms of high-level action types and involved objects. The XSAMPL3D action description can then be used for the synthesis of animations where virtual humans of different body sizes and proportions reproduce the demonstrated action. Actions are encoded in a compact and human-readable XML-format. Thus, XSAMPL3D describtions are also amenable to manual authoring, e.g. for rapid prototyping of animations when no immersive VR environment is at the animator's disposal. However, when XSAMPL3D descriptions are derived from VR interactions, they can accomodate many details of the demonstrated action, such as motion trajectiories,hand shapes and other hand-object relations during grasping. Such detail would be hard to specify with manual motion authoring techniques only. Through the inclusion of language features that allow the representation of all relevant aspects of demonstrated object manipulations, XSAMPL3D is a suitable action representation language for the imitation-based approach to character animation.

Integrating interaction resources in the graphics clusters

This paper aims to describe the basic concepts and necessary for Java programs can invoke libraries of programming language C/C ++, through the JNA API. We used a library developed in C/C ++ called Glass [8], which offers a solution for viewing 3D graphics, using graphics clusters, reducing the cost of viewing. The purpose of the work is to interact with the humanoid developed using Java, which makes movements of LIBRAS language for the deaf, as Glass's, so that through this they can view the information using stereoscopic multi-view in full size. ©2010 IEEE.

Learning Two-Person Interaction Models for Responsive Synthetic Humanoids

Imitation learning is a promising approach for generating life-like behaviors of virtual humans and humanoid robots. So far, however, imitation learning has been mostly restricted to single agent settings where observed motions are adapted to new environment conditions but not to the dynamic behavior of interaction partners. In this paper, we introduce a new imitation learning approach that is based on the simultaneous motion capture of two human interaction partners. From the observed interactions, low-dimensional motion models are extracted and a mapping between these motion models is learned. This interaction model allows the real-time generation of agent behaviors that are responsive to the body movements of an interaction partner. The interaction model can be applied both to the animation of virtual characters as well as to the behavior generation for humanoid robots.

Westminster Serious Games Platform (wmin-SGP) a tool for real-time authoring of roleplay simulations for learning

The use of serious games in education and their pedagogical benefit is being widely recognized. However, effective integration of serious games in education depends on addressing two big challenges: the successful incorporation of motivation and engagement that can lead to learning; and the highly specialised skills associated with customised development to meet the required pedagogical objectives. This paper presents the Westminster Serious Games Platform (wmin-SGP) an authoring tool that allows educators/domain experts without games design and development technical skills to create bespoke roleplay simulations in three dimensional scenes featuring fully embodied virtual humans capable of verbal and non-verbal interaction with users fit for specific educational objectives. The paper presents the wmin-SGP system architecture and it evaluates its effectiveness in fulfilling its purpose via the implementation of two roleplay simulations, one for Politics and one for Law. In addition, it presents the results of two types of evaluation that address how successfully the wmin-SGP combines usability principles and game core drives based on the Octalysis gamification framework that lead to motivating games experiences. The evaluation results shows that the wmin-SGP: provides an intuitive environment and tools that support users without advanced technical skills to create in real-time bespoke roleplay simulations in advanced graphical interfaces; satisfies most of the usability principles; and provides balanced simulations based on the Octalysis framework core drives. The paper concludes with a discussion of future extension of this real time authoring tool and directions for further development of the Octalysis framework to address learning.

User Interface Abstraction for enabling TV set based Inclusive Access to the Information Society

199 p.

THE USE OF SIMULATIONS IN EVOLUTIONARY POPULATION GENETICS: APPLICATIONS ON HUMANS, OWLS AND VIRTUAL ORGANISMS

Summary (in English) Computer simulations provide a practical way to address scientific questions that would be otherwise intractable. In evolutionary biology, and in population genetics in particular, the investigation of evolutionary processes frequently involves the implementation of complex models, making simulations a particularly valuable tool in the area. In this thesis work, I explored three questions involving the geographical range expansion of populations, taking advantage of spatially explicit simulations coupled with approximate Bayesian computation. First, the neutral evolutionary history of the human spread around the world was investigated, leading to a surprisingly simple model: A straightforward diffusion process of migrations from east Africa throughout a world map with homogeneous landmasses replicated to very large extent the complex patterns observed in real human populations, suggesting a more continuous (as opposed to structured) view of the distribution of modern human genetic diversity, which may play a better role as a base model for further studies. Second, the postglacial evolution of the European barn owl, with the formation of a remarkable coat-color cline, was inspected with two rounds of simulations: (i) determine the demographic background history and (ii) test the probability of a phenotypic cline, like the one observed in the natural populations, to appear without natural selection. We verified that the modern barn owl population originated from a single Iberian refugium and that they formed their color cline, not due to neutral evolution, but with the necessary participation of selection. The third and last part of this thesis refers to a simulation-only study inspired by the barn owl case above. In this chapter, we showed that selection is, indeed, effective during range expansions and that it leaves a distinguished signature, which can then be used to detect and measure natural selection in range-expanding populations. Résumé (en français) Les simulations fournissent un moyen pratique pour répondre à des questions scientifiques qui seraient inabordable autrement. En génétique des populations, l'étude des processus évolutifs implique souvent la mise en oeuvre de modèles complexes, et les simulations sont un outil particulièrement précieux dans ce domaine. Dans cette thèse, j'ai exploré trois questions en utilisant des simulations spatialement explicites dans un cadre de calculs Bayésiens approximés (approximate Bayesian computation : ABC). Tout d'abord, l'histoire de la colonisation humaine mondiale et de l'évolution de parties neutres du génome a été étudiée grâce à un modèle étonnement simple. Un processus de diffusion des migrants de l'Afrique orientale à travers un monde avec des masses terrestres homogènes a reproduit, dans une très large mesure, les signatures génétiques complexes observées dans les populations humaines réelles. Un tel modèle continu (opposé à un modèle structuré en populations) pourrait être très utile comme modèle de base dans l'étude de génétique humaine à l'avenir. Deuxièmement, l'évolution postglaciaire d'un gradient de couleur chez l'Effraie des clocher (Tyto alba) Européenne, a été examiné avec deux séries de simulations pour : (i) déterminer l'histoire démographique de base et (ii) tester la probabilité qu'un gradient phénotypique, tel qu'observé dans les populations naturelles puisse apparaître sans sélection naturelle. Nous avons montré que la population actuelle des chouettes est sortie d'un unique refuge ibérique et que le gradient de couleur ne peux pas s'être formé de manière neutre (sans l'action de la sélection naturelle). La troisième partie de cette thèse se réfère à une étude par simulations inspirée par l'étude de l'Effraie. Dans ce dernier chapitre, nous avons montré que la sélection est, en effet, aussi efficace dans les cas d'expansion d'aire de distribution et qu'elle laisse une signature unique, qui peut être utilisée pour la détecter et estimer sa force.

A virtual reality approach to the study of visually driven postural control in developing and aging humans

L'être humain utilise trois systèmes sensoriels distincts pour réguler le maintien de la station debout: la somesthésie, le système vestibulaire, et le système visuel. Le rôle de la vision dans la régulation posturale demeure peu connu, notamment sa variabilité en fonction de l'âge, du type développemental, et des atteintes neurologiques. Dans notre travail, la régulation posturale induite visuellement a été évaluée chez des participants au développement et vieillissement normaux âgés de 5-85 ans, chez des individus autistes (développement atypique) âgés de 12-33 ans, ainsi que chez des enfants entre 9-18 ans ayant subi un TCC léger. À cet effet, la réactivité posturale des participants en réponse à un tunnel virtuel entièrement immersif, se mouvant à trois niveaux de vélocité, a été mesurée; des conditions contrôles, où le tunnel était statique ou absent, ont été incluses. Les résultats montrent que la réactivité (i.e. instabilité) posturale induite visuellement est plus élevée chez les jeunes enfants; ensuite, elle s'atténue pour rejoindre des valeurs adultes vers 16-19 ans et augmente de façon linéaire en fonction de l'âge après 45 ans jusqu'à redevenir élevée vers 60 ans. De plus, à la plus haute vélocité du tunnel, les plus jeunes participants autistes ont manifesté significativement moins de réactivité posturale comparativement à leurs contrôles; cette différence n'était pas présente chez des participants plus âgés (16-33 ans). Enfin, les enfants ayant subi un TCC léger, et qui étaient initialement modérément symptomatiques, ont montré un niveau plus élevé d'instabilité posturale induite visuellement que les contrôles, et ce jusqu'à 12 semaines post-trauma malgré le fait que la majorité d'entre eux (89%) n'étaient plus symptomatiques à ce stade. En somme, cela suggère la présence d'une importante période de transition dans la maturation des systèmes sous-tendant l'intégration sensorimotrice impliquée dans le contrôle postural vers l'âge de 16 ans, et d'autres changements sensorimoteurs vers l'âge de 60 ans; cette sur-dépendance visuelle pour la régulation posturale chez les enfants et les aînés pourrait guider l'aménagement d'espaces et l'élaboration d'activités ajustés à l'âge des individus. De plus, le fait que l'hypo-réactivité posturale aux informations visuelles chez les autistes dépende des caractéristiques de l'environnement visuel et de l'âge chronologique, affine notre compréhension des anomalies sensorielles propres à l'autisme. Par ailleurs, le fait que les enfants ayant subi un TCC léger montrent des anomalies posturales jusqu'à 3 mois post-trauma, malgré une diminution significative des symptômes rapportés, pourrait être relié à une altération du traitement de l'information visuelle dynamique et pourrait avoir des implications quant à la gestion clinique des patients aux prises avec un TCC léger, puisque la résolution des symptômes est actuellement le principal critère utilisé pour la prise de décision quant au retour aux activités. Enfin, les résultats obtenus chez une population à développement atypique (autisme) et une population avec atteinte neurologique dite transitoire (TCC léger), contribuent non seulement à une meilleure compréhension des mécanismes d'intégration sensorimotrice sous-tendant le contrôle postural mais pourraient aussi servir comme marqueurs sensibles et spécifiques de dysfonction chez ces populations. Mots-clés : posture, équilibre, vision, développement/vieillissement sensorimoteur, autisme, TCC léger symptomatique, réalité virtuelle.

Estimation of sex and age of "virtual skeletons"--a feasibility study.

This article presents a feasibility study with the objective of investigating the potential of multi-detector computed tomography (MDCT) to estimate the bone age and sex of deceased persons. To obtain virtual skeletons, the bodies of 22 deceased persons with known age at death were scanned by MDCT using a special protocol that consisted of high-resolution imaging of the skull, shoulder girdle (including the upper half of the humeri), the symphysis pubis and the upper halves of the femora. Bone and soft-tissue reconstructions were performed in two and three dimensions. The resulting data were investigated by three anthropologists with different professional experience. Sex was determined by investigating three-dimensional models of the skull and pelvis. As a basic orientation for the age estimation, the complex method according to Nemeskéri and co-workers was applied. The final estimation was effected using additional parameters like the state of dentition, degeneration of the spine, etc., which where chosen individually by the three observers according to their experience. The results of the study show that the estimation of sex and age is possible by the use of MDCT. Virtual skeletons present an ideal collection for anthropological studies, because they are obtained in a non-invasive way and can be investigated ad infinitum.

Assessment of construct validity of a virtual reality laparoscopy simulator.

BACKGROUND: The aim of this study was to assess whether virtual reality (VR) can discriminate between the skills of novices and intermediate-level laparoscopic surgical trainees (construct validity), and whether the simulator assessment correlates with an expert's evaluation of performance. METHODS: Three hundred and seven (307) participants of the 19th-22nd Davos International Gastrointestinal Surgery Workshops performed the clip-and-cut task on the Xitact LS 500 VR simulator (Xitact S.A., Morges, Switzerland). According to their previous experience in laparoscopic surgery, participants were assigned to the basic course (BC) or the intermediate course (IC). Objective performance parameters recorded by the simulator were compared to the standardized assessment by the course instructors during laparoscopic pelvitrainer and conventional surgery exercises. RESULTS: IC participants performed significantly better on the VR simulator than BC participants for the task completion time as well as the economy of movement of the right instrument, not the left instrument. Participants with maximum scores in the pelvitrainer cholecystectomy task performed the VR trial significantly faster, compared to those who scored less. In the conventional surgery task, a significant difference between those who scored the maximum and those who scored less was found not only for task completion time, but also for economy of movement of the right instrument. CONCLUSIONS: VR simulation provides a valid assessment of psychomotor skills and some basic aspects of spatial skills in laparoscopic surgery. Furthermore, VR allows discrimination between trainees with different levels of experience in laparoscopic surgery establishing construct validity for the Xitact LS 500 clip-and-cut task. Virtual reality may become the gold standard to assess and monitor surgical skills in laparoscopic surgery.

Virtual autopsy with multiphase postmortem computed tomographic angiography versus traditional medical autopsy to investigate unexpected deaths of hospitalized patients: a cohort study.

BACKGROUND: "Virtual" autopsy by postmortem computed tomography (PMCT) can replace medical autopsy to a certain extent but has limitations for cardiovascular diseases. These limitations might be overcome by adding multiphase PMCT angiography. OBJECTIVE: To compare virtual autopsy by multiphase PMCT angiography with medical autopsy. DESIGN: Prospective cohort study. (ClinicalTrials.gov: NCT01541995) SETTING: Single-center study at the University Medical Center Hamburg-Eppendorf, Hamburg, Germany, between 1 April 2012 and 31 March 2013. PATIENTS: Hospitalized patients who died unexpectedly or within 48 hours of an event necessitating cardiopulmonary resuscitation. MEASUREMENTS: Diagnoses from clinical records were compared with findings from both types of autopsy. New diagnoses identified by autopsy were classified as major or minor, depending on whether they would have altered clinical management. RESULTS: Of 143 eligible patients, 50 (35%) had virtual and medical autopsy. Virtual autopsy confirmed 93% of all 336 diagnoses identified from antemortem medical records, and medical autopsy confirmed 80%. In addition, virtual and medical autopsy identified 16 new major and 238 new minor diagnoses. Seventy-three of the virtual autopsy diagnoses, including 32 cases of coronary artery stenosis, were identified solely by multiphase PMCT angiography. Of the 114 clinical diagnoses classified as cardiovascular, 110 were confirmed by virtual autopsy and 107 by medical autopsy. In 11 cases, multiphase PMCT angiography showed "unspecific filling defects," which were not reported by medical autopsy. LIMITATION: These results come from a single center with concerted interest and expertise in postmortem imaging; further studies are thus needed for generalization. CONCLUSION: In cases of unexpected death, the addition of multiphase PMCT angiography increases the value of virtual autopsy, making it a feasible alternative for quality control and identification of diagnoses traditionally made by medical autopsy. PRIMARY FUNDING SOURCE: University Medical Center Hamburg-Eppendorf.

Mort des voyageurs: risques réels, risques virtuels [True and virtual risks of travellers].

Evidence-based information on travel associated mortality is scarce. Perception, intuition and the availability of interventions such as vaccinations and chemoprophylaxis often guide pre-travel advice. Important risks including accidents and cardiovascular events are not routinely included in pre-travel consultations although they cause more fatalities and costs than infectious diseases. The increased risk of sustaining a road accident in poor economy countries should always be mentioned. The general practitioner is further best placed to discuss possible problems of travellers with chronic diseases before travel.

Correlations between Vocal Input and Visual Response Apparently Enhance Presence in a Virtual Environment

This work investigates novel alternative means of interaction in a virtual environment (VE).We analyze whether humans can remap established body functions to learn to interact with digital information in an environment that is cross-sensory by nature and uses vocal utterances in order to influence (abstract) virtual objects. We thus establish a correlation among learning, control of the interface, and the perceived sense of presence in the VE. The application enables intuitive interaction by mapping actions (the prosodic aspects of the human voice) to a certain response (i.e., visualization). A series of single-user and multiuser studies shows that users can gain control of the intuitive interface and learn to adapt to new and previously unseen tasks in VEs. Despite the abstract nature of the presented environment, presence scores were generally very high.