Correlations between vocal input and visual response apparently enhance presence in a virtual environment

This work investigates novel alternative means of interaction in a virtual environment (VE).We analyze whether humans can remap established body functions to learn to interact with digital information in an environment that is cross-sensory by nature and uses vocal utterances in order to influence (abstract) virtual objects. We thus establish a correlation among learning, control of the interface, and the perceived sense of presence in the VE. The application enables intuitive interaction by mapping actions (the prosodic aspects of the human voice) to a certain response (i.e., visualization). A series of single-user and multiuser studies shows that users can gain control of the intuitive interface and learn to adapt to new and previously unseen tasks in VEs. Despite the abstract nature of the presented environment, presence scores were generally very high.

Spatial radial maze procedures and setups to dissociate local and distal relational spatial frameworks in humans.

BACKGROUND: Radial maze tasks have been used to assess optimal foraging and spatial abilities in rodents. The spatial performance was based on a capacity to rely on a configuration of local and distant cues. We adapted maze procedures assessing the relative weight of local cues and distant landmarks for arm choice in humans. NEW METHOD: The procedure allowed testing memory of places in four experimental setups: a fingertip texture-groove maze, a tactile screen maze, a virtual radial maze and a walking size maze. During training, the four reinforced positions remained fixed relative to local and distal cues. During subsequent conflict trials, these frameworks were made conflictive in the prediction of reward locations. RESULTS: Three experiments showed that the relative weight of local and distal relational cues is affected by different factors such as cues' nature, visual access to the environment, real vs. virtual environment, and gender. A fourth experiment illustrated how a walking maze can be used with people suffering intellectual disability. COMPARISON WITH EXISTING METHODS: In our procedure, long-term (reference) and short-term (working) memory can be assessed. It is the first radial task adapted to human that enables dissociating local and distal cues, to provides an indication as to their relative salience. Our mazes are moveable and easily used in limited spaces. Tasks are performed with realistic and spontaneous though controlled exploratory movements. CONCLUSION: Our tasks enabled highlighting the use of different strategies. In a clinical perspective, considering the use of compensatory strategies should orient towards adapted behavioural rehabilitation.

Protein pocket and ligand shape comparison and its application in virtual screening.

Understanding molecular recognition is one major requirement for drug discovery and design. Physicochemical and shape complementarity between two binding partners is the driving force during complex formation. In this study, the impact of shape within this process is analyzed. Protein binding pockets and co-crystallized ligands are represented by normalized principal moments of inertia ratios (NPRs). The corresponding descriptor space is triangular, with its corners occupied by spherical, discoid, and elongated shapes. An analysis of a selected set of sc-PDB complexes suggests that pockets and bound ligands avoid spherical shapes, which are, however, prevalent in small unoccupied pockets. Furthermore, a direct shape comparison confirms previous studies that on average only one third of a pocket is filled by its bound ligand, supplemented by a 50 % subpocket coverage. In this study, we found that shape complementary is expressed by low pairwise shape distances in NPR space, short distances between the centers-of-mass, and small deviations in the angle between the first principal ellipsoid axes. Furthermore, it is assessed how different binding pocket parameters are related to bioactivity and binding efficiency of the co-crystallized ligand. In addition, the performance of different shape and size parameters of pockets and ligands is evaluated in a virtual screening scenario performed on four representative targets.

3D Object Recognition: Symmetry and Virtual Views

Many 3D objects in the world around us are strongly constrained. For instance, not only cultural artifacts but also many natural objects are bilaterally symmetric. Thoretical arguments suggest and psychophysical experiments confirm that humans may be better in the recognition of symmetric objects. The hypothesis of symmetry-induced virtual views together with a network model that successfully accounts for human recognition of generic 3D objects leads to predictions that we have verified with psychophysical experiments.

Humans ignore motion and stereo cues in favor of a fictional stable world

As we move through the world, our eyes acquire a sequence of images. The information from this sequence is sufficient to determine the structure of a three-dimensional scene, up to a scale factor determined by the distance that the eyes have moved [1, 2]. Previous evidence shows that the human visual system accounts for the distance the observer has walked [3,4] and the separation of the eyes [5-8] when judging the scale, shape, and distance of objects. However, in an immersive virtual-reality environment, observers failed to notice when a scene expanded or contracted, despite having consistent information about scale from both distance walked and binocular vision. This failure led to large errors in judging the size of objects. The pattern of errors cannot be explained by assuming a visual reconstruction of the scene with an incorrect estimate of interocular separation or distance walked. Instead, it is consistent with a Bayesian model of cue integration in which the efficacy of motion and disparity cues is greater at near viewing distances. Our results imply that observers are more willing to adjust their estimate of interocular separation or distance walked than to accept that the scene has changed in size.

Using high-fidelity virtual reality to study perception in freely moving observers

Technological innovations have had a profound influence on how we study the sensory perception in humans and other animals. One example was the introduction of affordable computers, which radically changed the nature of visual experiments. It is clear that vision research is now at cusp of a similar shift, this time driven by the use of commercially available, low-cost, high- fidelity virtual reality (VR). In this review we will focus on: (a) the research questions VR allows experimenters to address and why these research questions are important, (b) the things that need to be considered when using VR to study human perception, (c) the drawbacks of current VR systems, and (d) the future direction vision research may take, now that VR has become a viable research tool.

Hovercraft telemanipulation virtual models and interface design

A control system was designed to allow humans to manually drive an, usually automatic, two wheeled hovercraft. The size, the mass and the way of driving this vehicle proves to be an issue for the everyday, untrained person to achieve. During this thesis several control layouts were designed with the objective of creating an intuitive and easy way of driving such a vehicle. At the end two where usertested using a simulation (also developed during this thesis) of the said hovercraft set against obstacles similar to those expected to be encountered on its real environment. The two layouts are just slightly apart in performance but numerous issues were found that can be used to redesign a better control layout. This means that no definitive winner was found but a foundation for a better design was indeed found.

A produção matemática em um ambiente virtual de aprendizagem: o caso da geometria euclidiana espacial

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Efeito do filme de carbono DLC aplicado por plasma como recobrimento em Polietileno de Ultra Densidade e Alto Peso Molecular (UHWPE) e planejamento virtual utilizando a técnica de elementos finitos em cirúrgias de próteses totais da ATM

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Efeito do filme de carbono DLC aplicado por plasma como recobrimento em Polietileno de Ultra Densidade e Alto Peso Molecular (UHWPE) e planejamento virtual utilizando a técnica de elementos finitos em cirúrgias de próteses totais da ATM

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

VIRTOPSY: minimally invasive, imaging-guided virtual autopsy

Invasive "body-opening" autopsy represents the traditional means of postmortem investigation in humans. However, modern cross-sectional imaging techniques can supplement and may even partially replace traditional autopsy. Computed tomography (CT) is the imaging modality of choice for two- and three-dimensional documentation and analysis of autopsy findings including fracture systems, pathologic gas collections (eg, air embolism, subcutaneous emphysema after trauma, hyperbaric trauma, decomposition effects), and gross tissue injury. Various postprocessing techniques can provide strong forensic evidence for use in legal proceedings. Magnetic resonance (MR) imaging has had a greater impact in demonstrating soft-tissue injury, organ trauma, and nontraumatic conditions. However, the differences in morphologic features and signal intensity characteristics seen at antemortem versus postmortem MR imaging have not yet been studied systematically. The documentation and analysis of postmortem findings with CT and MR imaging and postprocessing techniques ("virtopsy") is investigator independent, objective, and noninvasive and will lead to qualitative improvements in forensic pathologic investigation. Future applications of this approach include the assessment of morbidity and mortality in the general population and, perhaps, routine screening of bodies prior to burial.

Oscillatory beta activity mediates neuroplastic effects of motor cortex stimulation in humans

Continuous theta burst stimulation (cTBS) is a repetitive transcranial magnetic stimulation protocol that can inhibithumanmotor cortex (M1) excitability and impair movement for ≤1 h. While offering valuable insights into brain function and potential therapeutic benefits, these neuroplastic effects are highly variable between individuals. The source of this variability, and the electrophysiological mechanisms underlying the inhibitory after-effects, are largely unknown. In this regard, oscillatory activity at beta frequency (15-35 Hz) is of particular interest as it is elevated in motor disorders such as Parkinson's disease and modulated during the generation of movements. Here, we used a source-level magnetoencephalography approach to investigate the hypothesis that the presence of neuroplastic effects following cTBS is associated with concurrent changes in oscillatory M1 beta activity. M1 cortices were localized with a synthetic aperture magnetometry beamforming analysis of visually cued index finger movements. Virtual electrode analysis was used to reconstruct the spontaneous and movement-related oscillatory activity in bilateral M1 cortices, before and from 10 to 45 min after cTBS. We demonstrate that 40 s of cTBS applied over left M1 reduced corticospinal excitability in the right index finger of 8/16 participants. In these responder participants only, cTBS increased the power of the spontaneous beta oscillations in stimulated M1 and delayed reaction times in the contralateral index finger. No further changes were observed in the latency or power of movement-related beta oscillations. These data provide insights into the electrophysiological mechanisms underlying cTBS-mediated impairment of motor function and demonstrate the association between spontaneous oscillatory beta activity in M1 and the inhibition of motor function. © 2013 the authors.

Learning Data-Driven Models of Non-Verbal Behaviors for Building Rapport Using an Intelligent Virtual Agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness. Evidence-based patient-centered Brief Motivational Interviewing (BMI) interven- tions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary. Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems. To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].

Learning at the Interstices; Locating Practical Philosophies for Understanding Physical/virtual Inter-spaces

Virtual worlds are relatively recent developments, and so it is tempting to believe that they need to be understood through newly developed theories and philosophies. However, humans have long thought about the nature of reality and what it means to be “real.” This paper examines the three persistent philosophical concepts of Metaxis, Liminality and Space that have evolved across more than 2000 years of meditation, contemplation and reflection. Our particular focus here is on the nature of the interface between the virtual and the physical: at the interstices, and how the nature of transactions and transitions across those interfaces may impact upon learning. This may, at first, appear to be an esoteric pursuit, but we ground our arguments in primary and secondary data from research studies in higher education.

Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness.^ Evidence-based patient-centered Brief Motivational Interviewing (BMI) interventions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary.^ Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems.^ To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].^