Novel Interaction Techniques for Mobile Augmented Reality applications. A Systematic Literature Review

This study reviews the research on interaction techniques and methods that could be applied in mobile augmented reality scenarios. The review is focused on themost recent advances and considers especially the use of head-mounted displays. Inthe review process, we have followed a systematic approach, which makes the reviewtransparent, repeatable, and less prone to human errors than if it was conducted in amore traditional manner. The main research subjects covered in the review are headorientation and gaze-tracking, gestures and body part-tracking, and multimodality– as far as the subjects are related to human-computer interaction. Besides these,also a number of other areas of interest will be discussed.

High-Level Modeling of Multimodal Interaction Techniques Using NiMMiT

The past few years, multimodal interaction has been gaining importance in virtual environments. Although multimodality renders interacting with an environment more natural and intuitive, the development cycle of such an application is often long and expensive. In our overall field of research, we investigate how modelbased design can facilitate the development process by designing environments through the use of highlevel diagrams. In this scope, we present ‘NiMMiT’, a graphical notation for expressing and evaluating multimodal user interaction; we elaborate on the NiMMiT primitives and demonstrate its use by means of a comprehensive example.

A paradigm shift:alternative interaction techniques for use with mobile & wearable devices

Desktop user interface design originates from the fact that users are stationary and can devote all of their visual resource to the application with which they are interacting. In contrast, users of mobile and wearable devices are typically in motion whilst using their device which means that they cannot devote all or any of their visual resource to interaction with the mobile application -- it must remain with the primary task, often for safety reasons. Additionally, such devices have limited screen real estate and traditional input and output capabilities are generally restricted. Consequently, if we are to develop effective applications for use on mobile or wearable technology, we must embrace a paradigm shift with respect to the interaction techniques we employ for communication with such devices.This paper discusses why it is necessary to embrace a paradigm shift in terms of interaction techniques for mobile technology and presents two novel multimodal interaction techniques which are effective alternatives to traditional, visual-centric interface designs on mobile devices as empirical examples of the potential to achieve this shift.

Multimodal 'Eyes-Free' interaction techniques for mobile devices

Mobile and wearable computers present input/output prob-lems due to limited screen space and interaction techniques. When mobile, users typically focus their visual attention on navigating their environment - making visually demanding interface designs hard to operate. This paper presents two multimodal interaction techniques designed to overcome these problems and allow truly mobile, 'eyes-free' device use. The first is a 3D audio radial pie menu that uses head gestures for selecting items. An evaluation of a range of different audio designs showed that egocentric sounds re-duced task completion time, perceived annoyance, and al-lowed users to walk closer to their preferred walking speed. The second is a sonically enhanced 2D gesture recognition system for use on a belt-mounted PDA. An evaluation of the system with and without audio feedback showed users' ges-tures were more accurate when dynamically guided by au-dio-feedback. These novel interaction techniques demon-strate effective alternatives to visual-centric interface de-signs on mobile devices.

Comparing interaction techniques for serious games through brain-computer interfaces:a user perception evaluation study

This paper examines the application of commercial and non-invasive electroencephalography (EEG)-based brain-computer (BCIs) interfaces with serious games. Two different EEG-based BCI devices were used to fully control the same serious game. The first device (NeuroSky MindSet) uses only a single dry electrode and requires no calibration. The second device (Emotiv EPOC) uses 14 wet sensors requiring additional training of a classifier. User testing was performed on both devices with sixty-two participants measuring the player experience as well as key aspects of serious games, primarily learnability, satisfaction, performance and effort. Recorded feedback indicates that the current state of BCIs can be used in the future as alternative game interfaces after familiarisation and in some cases calibration. Comparative analysis showed significant differences between the two devices. The first device provides more satisfaction to the players whereas the second device is more effective in terms of adaptation and interaction with the serious game.

Interaction techniques with novel multimodal feedback for addressing gesture-sensing systems

Users need to be able to address in-air gesture systems, which means finding where to perform gestures and how to direct them towards the intended system. This is necessary for input to be sensed correctly and without unintentionally affecting other systems. This thesis investigates novel interaction techniques which allow users to address gesture systems properly, helping them find where and how to gesture. It also investigates audio, tactile and interactive light displays for multimodal gesture feedback; these can be used by gesture systems with limited output capabilities (like mobile phones and small household controls), allowing the interaction techniques to be used by a variety of device types. It investigates tactile and interactive light displays in greater detail, as these are not as well understood as audio displays. Experiments 1 and 2 explored tactile feedback for gesture systems, comparing an ultrasound haptic display to wearable tactile displays at different body locations and investigating feedback designs. These experiments found that tactile feedback improves the user experience of gesturing by reassuring users that their movements are being sensed. Experiment 3 investigated interactive light displays for gesture systems, finding this novel display type effective for giving feedback and presenting information. It also found that interactive light feedback is enhanced by audio and tactile feedback. These feedback modalities were then used alongside audio feedback in two interaction techniques for addressing gesture systems: sensor strength feedback and rhythmic gestures. Sensor strength feedback is multimodal feedback that tells users how well they can be sensed, encouraging them to find where to gesture through active exploration. Experiment 4 found that they can do this with 51mm accuracy, with combinations of audio and interactive light feedback leading to the best performance. Rhythmic gestures are continuously repeated gesture movements which can be used to direct input. Experiment 5 investigated the usability of this technique, finding that users can match rhythmic gestures well and with ease. Finally, these interaction techniques were combined, resulting in a new single interaction for addressing gesture systems. Using this interaction, users could direct their input with rhythmic gestures while using the sensor strength feedback to find a good location for addressing the system. Experiment 6 studied the effectiveness and usability of this technique, as well as the design space for combining the two types of feedback. It found that this interaction was successful, with users matching 99.9% of rhythmic gestures, with 80mm accuracy from target points. The findings show that gesture systems could successfully use this interaction technique to allow users to address them. Novel design recommendations for using rhythmic gestures and sensor strength feedback were created, informed by the experiment findings.

Embedding metadata in computer graphics for interaction

Illustration Watermarks, Image annotation, Virtual data exploration, Interaction techniques

Marker-Based Embodied Interaction for Handheld Augmented Reality Games

This article deals with embodied user interfaces for handheld augmented reality games, which consist of both physical and virtual components. We have developed a number of spatial interaction techniques that optically capture the device's movement and orientation relative to a visual marker. Such physical interactions in 3-D space enable manipulative control of mobile games. In addition to acting as a physical controller that recognizes multiple game-dependent gestures, the mobile device augments the camera view with graphical overlays. We describe three game prototypes that use ubiquitous product packaging and other passive media as backgrounds for handheld augmentation. The prototypes can be realized on widely available off-the-shelf hardware and require only minimal setup and infrastructure support.

Efficient Bimanual Symmetric 3D Manipulation for Bare-Handed Interaction

Recently, stable markerless 6 DOF video based handtracking devices became available. These devices simultaneously track the positions and orientations of both user hands in different postures with at least 25 frames per second. Such hand-tracking allows for using the human hands as natural input devices. However, the absence of physical buttons for performing click actions and state changes poses severe challenges in designing an efficient and easy to use 3D interface on top of such a device. In particular, for coupling and decoupling a virtual object’s movements to the user’s hand (i.e. grabbing and releasing) a solution has to be found. In this paper, we introduce a novel technique for efficient two-handed grabbing and releasing objects and intuitively manipulating them in the virtual space. This technique is integrated in a novel 3D interface for virtual manipulations. A user experiment shows the superior applicability of this new technique. Last but not least, we describe how this technique can be exploited in practice to improve interaction by integrating it with RTT DeltaGen, a professional CAD/CAS visualization and editing tool.

A paradigm shift for mobile interaction:a decade later

In the proceedings of CASCON'2003 we published an article entitled "A Paradigm Shift: Alternative Interaction Techniques for Use with Mobile & Wearable Devices" and were honoured to receive the Best Paper Award for that year's conference. A decade on, we are equally honoured to have the same paper recognised as the Most Influential Paper of the 2003 conference. To this end, we have been asked to provide a brief overview of our intervening research in relation to that original publication, and so that is the purpose of this extended abstract.

User-interface technologies for the industrial environment: towards the cyber-factory

In the past few years the so-called gadgets like cellular phones, personal data assistants and digital cameras are more widespread even with less technological aware users. However, for several reasons, the factory-floor itself seems to be hermetic to this changes ... After the fieldbus revolution, the factory-floor has seen an increased use of more and more powerful programmable logic controllers and user interfaces but the way they are used remains almost the same. We believe that new user-computer interaction techniques including multimedia and augmented rcaliry combined with now affordable technologies like wearable computers and wireless networks can change the way the factory personal works together with the roachines and the information system on the factory-floor. This new age is already starting with innovative uses of communication networks on the factory-floor either using "standard" networks or enhancing industrial networks with multimedia and wireless capabilities.

Interactive acquisition of spatial information from images for multimedia applications

Dissertação para obtenção do Grau de Doutor em Informática

Cooperative binding of estrogen receptor to imperfect estrogen-responsive DNA elements correlates with their synergistic hormone-dependent enhancer activity.

The Xenopus vitellogenin (vit) gene B1 estrogen-inducible enhancer is formed by two closely adjacent 13 bp imperfect palindromic estrogen-responsive elements (EREs), i.e. ERE-2 and ERE-1, having one and two base substitutions respectively, when compared to the perfect palindromic consensus ERE (GGTCANNNTGACC). Gene transfer experiments indicate that these degenerated elements, on their own, have a low or no regulatory capacity at all, but in vivo act together synergistically to confer high receptor- and hormone-dependent transcription activation to the heterologous HSV thymidine kinase promoter. Thus, the DNA region upstream of the vitB1 gene comprising these two imperfect EREs separated by 7 bp, was called the vitB1 estrogen-responsive unit (vitB1 ERU). Using in vitro protein-DNA interaction techniques, we demonstrate that estrogen receptor dimers bind cooperatively to the imperfect EREs of the vitB1 ERU. Binding of a first receptor dimer to the more conserved ERE-2 increases approximately 4- to 8-fold the binding affinity of the receptor to the adjacent less conserved ERE-1. Thus, we suggest that the observed synergistic estrogen-dependent transcription activation conferred by the pair of hormone-responsive DNA elements of the vit B1 ERU is the result of cooperative binding of two estrogen receptor dimers to these two adjacent imperfect EREs.

Performance of correlation functionals in ab initio chemisorption cluster-model calculations: Alkali metals on Si(111)

The performance of different correlation functionals has been tested for alkali metals, Li to Cs, interacting with cluster models simulating different active sites of the Si(111) surface. In all cases, the ab initio Hartree-Fock density has been obtained and used as a starting point. The electronic correlation energy is then introduced as an a posteriori correction to the Hartree-Fock energy using different correlation functionals. By making use of the ionic nature of the interaction and of different dissociation limits we have been able to prove that all functionals tested introduce the right correlation energy, although to a different extent. Hence, correlation functionals appear as an effective and easy way to introduce electronic correlation in the ab initio Hartree-Fock description of the chemisorption bond in complex systems where conventional configuration interaction techniques cannot be used. However, the calculated energies may differ by some tens of eV. Therefore, these methods can be employed to get a qualitative idea of how important correlation effects are, but they have some limitations if accurate binding energies are to be obtained.

The ViRVIG Institute

In this paper we present the ViRVIG Institute, a recently created institution that joins two well-known research groups: MOVING in Barcelona, and GGG in Girona. Our main research topics are Virtual Reality devices and interaction techniques, complex data models, realistic materials and lighting, geometry processing, and medical image visualization. We briefly introduce the history of both research groups and present some representative projects. Finally, we sketch our lines for future research