Smart object for 3D interaction

This paper reports on the creation of an interface for 3D virtual environments, computer-aided design applications or computer games. Standard computer interfaces are bound to 2D surfaces, e.g., computer mouses, keyboards, touch pads or touch screens. The Smart Object is intended to provide the user with a 3D interface by using sensors that register movement (inertial measurement unit), touch (touch screen) and voice (microphone). The design and development process as well as the tests and results are presented in this paper. The Smart Object was developed by a team of four third-year engineering students from diverse scientific backgrounds and nationalities during one semester.

Recorte volumétrico usando técnicas de interação 2D e 3D

A visualização de conjuntos de dados volumétricos é comum em diversas áreas de aplicação e há já alguns anos os diversos aspectos envolvidos nessas técnicas vêm sendo pesquisados. No entanto, apesar dos avanços das técnicas de visualização de volumes, a interação com grandes volumes de dados ainda apresenta desafios devido a questões de percepção (ou isolamento) de estruturas internas e desempenho computacional. O suporte do hardware gráfico para visualização baseada em texturas permite o desenvolvimento de técnicas eficientes de rendering que podem ser combinadas com ferramentas de recorte interativas para possibilitar a inspeção de conjuntos de dados tridimensionais. Muitos estudos abordam a otimização do desempenho de ferramentas de recorte, mas muito poucos tratam das metáforas de interação utilizadas por essas ferramentas. O objetivo deste trabalho é desenvolver ferramentas interativas, intuitivas e fáceis de usar para o recorte de imagens volumétricas. Inicialmente, é apresentado um estudo sobre as principais técnicas de visualização direta de volumes e como é feita a exploração desses volumes utilizando-se recorte volumétrico. Nesse estudo é identificada a solução que melhor se enquadra no presente trabalho para garantir a interatividade necessária. Após, são apresentadas diversas técnicas de interação existentes, suas metáforas e taxonomias, para determinar as possíveis técnicas de interação mais fáceis de serem utilizadas por ferramentas de recorte. A partir desse embasamento, este trabalho apresenta o desenvolvimento de três ferramentas de recorte genéricas implementadas usando-se duas metáforas de interação distintas que são freqüentemente utilizadas por usuários de aplicativos 3D: apontador virtual e mão virtual. A taxa de interação dessas ferramentas é obtida através de programas de fragmentos especiais executados diretamente no hardware gráfico. Estes programas especificam regiões dentro do volume a serem descartadas durante o rendering, com base em predicados geométricos. Primeiramente, o desempenho, precisão e preferência (por parte dos usuários) das ferramentas de recorte volumétrico são avaliados para comparar as metáforas de interação empregadas. Após, é avaliada a interação utilizando-se diferentes dispositivos de entrada para a manipulação do volume e ferramentas. A utilização das duas mãos ao mesmo tempo para essa manipulação também é testada. Os resultados destes experimentos de avaliação são apresentados e discutidos.

Interação com objetos digitais 3D em estúdios virtuais

Pós-graduação em Televisão Digital: Informação e Conhecimento - FAAC

Comparison of 2D and 3D GUI Widgets for Stereoscopic Multitouch Setups

Recent developments in the area of interactive entertainment have suggested to combine stereoscopic visualization with multi-touch displays, which has the potential to open up new vistas for natural interaction with interactive three-dimensional (3D) applications. However, the question arises how the user interfaces for system control in such 3D setups should be designed in order to provide an effective user experience. In this article we introduce 3D GUI widgets for interaction with stereoscopic touch displays. The design of the widgets was inspired to skeuomorphism and affordances in such a way that the user should be able to operate the virtual objects in the same way as their real-world equivalents. We evaluate the developed widgets and compared them with their 2D counterparts in the scope of an example application in order to analyze the usability of and user behavior with the widgets. The results reveal differences in user behavior with and without stereoscopic display during touch interaction, and show that the developed 2D as well as 3D GUI widgets can be used effectively in different applications.

Influence of Comfort on 3D Selection Task Performance in Immersive Desktop Setups

Immersive virtual environments (IVEs) have the potential to afford natural interaction in the three-dimensional (3D) space around a user. However, interaction performance in 3D mid-air is often reduced and depends on a variety of ergonomics factors, the user's endurance, muscular strength, as well as fitness. In particular, in contrast to traditional desktop-based setups, users often cannot rest their arms in a comfortable pose during the interaction. In this article we analyze the impact of comfort on 3D selection tasks in an immersive desktop setup. First, in a pre-study we identified how comfortable or uncomfortable specific interaction positions and poses are for users who are standing upright. Then, we investigated differences in 3D selection task performance when users interact with their hands in a comfortable or uncomfortable body pose, while sitting on a chair in front of a table while the VE was displayed on a headmounted display (HMD). We conducted a Fitts' Law experiment to evaluate selection performance in different poses. The results suggest that users achieve a significantly higher performance in a comfortable pose when they rest their elbow on the table.

Human protein Sam68 relocalization and interaction with poliovirus RNA polymerase in infected cells.

A HeLa cDNA expression library was screened for human polypeptides that interacted with the poliovirus RNA-dependent RNA polymerase, 3D, using the two-hybrid system in the yeast Saccharomyces cerevisiae. Sam68 (Src-associated in mitosis, 68 kDa) emerged as the human cDNA that, when fused to a transcriptional activation domain, gave the strongest 3D interaction signal with a LexA-3D hybrid protein. 3D polymerase and Sam68 coimmunoprecipitated from infected human cell lysates with antibodies that recognized either protein. Upon poliovirus infection, Sam68 relocalized from the nucleus to the cytoplasm, where poliovirus replication occurs. Sam68 was isolated from infected cell lysates with an antibody that recognizes poliovirus protein 2C, suggesting that it is found on poliovirus-induced membranes upon which viral RNA synthesis occurs. These data, in combination with the known RNA- and protein-binding properties of Sam68, make Sam68 a strong candidate for a host protein with a functional role in poliovirus replication.

RGBD Human-Hand recognition for the Interaction with Robot-Hand

New low cost sensors and the new open free libraries for 3D image processing are permitting to achieve important advances for robot vision applications such as tridimensional object recognition, semantic mapping, navigation and localization of robots, human detection and/or gesture recognition for human-machine interaction. In this paper, a method to recognize the human hand and to track the fingers is proposed. This new method is based on point clouds from range images, RGBD. It does not require visual marks, camera calibration, environment knowledge and complex expensive acquisition systems. Furthermore, this method has been implemented to create a human interface in order to move a robot hand. The human hand is recognized and the movement of the fingers is analyzed. Afterwards, it is imitated from a Barret hand, using communication events programmed from ROS.

Redefinición de las TIC en el museo: del discurso invasivo al inclusivo

Las TIC son inseparables de la museografía in situ e imprescindibles en la museografía en red fija y móvil. En demasiados casos se han instalado prótesis tecnológicas para barnizar de modernidad el espacio cultural, olvidando que la tecnología debe estar al servicio de los contenidos de manera que resulte invisible y perfectamente imbricada con la museografía tradicional. Las interfaces móviles pueden fusionar museo in situ y en red y acompañar a las personas más allá del espacio físico. Esa fusión debe partir de una base de datos narrativa y abierta a obras materiales e inmateriales de otros museos de manera que no se trasladen las limitaciones del museo físico al virtual. En el museo in situ tienen sentido las instalaciones hipermedia inmersivas que faciliten experiencias culturales innovadoras. La interactividad (relaciones virtuales) debe convivir con la interacción (relaciones físicas y personales) y estar al servicio de todas las personas, partiendo de que todas, todos tenemos limitaciones. Trabajar interdisciplinarmente ayuda a comprender mejor el museo para ponerlo al servicio de las personas.

3D path planning with novel multiple 2D layered approach for complex human-robot interaction

Navigating cluttered indoor environments is a difficult problem in indoor service robotics. The Acroboter concept, a novel approach to indoor locomotion, represents unique opportunity to avoid obstacles in indoor environments by navigating the ceiling plane. This mode of locomotion requires the ability to accurately detect obstacles, and plan 3D trajectories through the environment. This paper presents the development of a resilient object tracking system, as well as a novel approach to generating 3D paths suitable for such robot configurations. Distributed human-machine interfacing allowing simulation previewing of actions is also considered in the developed system architecture.

Soil-structure interaction during tunnelling in urban area: observations and 3D numerical modelling

This work illustrates a soil-tunnel-structure interaction study performed by an integrated,geotechnical and structural,approach based on 3D finite element analyses and validated against experimental observations.The study aims at analysing the response of reinforced concrete framed buildings on discrete foundations in interaction with metro lines.It refers to the case of the twin tunnels of the Milan (Italy) metro line 5,recently built in coarse grained materials using EPB machines,for which subsidence measurements collected along ground and building sections during tunnelling were available.Settlements measured under freefield conditions are firstly back interpreted using Gaussian empirical predictions. Then,the in situ measurements’ analysis is extended to include the evolving response of a 9 storey reinforced concrete building while being undercrossed by the metro line.In the finite element study,the soil mechanical behaviour is described using an advanced constitutive model. This latter,when combined with a proper simulation of the excavation process, proves to realistically reproduce the subsidence profiles under free field conditions and to capture the interaction phenomena occurring between the twin tunnels during the excavation. Furthermore, when the numerical model is extended to include the building, schematised in a detailed manner, the results are in good agreement with the monitoring data for different stages of the twin tunnelling. Thus, they indirectly confirm the satisfactory performance of the adopted numerical approach which also allows a direct evaluation of the structural response as an outcome of the analysis. Further analyses are also carried out modelling the building with different levels of detail. The results highlight that, in this case, the simplified approach based on the equivalent plate schematisation is inadequate to capture the real tunnelling induced displacement field. The overall behaviour of the system proves to be mainly influenced by the buried portion of the building which plays an essential role in the interaction mechanism, due to its high stiffness.

Evaluation of 3D cell culture systems for host-pathogen interaction studies

Traditional cell culture models have limitations in extrapolating functional mechanisms that underlie strategies of microbial virulence. Indeed during the infection the pathogens adapt to different tissue-specific environmental factors. The development of in vitro models resembling human tissue physiology might allow the replacement of inaccurate or aberrant animal models. Three-dimensional (3D) cell culture systems are more reliable and more predictive models that can be used for the meaningful dissection of host–pathogen interactions. The lung and gut mucosae often represent the first site of exposure to pathogens and provide a physical barrier against their entry. Within this context, the tracheobronchial and small intestine tract were modelled by tissue engineering approach. The main work was focused on the development and the extensive characterization of a human organotypic airway model, based on a mechanically supported co-culture of normal primary cells. The regained morphological features, the retrieved environmental factors and the presence of specific epithelial subsets resembled the native tissue organization. In addition, the respiratory model enabled the modular insertion of interesting cell types, such as innate immune cells or multipotent stromal cells, showing a functional ability to release pertinent cytokines differentially. Furthermore this model responded imitating known events occurring during the infection by Non-typeable H. influenzae. Epithelial organoid models, mimicking the small intestine tract, were used for a different explorative analysis of tissue-toxicity. Further experiments led to detection of a cell population targeted by C. difficile Toxin A and suggested a role in the impairment of the epithelial homeostasis by the bacterial virulence machinery. The described cell-centered strategy can afford critical insights in the evaluation of the host defence and pathogenic mechanisms. The application of these two models may provide an informing step that more coherently defines relevant molecular interactions happening during the infection.

Investigating the interaction of cellulose nanofibers derived from cotton with a sophisticated 3D human lung cell coculture

Cellulose nanofibers are an attractive component of a broad range of nanomaterials. Their intriguing mechanical properties and low cost, as well as the renewable nature of cellulose make them an appealing alternative to carbon nanotubes (CNTs), which may pose a considerable health risk when inhaled. Little is known, however, concerning the potential toxicity of aerosolized cellulose nanofibers. Using a 3D in vitro triple cell coculture model of the human epithelial airway barrier, it was observed that cellulose nanofibers isolated from cotton (CCN) elicited a significantly (p < 0.05) lower cytotoxicity and (pro-)inflammatory response than multiwalled CNTs (MWCNTs) and crocidolite asbestos fibers (CAFs). Electron tomography analysis also revealed that the intracellular localization of CCNs is different from that of both MWCNTs and CAFs, indicating fundamental differences between each different nanofibre type in their interaction with the human lung cell coculture. Thus, the data shown in the present study highlights that not only the length and stiffness determine the potential detrimental (biological) effects of any nanofiber, but that the material used can significantly affect nanofiber-cell interactions.

Evaluation of Binocular Eye Trackers and Algorithms for 3D Gaze Interaction in Virtual Reality Environments

Tracking user’s visual attention is a fundamental aspect in novel human-computer interaction paradigms found in Virtual Reality. For example, multimodal interfaces or dialogue-based communications with virtual and real agents greatly benefit from the analysis of the user’s visual attention as a vital source for deictic references or turn-taking signals. Current approaches to determine visual attention rely primarily on monocular eye trackers. Hence they are restricted to the interpretation of two-dimensional fixations relative to a defined area of projection. The study presented in this article compares precision, accuracy and application performance of two binocular eye tracking devices. Two algorithms are compared which derive depth information as required for visual attention-based 3D interfaces. This information is further applied to an improved VR selection task in which a binocular eye tracker and an adaptive neural network algorithm is used during the disambiguation of partly occluded objects.

Efficient Bimanual Symmetric 3D Manipulation for Bare-Handed Interaction

Recently, stable markerless 6 DOF video based handtracking devices became available. These devices simultaneously track the positions and orientations of both user hands in different postures with at least 25 frames per second. Such hand-tracking allows for using the human hands as natural input devices. However, the absence of physical buttons for performing click actions and state changes poses severe challenges in designing an efficient and easy to use 3D interface on top of such a device. In particular, for coupling and decoupling a virtual object’s movements to the user’s hand (i.e. grabbing and releasing) a solution has to be found. In this paper, we introduce a novel technique for efficient two-handed grabbing and releasing objects and intuitively manipulating them in the virtual space. This technique is integrated in a novel 3D interface for virtual manipulations. A user experiment shows the superior applicability of this new technique. Last but not least, we describe how this technique can be exploited in practice to improve interaction by integrating it with RTT DeltaGen, a professional CAD/CAS visualization and editing tool.

Foundations of a New Interaction Paradigm for Immersive 3D Multimedia

Immersion and interaction have been identified as key factors influencing the quality of experience in stereoscopic video systems. The work presented here aims to create a new paradigm for 3D Multimedia consumption exploiting these factors in order to increase user involvement. We use a 5-sided CAVETM environment to support 3D panoramic video reproduction, real-time insertion of synthetic objects into the three-dimensional scene and real-time user interaction with the inserted elements. In this paper we describe our system requirements, functionalities, conceptual design and preliminary implementation results emphasizing the most relevant challenges accomplished. The focus is on three main issues: the generation of stereoscopic video panoramas; the synchronous reproduction of immersive 3D video across multiple screens; and, the real-time insertion algorithm implemented for the integration of synthetic objects into the stereoscopic video. These results have been successfully integrated into the graphic engine managing the operation of the CAVETM infrastructure.