Portable 3D laser-camera calibration system with color fusion for SLAM

Paper submitted to the 43rd International Symposium on Robotics (ISR2012), Taipei, Taiwan, Aug. 29-31, 2012.

3D visual sensing of human hand for remote operation of a robotic hand

New low cost sensors and open free libraries for 3D image processing are making important advances in robot vision applications possible, such as three-dimensional object recognition, semantic mapping, navigation and localization of robots, human detection and/or gesture recognition for human-machine interaction. In this paper, a novel method for recognizing and tracking the fingers of a human hand is presented. This method is based on point clouds from range images captured by a RGBD sensor. It works in real time and it does not require visual marks, camera calibration or previous knowledge of the environment. Moreover, it works successfully even when multiple objects appear in the scene or when the ambient light is changed. Furthermore, this method was designed to develop a human interface to control domestic or industrial devices, remotely. In this paper, the method was tested by operating a robotic hand. Firstly, the human hand was recognized and the fingers were detected. Secondly, the movement of the fingers was analysed and mapped to be imitated by a robotic hand.

Combining visual features and Growing Neural Gas networks for robotic 3D SLAM

The use of 3D data in mobile robotics provides valuable information about the robot’s environment. Traditionally, stereo cameras have been used as a low-cost 3D sensor. However, the lack of precision and texture for some surfaces suggests that the use of other 3D sensors could be more suitable. In this work, we examine the use of two sensors: an infrared SR4000 and a Kinect camera. We use a combination of 3D data obtained by these cameras, along with features obtained from 2D images acquired from these cameras, using a Growing Neural Gas (GNG) network applied to the 3D data. The goal is to obtain a robust egomotion technique. The GNG network is used to reduce the camera error. To calculate the egomotion, we test two methods for 3D registration. One is based on an iterative closest points algorithm, and the other employs random sample consensus. Finally, a simultaneous localization and mapping method is applied to the complete sequence to reduce the global error. The error from each sensor and the mapping results from the proposed method are examined.

Geometric 3D point cloud compression

The use of 3D data in mobile robotics applications provides valuable information about the robot’s environment but usually the huge amount of 3D information is unmanageable by the robot storage and computing capabilities. A data compression is necessary to store and manage this information but preserving as much information as possible. In this paper, we propose a 3D lossy compression system based on plane extraction which represent the points of each scene plane as a Delaunay triangulation and a set of points/area information. The compression system can be customized to achieve different data compression or accuracy ratios. It also supports a color segmentation stage to preserve original scene color information and provides a realistic scene reconstruction. The design of the method provides a fast scene reconstruction useful for further visualization or processing tasks.

Towards a real-time 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors

In this project, we propose the implementation of a 3D object recognition system which will be optimized to operate under demanding time constraints. The system must be robust so that objects can be recognized properly in poor light conditions and cluttered scenes with significant levels of occlusion. An important requirement must be met: the system must exhibit a reasonable performance running on a low power consumption mobile GPU computing platform (NVIDIA Jetson TK1) so that it can be integrated in mobile robotics systems, ambient intelligence or ambient assisted living applications. The acquisition system is based on the use of color and depth (RGB-D) data streams provided by low-cost 3D sensors like Microsoft Kinect or PrimeSense Carmine. The range of algorithms and applications to be implemented and integrated will be quite broad, ranging from the acquisition, outlier removal or filtering of the input data and the segmentation or characterization of regions of interest in the scene to the very object recognition and pose estimation. Furthermore, in order to validate the proposed system, we will create a 3D object dataset. It will be composed by a set of 3D models, reconstructed from common household objects, as well as a handful of test scenes in which those objects appear. The scenes will be characterized by different levels of occlusion, diverse distances from the elements to the sensor and variations on the pose of the target objects. The creation of this dataset implies the additional development of 3D data acquisition and 3D object reconstruction applications. The resulting system has many possible applications, ranging from mobile robot navigation and semantic scene labeling to human-computer interaction (HCI) systems based on visual information.

Point clouds of measurements in the Dechen Cave near Iserlohn, Germany

Modeling natural phenomena from 3D information enhances our understanding of the environment. Dense 3D point clouds are increasingly used as highly detailed input datasets. In addition to the capturing techniques of point clouds with LiDAR, low-cost sensors have been released in the last few years providing access to new research fields and facilitating 3D data acquisition for a broader range of applications. This letter presents an analysis of different speleothem features using 3D point clouds acquired with the gaming device Microsoft® Kinect. We compare the Kinect sensor with terrestrial LiDAR reference measurements using the KinFu pipeline for capturing complete 3D objects (< 4m**3). The results demonstrate the suitability of the Kinect to capture flowstone walls and to derive morphometric parameters of cave features. Although the chosen capturing strategy (KinFu) reveals a high correlation (R2=0.92) of stalagmite morphometry along the vertical object axis, a systematic overestimation (22% for radii and 44% for volume) is found. The comparison of flowstone wall datasets predominantly shows low differences (mean of 1 mm with 7 mm standard deviation) of the order of the Kinect depth precision. For both objects the major differences occur at strongly varying and curved surface structures (e.g. with fine concave parts).

Multi-frequency segmental bio-impedance device:design, development and applications

Bio-impedance analysis (BIA) provides a rapid, non-invasive technique for body composition estimation. BIA offers a convenient alternative to standard techniques such as MRI, CT scan or DEXA scan for selected types of body composition analysis. The accuracy of BIA is limited because it is an indirect method of composition analysis. It relies on linear relationships between measured impedance and morphological parameters such as height and weight to derive estimates. To overcome these underlying limitations of BIA, a multi-frequency segmental bio-impedance device was constructed through a series of iterative enhancements and improvements of existing BIA instrumentation. Key features of the design included an easy to construct current-source and compact PCB design. The final device was trialled with 22 human volunteers and measured impedance was compared against body composition estimates obtained by DEXA scan. This enabled the development of newer techniques to make BIA predictions. To add a ‘visual aspect’ to BIA, volunteers were scanned in 3D using an inexpensive scattered light gadget (Xbox Kinect controller) and 3D volumes of their limbs were compared with BIA measurements to further improve BIA predictions. A three-stage digital filtering scheme was also implemented to enable extraction of heart-rate data from recorded bio-electrical signals. Additionally modifications have been introduced to measure change in bio-impedance with motion, this could be adapted to further improve accuracy and veracity for limb composition analysis. The findings in this thesis aim to give new direction to the prediction of body composition using BIA. The design development and refinement applied to BIA in this research programme suggest new opportunities to enhance the accuracy and clinical utility of BIA for the prediction of body composition analysis. In particular, the use of bio-impedance to predict limb volumes which would provide an additional metric for body composition measurement and help distinguish between fat and muscle content.

3D navigation with six degrees-of-freedom using a multi-touch display

With the introduction of new input devices, such as multi-touch surface displays, the Nintendo WiiMote, the Microsoft Kinect, and the Leap Motion sensor, among others, the field of Human-Computer Interaction (HCI) finds itself at an important crossroads that requires solving new challenges. Given the amount of three-dimensional (3D) data available today, 3D navigation plays an important role in 3D User Interfaces (3DUI). This dissertation deals with multi-touch, 3D navigation, and how users can explore 3D virtual worlds using a multi-touch, non-stereo, desktop display. ^ The contributions of this dissertation include a feature-extraction algorithm for multi-touch displays (FETOUCH), a multi-touch and gyroscope interaction technique (GyroTouch), a theoretical model for multi-touch interaction using high-level Petri Nets (PeNTa), an algorithm to resolve ambiguities in the multi-touch gesture classification process (Yield), a proposed technique for navigational experiments (FaNS), a proposed gesture (Hold-and-Roll), and an experiment prototype for 3D navigation (3DNav). The verification experiment for 3DNav was conducted with 30 human-subjects of both genders. The experiment used the 3DNav prototype to present a pseudo-universe, where each user was required to find five objects using the multi-touch display and five objects using a game controller (GamePad). For the multi-touch display, 3DNav used a commercial library called GestureWorks in conjunction with Yield to resolve the ambiguity posed by the multiplicity of gestures reported by the initial classification. The experiment compared both devices. The task completion time with multi-touch was slightly shorter, but the difference was not statistically significant. The design of experiment also included an equation that determined the level of video game console expertise of the subjects, which was used to break down users into two groups: casual users and experienced users. The study found that experienced gamers performed significantly faster with the GamePad than casual users. When looking at the groups separately, casual gamers performed significantly better using the multi-touch display, compared to the GamePad. Additional results are found in this dissertation.^

Desenvolvimento de um sistema de captura de baixo custo destinado a determinação da cinemática humana

The motion capture is a main tool for quantitative motion analyses. Since the XIX century, several motion caption systems have been developed for biomechanics study, animations, games and movies. The biomechanics and kinesiology involves and depends on knowledge from distinct fields, the engineering and health sciences. A precise human motion analysis requires knowledge from both fields. It is necessary then the use of didactics tools and methods for research and teaching for learning aid. The devices for analysis and motion capture currently that are found on the market and on educational institutes presents difficulties for didactical practice, which are the difficulty of transportation, high cost and limited freedom for the user towards the data acquisition. Therefore, the motion analysis is qualitatively performed or is quantitatively performed in highly complex laboratories. Based is these problems, this work presents the development of a motion capture system for didactic use hence a cheap, light, portable and easily used device with a free software. This design includes the selection of the device, the software development for that and tests. The developed system uses the device Kinect, from Microsoft, for its low cost, low weight, portability and easy use, and delivery tree-dimensional data with only one peripheral device. The proposed programs use the hardware to make motion captures, store them, reproduce them, process the motion data and graphically presents the data.

Desenvolvimento e caracterização de nanoemulsões cosméticas contendo extrato de Opuntia ficus-indica L. Mill e avaliação in vivo da eficácia hidratante

Nanoemulsions are emulsified systems, characterized for reduced droplet size (50- 500nm), which the main characteristic are kinect stability and thermodynamic instability. These are promising systems on cosmetic area due to their droplet size that provide different advantages when compared to conventional systems, among others, larger surface area and better permeability. The Opuntia ficus-indica (L.) Mill is a plant cultivated on Caatinga Brazilian biome, which has great socioeconomic importance to region. This plant shows carbohydrates utilized for cosmetic industry as moisturizing active in their chemical composition. The aim of study was to develop, characterize, evaluate stability and moisturizing efficacy of cosmetic nanoemulsions added to Opuntia ficus-indica (L.) Mill extract. Nanoemulsions preparation was made using a low energy method. Different nanoemulsions were formulated varying the ratio of oil, water and surfactant phases beyond xanthan gum (0.5% e 1%) and Opuntia ficus-indica (L.) Mill hydroglycolic extract addition on 1% and 3%. Obtained nanoemulsions were submitted to preliminary and accelerated stability tests. The evaluated parameters monitored were: macroscopic aspect, pH value, droplet size, zeta potential and polydispersion index, during 60 days on different temperatures. Stable formulations were submitted to moisturizing efficacy assessment by capacitance and transepidermal water loss methodologies during 5 hours. Stable samples were white and showed homogeneous and fluid aspect, pH value was inside ideal range (4,5-6,0) to topical application and droplet size under 200nm characterizing these system as nanoemulsions. Developed nanoemulsions did not decrease transepidermal water loss, however increased the water content on stratum corneum, highlighting the nanoemulsions containing 0.5% of xanthan gum and 1% of hydroglycolic extract. This work presents cosmetic moisturizing nanoemulsions composed to vegetal raw material from Brazilian Caatinga with potential to be used on cosmetic area.

Registro global de nuvens de pontos RGB-D em tempo real usando fluxo óptico e marcadores

Registration of point clouds captured by depth sensors is an important task in 3D reconstruction applications based on computer vision. In many applications with strict performance requirements, the registration should be executed not only with precision, but also in the same frequency as data is acquired by the sensor. This thesis proposes theuse of the pyramidal sparse optical flow algorithm to incrementally register point clouds captured by RGB-D sensors (e.g. Microsoft Kinect) in real time. The accumulated errorinherent to the process is posteriorly minimized by utilizing a marker and pose graph optimization. Experimental results gathered by processing several RGB-D datasets validatethe system proposed by this thesis in visual odometry and simultaneous localization and mapping (SLAM) applications.

Atividade eletroencefalográfica de pacientes com acidente vascular cerebral na aprendizagem motora de um jogo baseado em realidade virtual

Stroke is the leading cause of long-term disability among adults and motor relearning is essential in motor sequelae recovery. Therefore, various techniques have been proposed to achieve this end, among them Virtual Reality. The aim of the study was to evaluate electroencephalographic activity of stroke patients in motor learning of a virtual reality-based game. The study included 10 patients with chronic stroke, right-hande; 5 with left brain injury (LP), mean age 48.8 years (± 4.76) and 5 with injury to the right (RP), mean age 52 years (± 10.93). Participants were evaluated for electroencephalographic (EEG) activity and performance while performing 15 repetitions of darts game in XBOX Kinect and also through the NIHSS, MMSE, Fugl-Meyer and the modified Ashworth scale. Patients underwent a trainning with 45 repetitions of virtual darts game, 12 sessions in four weeks. After training, patients underwent reassessment of EEG activity and performance in virtual game of darts (retention). Data were analyzed using ANOVA for repeated measures. According to the results, there were differences between the groups (PD and PE) in frequencies Low Alpha (p = 0.0001), High Alpha (p = 0.0001) and Beta (p = 0.0001). There was an increase in alpha activation powers and a decrease in beta in the phase retention of RP group. In LP group was observed increased alpha activation potency, but without decrease in beta activation. Considering the asymmetry score, RP group increased brain activation in the left hemisphere with the practice in the frontal areas, however, LP group had increased activation of the right hemisphere in fronto-central areas, temporal and parietal. As for performance, it was observed a decrease in absolute error in the game for RP group between assessment and retention (p = 0.015), but this difference was not observed for LP group (p = 0.135). It follows then that the right brain injury patients benefited more from darts game training in the virtual environment with respect to the motor learning process, reducing neural effort in ipsilesionais areas and errors with the practice of the task. In contrast, patients with lesions in left hemisphere decrease neural effort in contralesionais areas important for motor learning and showed no performance improvements with practice of 12 sessions of virtual dart game. Thus, the RV can be used in rehabilitation of stroke patients upper limb, but the laterality of the injury should be considered in programming the motor learning protocol.

Serious Game no tratamento de obesidade infantil: uma proposta de interação natural com monitoramento remoto

The number of overweight people has increased in the last few years. Factors such as attention to diet and changes in lifestyle are crucial in the prevention and control of obesity and diseases related to it. Experts believe that such actions are most effective when initiated during childhood, and that children raised in an environment that encourages physical activity ultimately become healthier adults. However, to arouse and maintain interest in such activities represent a major challenge, which are initially perceived as repetitive and boring, and, thus, soon abandoned. Computer games, traditionally seen as stimulants to a sedentary lifestyle are changing this perception using non-conventional controls that require constant movement of the player. Applications that combine the playfulness of such games to physical activity through devices, like Microsoft Kinect, might become interesting tools in this scenario, by using the familiarity of Natural User Interfaces along with the challenge and the fun of video games, in order to make attractive exercise routines for schoolchildren. The project carried out consists of an exergame composed of several activities designed and implemented with the participation of a Physical Educator, aimed at children between eight and ten years old, whose performance and progress can be remotely monitored by a professional via web interface. The application arising from this work was accompanied by tests with a group of graduating Physical Education students from the University of Rio Verde GO, and subsequently validated through questionnaires whose results are shown on this work.

3D Navigation with Six Degrees-of-Freedom using a Multi-Touch Display

With the introduction of new input devices, such as multi-touch surface displays, the Nintendo WiiMote, the Microsoft Kinect, and the Leap Motion sensor, among others, the field of Human-Computer Interaction (HCI) finds itself at an important crossroads that requires solving new challenges. Given the amount of three-dimensional (3D) data available today, 3D navigation plays an important role in 3D User Interfaces (3DUI). This dissertation deals with multi-touch, 3D navigation, and how users can explore 3D virtual worlds using a multi-touch, non-stereo, desktop display. The contributions of this dissertation include a feature-extraction algorithm for multi-touch displays (FETOUCH), a multi-touch and gyroscope interaction technique (GyroTouch), a theoretical model for multi-touch interaction using high-level Petri Nets (PeNTa), an algorithm to resolve ambiguities in the multi-touch gesture classification process (Yield), a proposed technique for navigational experiments (FaNS), a proposed gesture (Hold-and-Roll), and an experiment prototype for 3D navigation (3DNav). The verification experiment for 3DNav was conducted with 30 human-subjects of both genders. The experiment used the 3DNav prototype to present a pseudo-universe, where each user was required to find five objects using the multi-touch display and five objects using a game controller (GamePad). For the multi-touch display, 3DNav used a commercial library called GestureWorks in conjunction with Yield to resolve the ambiguity posed by the multiplicity of gestures reported by the initial classification. The experiment compared both devices. The task completion time with multi-touch was slightly shorter, but the difference was not statistically significant. The design of experiment also included an equation that determined the level of video game console expertise of the subjects, which was used to break down users into two groups: casual users and experienced users. The study found that experienced gamers performed significantly faster with the GamePad than casual users. When looking at the groups separately, casual gamers performed significantly better using the multi-touch display, compared to the GamePad. Additional results are found in this dissertation.

Redefinición de las TIC en el museo: del discurso invasivo al inclusivo

Las TIC son inseparables de la museografía in situ e imprescindibles en la museografía en red fija y móvil. En demasiados casos se han instalado prótesis tecnológicas para barnizar de modernidad el espacio cultural, olvidando que la tecnología debe estar al servicio de los contenidos de manera que resulte invisible y perfectamente imbricada con la museografía tradicional. Las interfaces móviles pueden fusionar museo in situ y en red y acompañar a las personas más allá del espacio físico. Esa fusión debe partir de una base de datos narrativa y abierta a obras materiales e inmateriales de otros museos de manera que no se trasladen las limitaciones del museo físico al virtual. En el museo in situ tienen sentido las instalaciones hipermedia inmersivas que faciliten experiencias culturales innovadoras. La interactividad (relaciones virtuales) debe convivir con la interacción (relaciones físicas y personales) y estar al servicio de todas las personas, partiendo de que todas, todos tenemos limitaciones. Trabajar interdisciplinarmente ayuda a comprender mejor el museo para ponerlo al servicio de las personas.