3D navigation with six degrees-of-freedom using a multi-touch display

With the introduction of new input devices, such as multi-touch surface displays, the Nintendo WiiMote, the Microsoft Kinect, and the Leap Motion sensor, among others, the field of Human-Computer Interaction (HCI) finds itself at an important crossroads that requires solving new challenges. Given the amount of three-dimensional (3D) data available today, 3D navigation plays an important role in 3D User Interfaces (3DUI). This dissertation deals with multi-touch, 3D navigation, and how users can explore 3D virtual worlds using a multi-touch, non-stereo, desktop display. ^ The contributions of this dissertation include a feature-extraction algorithm for multi-touch displays (FETOUCH), a multi-touch and gyroscope interaction technique (GyroTouch), a theoretical model for multi-touch interaction using high-level Petri Nets (PeNTa), an algorithm to resolve ambiguities in the multi-touch gesture classification process (Yield), a proposed technique for navigational experiments (FaNS), a proposed gesture (Hold-and-Roll), and an experiment prototype for 3D navigation (3DNav). The verification experiment for 3DNav was conducted with 30 human-subjects of both genders. The experiment used the 3DNav prototype to present a pseudo-universe, where each user was required to find five objects using the multi-touch display and five objects using a game controller (GamePad). For the multi-touch display, 3DNav used a commercial library called GestureWorks in conjunction with Yield to resolve the ambiguity posed by the multiplicity of gestures reported by the initial classification. The experiment compared both devices. The task completion time with multi-touch was slightly shorter, but the difference was not statistically significant. The design of experiment also included an equation that determined the level of video game console expertise of the subjects, which was used to break down users into two groups: casual users and experienced users. The study found that experienced gamers performed significantly faster with the GamePad than casual users. When looking at the groups separately, casual gamers performed significantly better using the multi-touch display, compared to the GamePad. Additional results are found in this dissertation.^

3D Navigation with Six Degrees-of-Freedom using a Multi-Touch Display

With the introduction of new input devices, such as multi-touch surface displays, the Nintendo WiiMote, the Microsoft Kinect, and the Leap Motion sensor, among others, the field of Human-Computer Interaction (HCI) finds itself at an important crossroads that requires solving new challenges. Given the amount of three-dimensional (3D) data available today, 3D navigation plays an important role in 3D User Interfaces (3DUI). This dissertation deals with multi-touch, 3D navigation, and how users can explore 3D virtual worlds using a multi-touch, non-stereo, desktop display. The contributions of this dissertation include a feature-extraction algorithm for multi-touch displays (FETOUCH), a multi-touch and gyroscope interaction technique (GyroTouch), a theoretical model for multi-touch interaction using high-level Petri Nets (PeNTa), an algorithm to resolve ambiguities in the multi-touch gesture classification process (Yield), a proposed technique for navigational experiments (FaNS), a proposed gesture (Hold-and-Roll), and an experiment prototype for 3D navigation (3DNav). The verification experiment for 3DNav was conducted with 30 human-subjects of both genders. The experiment used the 3DNav prototype to present a pseudo-universe, where each user was required to find five objects using the multi-touch display and five objects using a game controller (GamePad). For the multi-touch display, 3DNav used a commercial library called GestureWorks in conjunction with Yield to resolve the ambiguity posed by the multiplicity of gestures reported by the initial classification. The experiment compared both devices. The task completion time with multi-touch was slightly shorter, but the difference was not statistically significant. The design of experiment also included an equation that determined the level of video game console expertise of the subjects, which was used to break down users into two groups: casual users and experienced users. The study found that experienced gamers performed significantly faster with the GamePad than casual users. When looking at the groups separately, casual gamers performed significantly better using the multi-touch display, compared to the GamePad. Additional results are found in this dissertation.

Desenvolvimento de aplicações Web com base em modelagem 3D: módulo portal Web

This paper presents the development of a Web application called CityFreedom based on 3D modeling. The developed system demonstrates the use of most revolutionary and innovative techniques to create Web portals with the integrated 3D navigation scenarios to their own pages, without requiring any kind of plug-ins or external software. Everything works on the basis of compatible browsers. The CtyFreedom aims to give the user the feeling of immersion in virtual reality so get to interact with a three-dimensional city in order to see new places, traveling in an area of town that has always thought of knowing or even analyze establishments long before attend-them. It's the freedom to know and traveling around the city in a simple and trivial way. It is a new trend, the future of Web systems development

Hybrid coordination of reinforcement learning-based behaviors for AUV control

This paper proposes a hybrid coordination method for behavior-based control architectures. The hybrid method takes advantages of the robustness and modularity in competitive approaches as well as optimized trajectories in cooperative ones. This paper shows the feasibility of applying this hybrid method with a 3D-navigation to an autonomous underwater vehicle (AUV). The behaviors are learnt online by means of reinforcement learning. A continuous Q-learning implemented with a feed-forward neural network is employed. Realistic simulations were carried out. The results obtained show the good performance of the hybrid method on behavior coordination as well as the convergence of the behaviors

Intra- and postoperative complications of navigated and conventional techniques in percutaneous iliosacral screw fixation after pelvic fractures: Results from the German Pelvic Trauma Registry

Background: Percutaneous iliosacral screw placement following pelvic trauma is a very demanding technique involving a high rate of screw malpositions possibly associated with the risk of neurological damage or inadequate stability. In the conventional technique, the screw’s correct entry point and the small target corridor for the iliosacral screw may be difficult to visualise using an image intensifier. 2D and 3D navigation techniques may therefore be helpful tools. The aim of this multicentre study was to evaluate the intra- and postoperative complications after percutaneous screw implantation by classifying the fractures using data from a prospective pelvic trauma registry. The a priori hypothesis was that the navigation techniques have lower rates of intraoperative and postoperative complications. Methods: This study is based on data from the prospective pelvic trauma registry introduced by the German Society of Traumatology and the German Section of the AO/ASIF International in 1991. The registry provides data on all patients with pelvic fractures treated between July 2008 and June 2011 at any one of the 23 Level I trauma centres contributing to the registry. Results: A total of 2615 patients were identified. Out of these a further analysis was performed in 597 patients suffering injuries of the SI joint (187 � with surgical interventions) and 597 patients with sacral fractures (334 � with surgical interventions). The rate of intraoperative complications was not significantly different, with 10/114 patients undergoing navigated techniques (8.8%) and 14/239 patients in the conventional group (5.9%) for percutaneous screw implantation (p = 0.4242). Postoperative complications were analysed in 30/114 patients in the navigated group (26.3%) and in 70/239 patients (29.3%) in the conventional group (p = 0.6542). Patients who underwent no surgery had with 66/197 cases (33.5%) a relatively high rate of complications during their hospital stay. The rate of surgically-treated fractures was higher in the group with more unstable Type-C fractures, but the fracture classification had no significant influence on the rate of complications. Discussion: In this prospective multicentre study, the 2D/3D navigation techniques revealed similar results for the rate of intraoperative and postoperative complications compared to the conventional technique. The rate of neurological complications was significantly higher in the navigated group.

Efficient goal directed navigation using RatSLAM

RatSLAM is a system for vision based Simultaneous Localization and Mapping (SLAM) that has been shown to be capable of building stable representations of real world environments. In this paper we describe a method for using RatSLAM representations as the basis for navigation to designated goal locations. The method uses a new component, goal memory, to learn the temporal gradient between places. Paths are recalled or inferred from the goal memory by following the temporal gradient from the robot’s current position to the goal location. Experimental results have been gathered in a combined office and laboratory environment using a Pioneer robot. The experiments show that the robot can perform vision based SLAM on-line and in real time, and then use those representations immediately to navigate directly to designated goal locations.

Nonparametric semantic segmentation for 3D street scenes

In this paper we propose a method to generate a large scale and accurate dense 3D semantic map of street scenes. A dense 3D semantic model of the environment can significantly improve a number of robotic applications such as autonomous driving, navigation or localisation. Instead of using offline trained classifiers for semantic segmentation, our approach employs a data-driven, nonparametric method to parse scenes which easily scale to a large environment and generalise to different scenes. We use stereo image pairs collected from cameras mounted on a moving car to produce dense depth maps which are combined into a global 3D reconstruction using camera poses from stereo visual odometry. Simultaneously, 2D automatic semantic segmentation using a nonparametric scene parsing method is fused into the 3D model. Furthermore, the resultant 3D semantic model is improved with the consideration of moving objects in the scene. We demonstrate our method on the publicly available KITTI dataset and evaluate the performance against manually generated ground truth.

Vision-only autonomous navigation using topometric maps

This paper presents a mapping and navigation system for a mobile robot, which uses vision as its sole sensor modality. The system enables the robot to navigate autonomously, plan paths and avoid obstacles using a vision based topometric map of its environment. The map consists of a globally-consistent pose-graph with a local 3D point cloud attached to each of its nodes. These point clouds are used for direction independent loop closure and to dynamically generate 2D metric maps for locally optimal path planning. Using this locally semi-continuous metric space, the robot performs shortest path planning instead of following the nodes of the graph --- as is done with most other vision-only navigation approaches. The system exploits the local accuracy of visual odometry in creating local metric maps, and uses pose graph SLAM, visual appearance-based place recognition and point clouds registration to create the topometric map. The ability of the framework to sustain vision-only navigation is validated experimentally, and the system is provided as open-source software.

Long-term experiments with an adaptive spherical view representation for navigation in changing environments

Real-world environments such as houses and offices change over time, meaning that a mobile robot’s map will become out of date. In this work, we introduce a method to update the reference views in a hybrid metrictopological map so that a mobile robot can continue to localize itself in a changing environment. The updating mechanism, based on the multi-store model of human memory, incorporates a spherical metric representation of the observed visual features for each node in the map, which enables the robot to estimate its heading and navigate using multi-view geometry, as well as representing the local 3D geometry of the environment. A series of experiments demonstrate the persistence performance of the proposed system in real changing environments, including analysis of the long-term stability.

Biologically inspired visual odometer for navigation of a flying robot

Experimental research in biology has uncovered a number of different ways in which flying insects use cues derived from optical flow for navigational purposes, such as safe landing, obstacle avoidance and dead reckoning. In this study, we use a synthetic methodology to gain additional insights into the navigation behavior of bees. Specifically, we focus on the mechanisms of course stabilization behavior and visually mediated odometer by using a biological model of motion detector for the purpose of long-range goal-directed navigation in 3D environment. The performance tests of the proposed navigation method are conducted by using a blimp-type flying robot platform in uncontrolled indoor environments. The result shows that the proposed mechanism can be used for goal-directed navigation. Further analysis is also conducted in order to enhance the navigation performance of autonomous aerial vehicles. © 2003 Elsevier B.V. All rights reserved.

Electromagnetic tracking and steering for catheter navigation

This thesis explores the use of electromagnetics for both steering and tracking of medical instruments in minimally invasive surgeries. The end application is virtual navigation of the lung for biopsy of early stage cancer nodules. Navigation to the peripheral regions of the lung is difficult due to physical dimensions of the bronchi and current methods have low successes rates for accurate diagnosis. Firstly, the potential use of DC magnetic fields for the actuation of catheter devices with permanently magnetised distal attachments is investigated. Catheter models formed from various materials and magnetic tip formations are used to examine the usefulness of relatively low power and compact electromagnets. The force and torque that can be exerted on a small permanent magnet is shown to be extremely limited. Hence, after this initial investigation we turn our attention to electromagnetic tracking, in the development of a novel, low-cost implementation of a GPS-like system for navigating within a patient. A planar magnetic transmitter, formed on a printed circuit board for a low-profile and low cost manufacture, is used to generate a low frequency magnetic field distribution which is detected by a small induction coil sensor. The field transmitter is controlled by a novel closed-loop system that ensures a highly stable magnetic field with reduced interference from one transmitter coil to another. Efficient demodulation schemes are presented which utilise synchronous detection of each magnetic field component experienced by the sensor. The overall tracking accuracy of the system is shown to be less than 2 mm with an orientation error less than 1°. A novel demodulation implementation using a unique undersampling approach allows the use of reduced sample rates to sample the signals of interest without loss of tracking accuracy. This is advantageous for embedded microcontroller implementations of EM tracking systems. The EM tracking system is demonstrated in the pre-clinical environment of a breathing lung phantom. The airways of the phantom are successfully navigated using the system in combination with a 3D computer model rendered from CT data. Registration is achieved using both a landmark rigid registration method and a hybrid fiducial-free approach. The design of a planar magnetic shield structure for blocking the effects of metallic distortion from below the transmitter is presented which successfully blocks the impact of large ferromagnetic objects such as operating tables. A variety of shielding material are analysed with MuMetal and ferrite both providing excellent shieling performance and an increased signal to noise ratio. Finally, the effect of conductive materials and human tissue on magnetic field measurements is presented. Error due to induced eddy currents and capacitive coupling is shown to severely affect EM tracking accuracy at higher frequencies.

Application of Visual-Inertial SLAM for 3D Mapping of Underground Environments

The underground scenarios are one of the most challenging environments for accurate and precise 3d mapping where hostile conditions like absence of Global Positioning Systems, extreme lighting variations and geometrically smooth surfaces may be expected. So far, the state-of-the-art methods in underground modelling remain restricted to environments in which pronounced geometric features are abundant. This limitation is a consequence of the scan matching algorithms used to solve the localization and registration problems. This paper contributes to the expansion of the modelling capabilities to structures characterized by uniform geometry and smooth surfaces, as is the case of road and train tunnels. To achieve that, we combine some state of the art techniques from mobile robotics, and propose a method for 6DOF platform positioning in such scenarios, that is latter used for the environment modelling. A visual monocular Simultaneous Localization and Mapping (MonoSLAM) approach based on the Extended Kalman Filter (EKF), complemented by the introduction of inertial measurements in the prediction step, allows our system to localize himself over long distances, using exclusively sensors carried on board a mobile platform. By feeding the Extended Kalman Filter with inertial data we were able to overcome the major problem related with MonoSLAM implementations, known as scale factor ambiguity. Despite extreme lighting variations, reliable visual features were extracted through the SIFT algorithm, and inserted directly in the EKF mechanism according to the Inverse Depth Parametrization. Through the 1-Point RANSAC (Random Sample Consensus) wrong frame-to-frame feature matches were rejected. The developed method was tested based on a dataset acquired inside a road tunnel and the navigation results compared with a ground truth obtained by post-processing a high grade Inertial Navigation System and L1/L2 RTK-GPS measurements acquired outside the tunnel. Results from the localization strategy are presented and analyzed.

Large-Area Photo-Mosaics Using Global Alignment and Navigation Data

Seafloor imagery is a rich source of data for the study of biological and geological processes. Among several applications, still images of the ocean floor can be used to build image composites referred to as photo-mosaics. Photo-mosaics provide a wide-area visual representation of the benthos, and enable applications as diverse as geological surveys, mapping and detection of temporal changes in the morphology of biodiversity. We present an approach for creating globally aligned photo-mosaics using 3D position estimates provided by navigation sensors available in deep water surveys. Without image registration, such navigation data does not provide enough accuracy to produce useful composite images. Results from a challenging data set of the Lucky Strike vent field at the Mid Atlantic Ridge are reported

Accuracy estimation of a new omnidirectional 3D vision sensor

We present a computer vision system that associates omnidirectional vision with structured light with the aim of obtaining depth information for a 360 degrees field of view. The approach proposed in this article combines an omnidirectional camera with a panoramic laser projector. The article shows how the sensor is modelled and its accuracy is proved by means of experimental results. The proposed sensor provides useful information for robot navigation applications, pipe inspection, 3D scene modelling etc