On precise three-dimensional environment modeling via UAV-based photogrammetric systems

Abstract : Images acquired from unmanned aerial vehicles (UAVs) can provide data with unprecedented spatial and temporal resolution for three-dimensional (3D) modeling. Solutions developed for this purpose are mainly operating based on photogrammetry concepts, namely UAV-Photogrammetry Systems (UAV-PS). Such systems are used in applications where both geospatial and visual information of the environment is required. These applications include, but are not limited to, natural resource management such as precision agriculture, military and police-related services such as traffic-law enforcement, precision engineering such as infrastructure inspection, and health services such as epidemic emergency management. UAV-photogrammetry systems can be differentiated based on their spatial characteristics in terms of accuracy and resolution. That is some applications, such as precision engineering, require high-resolution and high-accuracy information of the environment (e.g. 3D modeling with less than one centimeter accuracy and resolution). In other applications, lower levels of accuracy might be sufficient, (e.g. wildlife management needing few decimeters of resolution). However, even in those applications, the specific characteristics of UAV-PSs should be well considered in the steps of both system development and application in order to yield satisfying results. In this regard, this thesis presents a comprehensive review of the applications of unmanned aerial imagery, where the objective was to determine the challenges that remote-sensing applications of UAV systems currently face. This review also allowed recognizing the specific characteristics and requirements of UAV-PSs, which are mostly ignored or not thoroughly assessed in recent studies. Accordingly, the focus of the first part of this thesis is on exploring the methodological and experimental aspects of implementing a UAV-PS. The developed system was extensively evaluated for precise modeling of an open-pit gravel mine and performing volumetric-change measurements. This application was selected for two main reasons. Firstly, this case study provided a challenging environment for 3D modeling, in terms of scale changes, terrain relief variations as well as structure and texture diversities. Secondly, open-pit-mine monitoring demands high levels of accuracy, which justifies our efforts to improve the developed UAV-PS to its maximum capacities. The hardware of the system consisted of an electric-powered helicopter, a high-resolution digital camera, and an inertial navigation system. The software of the system included the in-house programs specifically designed for camera calibration, platform calibration, system integration, onboard data acquisition, flight planning and ground control point (GCP) detection. The detailed features of the system are discussed in the thesis, and solutions are proposed in order to enhance the system and its photogrammetric outputs. The accuracy of the results was evaluated under various mapping conditions, including direct georeferencing and indirect georeferencing with different numbers, distributions and types of ground control points. Additionally, the effects of imaging configuration and network stability on modeling accuracy were assessed. The second part of this thesis concentrates on improving the techniques of sparse and dense reconstruction. The proposed solutions are alternatives to traditional aerial photogrammetry techniques, properly adapted to specific characteristics of unmanned, low-altitude imagery. Firstly, a method was developed for robust sparse matching and epipolar-geometry estimation. The main achievement of this method was its capacity to handle a very high percentage of outliers (errors among corresponding points) with remarkable computational efficiency (compared to the state-of-the-art techniques). Secondly, a block bundle adjustment (BBA) strategy was proposed based on the integration of intrinsic camera calibration parameters as pseudo-observations to Gauss-Helmert model. The principal advantage of this strategy was controlling the adverse effect of unstable imaging networks and noisy image observations on the accuracy of self-calibration. The sparse implementation of this strategy was also performed, which allowed its application to data sets containing a lot of tie points. Finally, the concepts of intrinsic curves were revisited for dense stereo matching. The proposed technique could achieve a high level of accuracy and efficiency by searching only through a small fraction of the whole disparity search space as well as internally handling occlusions and matching ambiguities. These photogrammetric solutions were extensively tested using synthetic data, close-range images and the images acquired from the gravel-pit mine. Achieving absolute 3D mapping accuracy of 11±7 mm illustrated the success of this system for high-precision modeling of the environment.

CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION OF VEHICLE MOTION

Simultaneous Localization and Mapping (SLAM) is a procedure used to determine the location of a mobile vehicle in an unknown environment, while constructing a map of the unknown environment at the same time. Mobile platforms, which make use of SLAM algorithms, have industrial applications in autonomous maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. A typical SLAM consists of four main components, namely, experimental setup (data gathering), vehicle pose estimation, feature extraction, and filtering. Feature extraction is the process of realizing significant features from the unknown environment such as corners, edges, walls, and interior features. In this work, an original feature extraction algorithm specific to distance measurements obtained through SONAR sensor data is presented. This algorithm has been constructed by combining the SONAR Salient Feature Extraction Algorithm and the Triangulation Hough Based Fusion with point-in-polygon detection. The reconstructed maps obtained through simulations and experimental data with the fusion algorithm are compared to the maps obtained with existing feature extraction algorithms. Based on the results obtained, it is suggested that the proposed algorithm can be employed as an option for data obtained from SONAR sensors in environment, where other forms of sensing are not viable. The algorithm fusion for feature extraction requires the vehicle pose estimation as an input, which is obtained from a vehicle pose estimation model. For the vehicle pose estimation, the author uses sensor integration to estimate the pose of the mobile vehicle. Different combinations of these sensors are studied (e.g., encoder, gyroscope, or encoder and gyroscope). The different sensor fusion techniques for the pose estimation are experimentally studied and compared. The vehicle pose estimation model, which produces the least amount of error, is used to generate inputs for the feature extraction algorithm fusion. In the experimental studies, two different environmental configurations are used, one without interior features and another one with two interior features. Numerical and experimental findings are discussed. Finally, the SLAM algorithm is implemented along with the algorithms for feature extraction and vehicle pose estimation. Three different cases are experimentally studied, with the floor of the environment intentionally altered to induce slipping. Results obtained for implementations with and without SLAM are compared and discussed. The present work represents a step towards the realization of autonomous inspection platforms for performing concurrent localization and mapping in harsh environments.

Electro-Optical/Infrared sensor turret integration on an aircraft - structural im-pact on LOCKHEED MARTIN C-130H and design methodology

The growing need to patrol and survey large maritime and terrestrial areas increased the need to integrate external sensors on aircraft in order to accomplish those patrols at increasingly higher altitudes, longer range and not depending upon vehicle type. The main focus of this work is to elaborate a practical, simple, effective and efficient methodology for the aircraft modification procedure resulting from the integration of an Elec-tro-Optical/Infra-Red (EO/IR) turret through a support structure. The importance of the devel-opment of a good methodology relies on the correct management of project variables as time, available resources and project complexity. The key is to deliver a proper tool for a project de-sign team that will be used to create a solution that fulfils all technical, non-technical and certi-fication requirements present in this field of transportation. The created methodology is inde-pendent of two main inputs: sensor model and aircraft model definition, and therefore it is in-tended to deliver the results for different projects besides the one that was presented in this work as a case study. This particular case study presents the development of a structure support for FLIR STAR SAPHIRE III turret integration on the front lower fuselage bulkhead (radome) of the LOCKHEED MARTIN C-130 H. Development of the case study focuses on the study of local structural analysis through the use of Finite Element Method (FEM). Development of this Dissertation resulted in a cooperation between Faculty of Science and Technology - Universidade Nova de Lisboa and the company OGMA - Indústria Aeronáutica de Portugal

jxSensor: a sensor network integration layer for JXTA

Nowadays, Wireless Sensor Networks (WSN) arealready a very important data source to obtain data about the environment. Thus, they are key to the creation of Cyber-Physical Systems (CPS). Given the popularity of P2P middlewares as ameans to efficiently process information and distribute services, being able to integrate them to WSN¿s is an interesting proposal. JXTA is a widely used P2P middleware that allows peers to easily exchange information, heavily relying on its main architectural highlight, the capability to organize peers with common interests into peer groups. However, right now, approaches to integrate WSNs to a JXTA network seldom take advantage of peer groups. For this reason, in this paper we present jxSensor, an integrationlayer for sensor motes which facilitates the deployment of CPS¿s under this architecture. This integration has been done taking into account JXTA¿s idiosyncrasies and proposing novel ideas,such as the Virtual Peer, a group of sensors that acts as a single entity within the peer group context.

Integration of wireless sensor networks to the internet of things using a 6LoWPAN Gateway

The Internet of Things is a new paradigm where smart embedded devices and systems are connected to the Internet. In this context, Wireless Sensor Networks (WSN) are becoming an important alternative for sensing and actuating critical applications like industrial automation, remote patient monitoring and domotics. The IEEE 802.15.4 protocol has been adopted as a standard for WSN and the 6LoWPAN protocol has been proposed to overcome the challenges of integrating WSN and Internet protocols. In this paper, the mechanisms of header compression and fragmentation of IPv6 datagrams proposed in the 6LoWPAN standard were evaluated through field experiments using a gateway prototype and IEEE 802.15.4 nodes.

Integration of wearable devices in a wireless sensor network for an e-health application

Applications based on Wireless Sensor Networks for Internet of Things scenarios are on the rise. The multiple possibilities they offer have spread towards previously hard to imagine fields, like e-health or human physiological monitoring. An application has been developed for its usage in scenarios where data collection is applied to smart spaces, aiming at its usage in fire fighting and sports. This application has been tested in a gymnasium with real, non-simulated nodes and devices. A Graphic User Interface has been implemented to suggest a series of exercises to improve a sportsman/woman s condition, depending on the context and their profile. This system can be adapted to a wide variety of e-health applications with minimum changes, and the user will interact using different devices, like smart phones, smart watches and/or tablets.

Integration of low-cost supervisory mobile robots in domestic wireless sensor networks

This paper presents a communication interface between supervisory low-cost mobile robots and domestic Wireless Sensor Network (WSN) based on the Zig Bee protocol from different manufacturers. The communication interface allows control and communication with other network devices using the same protocol. The robot can receive information from sensor devices (temperature, humidity, luminosity) and send commands to actuator devices (lights, shutters, thermostats) from different manufacturers. The architecture of the system, the interfaces and devices needed to establish the communication are described in the paper.