Visual Tracking, Pose Estimation, and Control for Aerial Vehicles

El principal objetivo de esta tesis es dotar a los vehículos aéreos no tripulados (UAVs, por sus siglas en inglés) de una fuente de información adicional basada en visión. Esta fuente de información proviene de cámaras ubicadas a bordo de los vehículos o en el suelo. Con ella se busca que los UAVs realicen tareas de aterrizaje o inspección guiados por visión, especialmente en aquellas situaciones en las que no haya disponibilidad de estimar la posición del vehículo con base en GPS, cuando las estimaciones de GPS no tengan la suficiente precisión requerida por las tareas a realizar, o cuando restricciones de carga de pago impidan añadir sensores a bordo de los vehículos. Esta tesis trata con tres de las principales áreas de la visión por computador: seguimiento visual y estimación visual de la pose (posición y orientación), que a su vez constituyen la base de la tercera, denominada control servo visual, que en nuestra aplicación se enfoca en el empleo de información visual para controlar los UAVs. Al respecto, esta tesis se ocupa de presentar propuestas novedosas que permitan solucionar problemas relativos al seguimiento de objetos mediante cámaras ubicadas a bordo de los UAVs, se ocupa de la estimación de la pose de los UAVs basada en información visual obtenida por cámaras ubicadas en el suelo o a bordo, y también se ocupa de la aplicación de las técnicas propuestas para solucionar diferentes problemas, como aquellos concernientes al seguimiento visual para tareas de reabastecimiento autónomo en vuelo o al aterrizaje basado en visión, entre otros. Las diversas técnicas de visión por computador presentadas en esta tesis se proponen con el fin de solucionar dificultades que suelen presentarse cuando se realizan tareas basadas en visión con UAVs, como las relativas a la obtención, en tiempo real, de estimaciones robustas, o como problemas generados por vibraciones. Los algoritmos propuestos en esta tesis han sido probados con información de imágenes reales obtenidas realizando pruebas on-line y off-line. Diversos mecanismos de evaluación han sido empleados con el propósito de analizar el desempeño de los algoritmos propuestos, entre los que se incluyen datos simulados, imágenes de vuelos reales, estimaciones precisas de posición empleando el sistema VICON y comparaciones con algoritmos del estado del arte. Los resultados obtenidos indican que los algoritmos de visión por computador propuestos tienen un desempeño que es comparable e incluso mejor al de algoritmos que se encuentran en el estado del arte. Los algoritmos propuestos permiten la obtención de estimaciones robustas en tiempo real, lo cual permite su uso en tareas de control visual. El desempeño de estos algoritmos es apropiado para las exigencias de las distintas aplicaciones examinadas: reabastecimiento autónomo en vuelo, aterrizaje y estimación del estado del UAV. Abstract The main objective of this thesis is to provide Unmanned Aerial Vehicles (UAVs) with an additional vision-based source of information extracted by cameras located either on-board or on the ground, in order to allow UAVs to develop visually guided tasks, such as landing or inspection, especially in situations where GPS information is not available, where GPS-based position estimation is not accurate enough for the task to develop, or where payload restrictions do not allow the incorporation of additional sensors on-board. This thesis covers three of the main computer vision areas: visual tracking and visual pose estimation, which are the bases the third one called visual servoing, which, in this work, focuses on using visual information to control UAVs. In this sense, the thesis focuses on presenting novel solutions for solving the tracking problem of objects when using cameras on-board UAVs, on estimating the pose of the UAVs based on the visual information collected by cameras located either on the ground or on-board, and also focuses on applying these proposed techniques for solving different problems, such as visual tracking for aerial refuelling or vision-based landing, among others. The different computer vision techniques presented in this thesis are proposed to solve some of the frequently problems found when addressing vision-based tasks in UAVs, such as obtaining robust vision-based estimations at real-time frame rates, and problems caused by vibrations, or 3D motion. All the proposed algorithms have been tested with real-image data in on-line and off-line tests. Different evaluation mechanisms have been used to analyze the performance of the proposed algorithms, such as simulated data, images from real-flight tests, publicly available datasets, manually generated ground truth data, accurate position estimations using a VICON system and a robotic cell, and comparison with state of the art algorithms. Results show that the proposed computer vision algorithms obtain performances that are comparable to, or even better than, state of the art algorithms, obtaining robust estimations at real-time frame rates. This proves that the proposed techniques are fast enough for vision-based control tasks. Therefore, the performance of the proposed vision algorithms has shown to be of a standard appropriate to the different explored applications: aerial refuelling and landing, and state estimation. It is noteworthy that they have low computational overheads for vision systems.

Dynamic Response Optimization of Vehicles through Efficient Multibody Formulations and Automatic Differentiation Techniques

El diseño y desarrollo de sistemas de suspensión para vehículos se basa cada día más en el diseño por ordenador y en herramientas de análisis por ordenador, las cuales permiten anticipar problemas y resolverlos por adelantado. El comportamiento y las características dinámicas se calculan con precisión, bajo coste, y recursos y tiempos de cálculo reducidos. Sin embargo, existe una componente iterativa en el proceso, que requiere la definición manual de diseños a través de técnicas “prueba y error”. Esta Tesis da un paso hacia el desarrollo de un entorno de simulación eficiente capaz de simular, analizar y evaluar diseños de suspensiones vehiculares, y de mejorarlos hacia la solución optima mediante la modificación de los parámetros de diseño. La modelización mediante sistemas multicuerpo se utiliza aquí para desarrollar un modelo de autocar con 18 grados de libertad, de manera detallada y eficiente. La geometría y demás características de la suspensión se ajustan a las del vehículo real, así como los demás parámetros del modelo. Para simular la dinámica vehicular, se utiliza una formulación multicuerpo moderna y eficiente basada en las ecuaciones de Maggi, a la que se ha incorporado un visor 3D. Así, se consigue simular maniobras vehiculares en tiempos inferiores al tiempo real. Una vez que la dinámica está disponible, los análisis de sensibilidad son cruciales para una optimización robusta y eficiente. Para ello, se presenta una técnica matemática que permite derivar las variables dinámicas dentro de la formulación, de forma algorítmica, general, con la precisión de la maquina, y razonablemente eficiente: la diferenciación automática. Este método propaga las derivadas con respecto a las variables de diseño a través del código informático y con poca intervención del usuario. En contraste con otros enfoques en la bibliografía, generalmente particulares y limitados, se realiza una comparación de librerías, se desarrolla una formulación híbrida directa-automática para el cálculo de sensibilidades, y se presentan varios ejemplos reales. Finalmente, se lleva a cabo la optimización de la respuesta dinámica del vehículo citado. Se analizan cuatro tipos distintos de optimización: identificación de parámetros, optimización de la maniobrabilidad, optimización del confort y optimización multi-objetivo, todos ellos aplicados al diseño del autocar. Además de resultados analíticos y gráficos, se incluyen algunas consideraciones acerca de la eficiencia. En resumen, se mejora el comportamiento dinámico de vehículos por medio de modelos multicuerpo y de técnicas de diferenciación automática y optimización avanzadas, posibilitando un ajuste automático, preciso y eficiente de los parámetros de diseño. ABSTRACT Each day, the design and development of vehicle suspension systems relies more on computer-aided design and computer-aided engineering tools, which allow anticipating the problems and solving them ahead of time. Dynamic behavior and characteristics are thus simulated accurately and inexpensively with moderate computational times and resources. There is, however, an iterative component in the process, which involves the manual definition of designs in a trialand-error manner. This Thesis takes a step towards the development of an efficient simulation framework capable of simulating, analyzing and evaluating vehicle suspension designs, and automatically improving them by varying the design parameters towards the optimal solution. The multibody systems approach is hereby used to model a three-dimensional 18-degrees-of-freedom coach in a comprehensive yet efficient way. The suspension geometry and characteristics resemble the ones from the real vehicle, as do the rest of vehicle parameters. In order to simulate vehicle dynamics, an efficient, state-of-the-art multibody formulation based on Maggi’s equations is employed, and a three-dimensional graphics viewer is developed. As a result, vehicle maneuvers can be simulated faster than real-time. Once the dynamics are ready, a sensitivity analysis is crucial for a robust optimization. To that end, a mathematical technique is introduced, which allows differentiating the dynamic variables within the multibody formulation in a general, algorithmic, accurate to machine precision, and reasonably efficient way: automatic differentiation. This method propagates the derivatives with respect to the design parameters throughout the computer code, with little user interaction. In contrast with other attempts in the literature, mostly not generalpurpose, a benchmarking of libraries is carried out, a hybrid direct-automatic differentiation approach for the computation of sensitivities is developed, and several real-life examples are analyzed. Finally, a design optimization process of the aforementioned vehicle is carried out. Four different types of dynamic response optimization are presented: parameter identification, handling optimization, ride comfort optimization and multi-objective optimization; all of which are applied to the design of the coach example. Together with analytical and visual proof of the results, efficiency considerations are made. In summary, the dynamic behavior of vehicles is improved by using the multibody systems approach, along with advanced differentiation and optimization techniques, enabling an automatic, accurate and efficient tuning of design parameters.

Vibration reduction for vision systems on board unmanned aerial vehicles using a neuro-fuzzy controller

In this paper, an intelligent control approach based on neuro-fuzzy systems performance is presented, with the objective of counteracting the vibrations that affect the low-cost vision platform onboard an unmanned aerial system of rotating nature. A scaled dynamical model of a helicopter is used to simulate vibrations on its fuselage. The impact of these vibrations on the low-cost vision system will be assessed and an intelligent control approach will be derived in order to reduce its detrimental influence. Different trials that consider a neuro-fuzzy approach as a fundamental part of an intelligent semi-active control strategy have been carried out. Satisfactory results have been achieved compared to those obtained by means of vibration reduction passive techniques.

Nationwide impact and vehicle to grid application of electric vehicles mobility using an activity based model

This paper describes the impact of electric mobility on the transmission grid in Flanders region (Belgium), using a micro-simulation activity based models. These models are used to provide temporal and spatial estimation of energy and power demanded by electric vehicles (EVs) in different mobility zones. The increment in the load demand due to electric mobility is added to the background load demand in these mobility areas and the effects over the transmission substations are analyzed. From this information, the total storage capacity per zone is evaluated and some strategies for EV aggregator are proposed, allowing the aggregator to fulfill bids on the electricity markets.

Flexible low-cost system to test batteries and ultracapacitors for electric and hybrid vehicles in real working conditions

Batteries and ultracapacitors for hybrid and electric vehicles must satisfy very demanding working conditions that are not usual in other applications. In this sense, specific tests must be performed in order to draw accurate conclusions about their behaviour. To do so, new advanced test benches are needed. These platforms must allow the study of a wide variety of energy storage systems under conditions similar to the real ones. In this paper, a flexible, low-cost and highly customizable system is presented. This system allows batteries and ultracapacitors to be tested in many and varied ways, effectively emulating the working conditions that they face in an electric vehicle. The platform was specifically designed to study energy storage systems for electric and hybrid vehicles, meaning that it is suitable to test different systems in many different working conditions, including real driving cycles. This flexibility is achieved keeping the cost of the platform low, which makes the proposed test bench a feasible alternative for the industry. As an example of the functionality of the platform, a test consisting of a 17-minute ARTEMIS urban cycle with a NiMH battery pack is presented.

Energy Storage Systems for Electric Vehicles: Performance Comparison based on a Simple Equivalent Circuit and Experimental Tests

The decision to select the most suitable type of energy storage system for an electric vehicle is always difficult, since many conditionings must be taken into account. Sometimes, this study can be made by means of complex mathematical models which represent the behavior of a battery, ultracapacitor or some other devices. However, these models are usually too dependent on parameters that are not easily available, which usually results in nonrealistic results. Besides, the more accurate the model, the more specific it needs to be, which becomes an issue when comparing systems of different nature. This paper proposes a practical methodology to compare different energy storage technologies. This is done by means of a linear approach of an equivalent circuit based on laboratory tests. Via these tests, the internal resistance and the self-discharge rate are evaluated, making it possible to compare different energy storage systems regardless their technology. Rather simple testing equipment is sufficient to give a comparative idea of the differences between each system, concerning issues such as efficiency, heating and self-discharge, when operating under a certain scenario. The proposed methodology is applied to four energy storage systems of different nature for the sake of illustration.

Evaluation of the utilization of electric vehicles for building energy management in hotels

Governments are working in new policies to slow down total energy consumption and greenhouse gases (GHG) emissions, promoting the deployment of electric vehicles (EVs) in all countries. In order to facilitate this deployment and help to reduce the final costs of their batteries, additional utilization of EVs when those are parked has been proposed. EVs can be used to minimize the total electricity cost of buildings (named vehicle to building applications, V2B). In this paper an economic evaluation of EVs in the Building Energy Management System is shown. The optimal storage capacity and its equivalent number of EVs are determined. This value is then used for determining the optimal charging schedule to be applied to the batteries. From this schedule, the total expected profit is derived for the case of a real hotel in Spain.

Sensitivity analysis to assess the influence of the inertial properties of railway vehicle bodies on the vehicle’s dynamic behaviour

A sensitivity analysis has been performed to assess the influence of the inertial properties of railway vehicles on their dynamic behaviour. To do this, 216 dynamic simulations were performed modifying, one at a time, the masses, moments of inertia and heights of the centre of gravity of the carbody, the bogie and the wheelset. Three values were assigned to each parameter, corresponding to the percentiles 10, 50 and 90 of a data set stored in a database of railway vehicles. After processing the results of these simulations, the analyzed parameters were sorted by increasing influence. It was also found which of these parameters could be estimated with a lesser degree of accuracy for future simulations without appreciably affecting the simulation results. In general terms, it was concluded that the most sensitive inertial properties are the mass and the vertical moment of inertia, and the least sensitive ones the longitudinal and lateral moments of inertia.

Impact of introducing longer and heavier vehicles on regional consumer price index and Spanish road freight transport system

This paper applies an integrated modeling approach to the case of Spain; the approach is based on a random utility-based multiregional input-output model and a road transport network model for assessing the effect of introducing longer and heavier vehicles (LHVs) on the regional consumer price index (CPI) and on the transportation system. The approach strongly supports the concept that changes in transport costs derived from the LHV allowance as well as the economic structure of regions have direct and indirect effects on the economy and on the transportation system. Results show that the introduction of LHVs might reduce prices paid by consumers for a representative basket of goods and services in the regions of Spain and would also lead to a reduction in the regional CPI. In addition, the magnitude and extent of changes in the transportation system are estimated by using the commodity-based structure of the approach to identify the effect of traffic changes on traffic flows and on pollutant emissions over the whole network.

Methodology for kinematic cycle characterization of vehicles with fixed routes in urban areas

This paper analyses the driving cycles of a fleet of vehicles with predetermined urban itineraries. Most driving cycles developed for such type of vehicles do not properly address variability among itineraries. Here we develop a polygonal driving cycle that assesses each group of related routes, based on microscopic parameters. It measures the kinematic cycles of the routes traveled by the vehicle fleet, segments cycles into micro-cycles, and characterizes their properties, groups them into clusters with homogeneous kinematic characteristics within their specific micro-cycles, and constructs a standard cycle for each cluster. The process is used to study public bus operations in Madrid.

Fuel Cells: A Real Option for Unmanned Aerial Vehicles Propulsion

The possibility of implementing fuel cell technology in Unmanned Aerial Vehicle (UAV) propulsion systems is considered. Potential advantages of the Proton Exchange Membrane or Polymer Electrolyte Membrane (PEMFC) and Direct Methanol Fuel Cells (DMFC), their fuels (hydrogen and methanol), and their storage systems are revised from technical and environmental standpoints. Some operating commercial applications are described. Main constraints for these kinds of fuel cells are analyzed in order to elucidate the viability of future developments. Since the low power density is the main problem of fuel cells, hybridization with electric batteries, necessary in most cases, is also explored.

Nonlinear models for lateral dynamics of railway vehicles on bridges subject to wind action

The study of lateral dynamics of running trains on bridges is of importance mainly for the safety of the traffic, and may be relevant for laterally compliant bridges. These studies require threedimensional coupled vehicle-bridge models, wheree consideration of wheel to rail contact is a key aspect. Furthermore, an adequate evaluation of safety of rail traffic requires nonlinear models. A nonlinear coupled model is proposed here for vehicle-structure vertical and lateral dynamics. Vehicles are considered as fully three-dimensional multibody systems including gyroscopic terms and large rotation effects. The bridge structure is modeled by means of finite elements which may be of beam, shell or continuum type and may include geometric or material nonlinearities. The track geometry includes distributed track alignment irregularities. Both subsystems (bridge and vehicles) are described with coordinates in absolute reference frames, as opposed to alternative approaches which describe the multibody system with coordinates relative to the base bridge motion. The wheelrail contact employed is a semi-Hertzian model based on realistic wheel-rail profiles. It allows a detailed geometrical description of the contact patch under each wheel including multiple-point contact, flange contact and uplift. Normal and tangential stresses in each contact are integrated at each time-step to obtain the resultant contact forces. The models have been implemented within an existing finite element analysis software with multibody capabilities, Abaqus (Simulia Ltd., 2010). Further details of the model are presented in Antolín et al. (2012). Representative applications are presented for railway vehicles under lateral wind action on laterally compliant viaducts, showing the relevance of the nonlinear wheel-rail contact model as well as the interaction between bridge and vehicle.

A General Purpose Configurable Controller for Indoors and Outdoors GPS-Denied Navigation for Multirotor Unmanned Aerial Vehicles

This research on odometry based GPS-denied navigation on multirotor Unmanned Aerial Vehicles is focused among the interactions between the odometry sensors and the navigation controller. More precisely, we present a controller architecture that allows to specify a speed specified flight envelope where the quality of the odometry measurements is guaranteed. The controller utilizes a simple point mass kinematic model, described by a set of configurable parameters, to generate a complying speed plan. For experimental testing, we have used down-facing camera optical-flow as odometry measurement. This work is a continuation of prior research to outdoors environments using an AR Drone 2.0 vehicle, as it provides reliable optical flow on a wide range of flying conditions and floor textures. Our experiments show that the architecture is realiable for outdoors flight on altitudes lower than 9 m. A prior version of our code was utilized to compete in the International Micro Air Vehicle Conference and Flight Competition IMAV 2012. The code will be released as an open-source ROS stack hosted on GitHub.

Lateral Dynamic Models for High-Speed Railway Bridges and Vehicles

The analysis of the running safety of railway vehicles on viaducts subject to strong lateral actions such as cross winds requires coupled nonlinear vehicle-bridge interaction models, capable to study extreme events. In this paper original models developed by the authors are described, based on finite elements for the structure, multibody and finite element models for the vehicle, and specially developed interaction elements for the interface between wheel and rail. The models have been implemented within ABAQUS and have full nonlinear capabilities for the structure, the vehicle and the contact interface. An application is developed for the Ulla Viaduct, a 105 m tall arch in the Spanish high-speed railway network. The dynamic analyses allow obtaining critical wind curves, which define the running safety conditions for a given train in terms of speed of circulation and wind speed

Interregional trade changes in Spain caused by the introduction of a road fee charge for heavy goods vehicles

The introduction of a homogeneous road charging system according to the Directive 2011/76/EU for the use of roads is still under development in most European Union (EU) member states. Spain, like other EU members, has been encouraged to introduce a charging system for Heavy Goods Vehicles (HGVs) throughout the country. This nationwide charge has been postponed because there are serious concerns about their advantages from an economic point of view. Within this context, this paper applies an integrated modeling approach to shape elastic trade coefficients among regions by using a random utility based multiregional Input- Output (RUBMRIO) approach and a road transport network model in order to determine regional distributive and substitutive economic effects by simulating the introduction of a distance-based charge (?/km) considering 7,053.8 kilometers of free highways linking the capitals of the Spanish regions. In addition, an in-depth analysis of interregional trade changes is developed to evaluate and characterize the role of the road charging approach in trade relations among regions and across freight intensive economic sectors. For this purpose, differences in trade relations are described and assessed between a base-case or ?do nothing? scenario and a road fee-charge setting scenario. The results show that the specific amount of the charge set for HGVs affect each region differently and to a different extent because in some regions the price of commodities and the Generalized Transport Cost will decrease its competiveness within the country.