Efficient methods for stochastic optimal control

The Hamilton Jacobi Bellman (HJB) equation is central to stochastic optimal control (SOC) theory, yielding the optimal solution to general problems specified by known dynamics and a specified cost functional. Given the assumption of quadratic cost on the control input, it is well known that the HJB reduces to a particular partial differential equation (PDE). While powerful, this reduction is not commonly used as the PDE is of second order, is nonlinear, and examples exist where the problem may not have a solution in a classical sense. Furthermore, each state of the system appears as another dimension of the PDE, giving rise to the curse of dimensionality. Since the number of degrees of freedom required to solve the optimal control problem grows exponentially with dimension, the problem becomes intractable for systems with all but modest dimension.

In the last decade researchers have found that under certain, fairly non-restrictive structural assumptions, the HJB may be transformed into a linear PDE, with an interesting analogue in the discretized domain of Markov Decision Processes (MDP). The work presented in this thesis uses the linearity of this particular form of the HJB PDE to push the computational boundaries of stochastic optimal control.

This is done by crafting together previously disjoint lines of research in computation. The first of these is the use of Sum of Squares (SOS) techniques for synthesis of control policies. A candidate polynomial with variable coefficients is proposed as the solution to the stochastic optimal control problem. An SOS relaxation is then taken to the partial differential constraints, leading to a hierarchy of semidefinite relaxations with improving sub-optimality gap. The resulting approximate solutions are shown to be guaranteed over- and under-approximations for the optimal value function. It is shown that these results extend to arbitrary parabolic and elliptic PDEs, yielding a novel method for Uncertainty Quantification (UQ) of systems governed by partial differential constraints. Domain decomposition techniques are also made available, allowing for such problems to be solved via parallelization and low-order polynomials.

The optimization-based SOS technique is then contrasted with the Separated Representation (SR) approach from the applied mathematics community. The technique allows for systems of equations to be solved through a low-rank decomposition that results in algorithms that scale linearly with dimensionality. Its application in stochastic optimal control allows for previously uncomputable problems to be solved quickly, scaling to such complex systems as the Quadcopter and VTOL aircraft. This technique may be combined with the SOS approach, yielding not only a numerical technique, but also an analytical one that allows for entirely new classes of systems to be studied and for stability properties to be guaranteed.

The analysis of the linear HJB is completed by the study of its implications in application. It is shown that the HJB and a popular technique in robotics, the use of navigation functions, sit on opposite ends of a spectrum of optimization problems, upon which tradeoffs may be made in problem complexity. Analytical solutions to the HJB in these settings are available in simplified domains, yielding guidance towards optimality for approximation schemes. Finally, the use of HJB equations in temporal multi-task planning problems is investigated. It is demonstrated that such problems are reducible to a sequence of SOC problems linked via boundary conditions. The linearity of the PDE allows us to pre-compute control policy primitives and then compose them, at essentially zero cost, to satisfy a complex temporal logic specification.

In-Flight Collision Avoidance Controller Based Only on OS4 Embedded Sensors

The major goal of this research was the development and implementation of a control system able to avoid collisions during the flight for a mini-quadrotor helicopter, based only on its embedded sensors without changing the environment. However, it is important to highlight that the design aspects must be seriously considered in order to overcome hardware limitations and achieve control simplification. The controllers of a UAV (Unmanned Aerial Vehicle) robot deal with highly unstable dynamics and strong axes coupling. Furthermore, any additional embedded sensor increases the robot total weight and therefore, decreases its operating time. The best balance between embedded electronics and robot operating time is desired. This paper focuses not only on the development and implementation of a collision avoidance controller for a mini-robotic helicopter using only its embedded sensors, but also on the mathematical model that was essential for the controller developing phases. Based on this model we carried out the development of a simulation tool based on MatLab/Simulink that was fundamental for setting the controllers' parameters. This tool allowed us to simulate and improve the OS4 controllers in different modeled environments and test different approaches. After that, the controllers were embedded in the real robot and the results proved to be very robust and feasible. In addition to this, the controller has the advantage of being compatible with future path planners that we are developing.

Design and Prototyping High Endurance Multi-Rotor

The topic of this thesis fo cus on the preliminary design and the p erformance analysis of a multirotor platform. A multirotor is an electrically p owered Vertical Take Off (VTOL) machine with more than two rotors that lift and control the platform. Multirotor are agile, compact and robust, making them ideally suited for b oth indo or and outdo or application especially to carry-on several sensors like electro optical multisp ectral sensor or gas sensor. The main disadvantage is the limited endurance due to heavy Li-Po batteries and high disk loading through the use of different small prop ellers. At the same time, the design of the multirotor do es not follow any engineering principle but it follow the ideas of amateurs’ builder. An adaptation of the classic airplane design theory for the preliminary design is implemented to fill the gap and detailed study of the endurance is p erformed to define the right way to make this kind of VTOL platforms.

Studio e ottimizzazione di una configurazione mista ad ala rotante e fissa per la nuova versione del CH-47 Chinook

Studio e ottimizzazione di una configurazione mista ad ala rotante e fissa per un aeromobile VTOL CH-47 Chinook ibrido con ala.

On-board and Ground Visual Pose Estimation Techniques for UAV Control

In this paper, two techniques to control UAVs (Unmanned Aerial Vehicles), based on visual information are presented. The first one is based on the detection and tracking of planar structures from an on-board camera, while the second one is based on the detection and 3D reconstruction of the position of the UAV based on an external camera system. Both strategies are tested with a VTOL (Vertical take-off and landing) UAV, and results show good behavior of the visual systems (precision in the estimation and frame rate) when estimating the helicopter¿s position and using the extracted information to control the UAV.

An engineering modification of the blade element momentum combination applicable in vertical descent flight :fundamentals and validation

An engineering modification of blade element/momentum theory is applied to describe the vertical autorotation of helicopter rotors. A full non‐linear aerodynamic model is considered for the airfoils, taking into account the dependence of lift and drag coefficients on both the angle of attack and the Reynolds number. The proposed model, which has been validated in previous work, has allowed the identification of different autorotation modes, which depend on the descent velocity and the twist of the rotor blades. These modes present different radial distributions of driven and driving blade regions, as well as different radial upwash/downwash patterns. The number of blade sections with zero tangential force, the existence of a downwash region in the rotor disk, the stability of the autorotation state, and the overall rotor autorotation efficiency, are all analyzed in terms of the flight velocity and the characteristics of the rotor. It is shown that, in vertical autorotation, larger blade twist leads to smaller values of descent velocity for a given thrust generated by the rotor in the autorotational state.