Formal methods for control synthesis in partially observed environments : application to autonomous robotic manipulation

Modern robots are increasingly expected to function in uncertain and dynamically challenging environments, often in proximity with humans. In addition, wide scale adoption of robots requires on-the-fly adaptability of software for diverse application. These requirements strongly suggest the need to adopt formal representations of high level goals and safety specifications, especially as temporal logic formulas. This approach allows for the use of formal verification techniques for controller synthesis that can give guarantees for safety and performance. Robots operating in unstructured environments also face limited sensing capability. Correctly inferring a robot's progress toward high level goal can be challenging.

This thesis develops new algorithms for synthesizing discrete controllers in partially known environments under specifications represented as linear temporal logic (LTL) formulas. It is inspired by recent developments in finite abstraction techniques for hybrid systems and motion planning problems. The robot and its environment is assumed to have a finite abstraction as a Partially Observable Markov Decision Process (POMDP), which is a powerful model class capable of representing a wide variety of problems. However, synthesizing controllers that satisfy LTL goals over POMDPs is a challenging problem which has received only limited attention.

This thesis proposes tractable, approximate algorithms for the control synthesis problem using Finite State Controllers (FSCs). The use of FSCs to control finite POMDPs allows for the closed system to be analyzed as finite global Markov chain. The thesis explicitly shows how transient and steady state behavior of the global Markov chains can be related to two different criteria with respect to satisfaction of LTL formulas. First, the maximization of the probability of LTL satisfaction is related to an optimization problem over a parametrization of the FSC. Analytic computation of gradients are derived which allows the use of first order optimization techniques.

The second criterion encourages rapid and frequent visits to a restricted set of states over infinite executions. It is formulated as a constrained optimization problem with a discounted long term reward objective by the novel utilization of a fundamental equation for Markov chains - the Poisson equation. A new constrained policy iteration technique is proposed to solve the resulting dynamic program, which also provides a way to escape local maxima.

The algorithms proposed in the thesis are applied to the task planning and execution challenges faced during the DARPA Autonomous Robotic Manipulation - Software challenge.

High pressure states in condensed matter: I. High pressure behavior of the iron-sulphur system with applications to the earth's core. II. Empirical equation of state for organic compounds at high pressures

Part I:

The earth's core is generally accepted to be composed primarily of iron, with an admixture of other elements. Because the outer core is observed not to transmit shear waves at seismic frequencies, it is known to be liquid or primarily liquid. A new equation of state is presented for liquid iron, in the form of parameters for the 4th order Birch-Murnaghan and Mie-Grüneisen equations of state. The parameters were constrained by a set of values for numerous properties compiled from the literature. A detailed theoretical model is used to constrain the P-T behavior of the heat capacity, based on recent advances in the understanding of the interatomic potentials for transition metals. At the reference pressure of 10⁵ Pa and temperature of 1811 K (the normal melting point of Fe), the parameters are: ρ = 7037 kg/m³, K_S0 = 110 GPa, K_S' = 4.53, K_S" = -.0337 GPa-1, and γ = 2.8, with γ α ρ^-1.17. Comparison of the properties predicted by this model with the earth model PREM indicates that the outer core is 8 to 10 % less dense than pure liquid Fe at the same conditions. The inner core is also found to be 3 to 5% less dense than pure liquid Fe, supporting the idea of a partially molten inner core. The density deficit of the outer core implies that the elements dissolved in the liquid Fe are predominantly of lower atomic weight than Fe. Of the candidate light elements favored by researchers, only sulfur readily dissolves into Fe at low pressure, which means that this element was almost certainly concentrated in the core at early times. New melting data are presented for FeS and FeS₂ which indicate that the FeS₂ is the S-hearing liquidus solid phase at inner core pressures. Consideration of the requirement that the inner core boundary be observable by seismological means and the freezing behavior of solutions leads to the possibility that the outer core may contain a significant fraction of solid material. It is found that convection in the outer core is not hindered if the solid particles are entrained in the fluid flow. This model for a core of Fe and S admits temperatures in the range 3450K to 4200K at the top of the core. An all liquid Fe-S outer core would require a temperature of about 4900 K at the top of the core.

Part II.

The abundance of uses for organic compounds in the modern world results in many applications in which these materials are subjected to high pressures. This leads to the desire to be able to describe the behavior of these materials under such conditions. Unfortunately, the number of compounds is much greater than the number of experimental data available for many of the important properties. In the past, one approach that has worked well is the calculation of appropriate properties by summing the contributions from the organic functional groups making up molecules of the compounds in question. A new set of group contributions for the molar volume, volume thermal expansivity, heat capacity, and the Rao function is presented for functional groups containing C, H, and O. This set is, in most cases, limited in application to low molecular liquids. A new technique for the calculation of the pressure derivative of the bulk modulus is also presented. Comparison with data indicates that the presented technique works very well for most low molecular hydrocarbon liquids and somewhat less well for oxygen-bearing compounds. A similar comparison of previous results for polymers indicates that the existing tabulations of group contributions for this class of materials is in need of revision. There is also evidence that the Rao function contributions for polymers and low molecular compounds are somewhat different.

Battery-Powered RF Pre-Ionization System for the Caltech Magnetohydrodynamically-Driven Jet Experiment: RF Discharge Properties and MHD-Driven Jet Dynamics

This thesis describes investigations of two classes of laboratory plasmas with rather different properties: partially ionized low pressure radiofrequency (RF) discharges, and fully ionized high density magnetohydrodynamically (MHD)-driven jets. An RF pre-ionization system was developed to enable neutral gas breakdown at lower pressures and create hotter, faster jets in the Caltech MHD-Driven Jet Experiment. The RF plasma source used a custom pulsed 3 kW 13.56 MHz RF power amplifier that was powered by AA batteries, allowing it to safely float at 4-6 kV with the cathode of the jet experiment. The argon RF discharge equilibrium and transport properties were analyzed, and novel jet dynamics were observed.

Although the RF plasma source was conceived as a wave-heated helicon source, scaling measurements and numerical modeling showed that inductive coupling was the dominant energy input mechanism. A one-dimensional time-dependent fluid model was developed to quantitatively explain the expansion of the pre-ionized plasma into the jet experiment chamber. The plasma transitioned from an ionizing phase with depressed neutral emission to a recombining phase with enhanced emission during the course of the experiment, causing fast camera images to be a poor indicator of the density distribution. Under certain conditions, the total visible and infrared brightness and the downstream ion density both increased after the RF power was turned off. The time-dependent emission patterns were used for an indirect measurement of the neutral gas pressure.

The low-mass jets formed with the aid of the pre-ionization system were extremely narrow and collimated near the electrodes, with peak density exceeding that of jets created without pre-ionization. The initial neutral gas distribution prior to plasma breakdown was found to be critical in determining the ultimate jet structure. The visible radius of the dense central jet column was several times narrower than the axial current channel radius, suggesting that the outer portion of the jet must have been force free, with the current parallel to the magnetic field. The studies of non-equilibrium flows and plasma self-organization being carried out at Caltech are relevant to astrophysical jets and fusion energy research.

Distributed Optimal Control of Cyber-Physical Systems: Controller Synthesis, Architecture Design and System Identification

The centralized paradigm of a single controller and a single plant upon which modern control theory is built is no longer applicable to modern cyber-physical systems of interest, such as the power-grid, software defined networks or automated highways systems, as these are all large-scale and spatially distributed. Both the scale and the distributed nature of these systems has motivated the decentralization of control schemes into local sub-controllers that measure, exchange and act on locally available subsets of the globally available system information. This decentralization of control logic leads to different decision makers acting on asymmetric information sets, introduces the need for coordination between them, and perhaps not surprisingly makes the resulting optimal control problem much harder to solve. In fact, shortly after such questions were posed, it was realized that seemingly simple decentralized optimal control problems are computationally intractable to solve, with the Wistenhausen counterexample being a famous instance of this phenomenon. Spurred on by this perhaps discouraging result, a concerted 40 year effort to identify tractable classes of distributed optimal control problems culminated in the notion of quadratic invariance, which loosely states that if sub-controllers can exchange information with each other at least as quickly as the effect of their control actions propagates through the plant, then the resulting distributed optimal control problem admits a convex formulation.

The identification of quadratic invariance as an appropriate means of "convexifying" distributed optimal control problems led to a renewed enthusiasm in the controller synthesis community, resulting in a rich set of results over the past decade. The contributions of this thesis can be seen as being a part of this broader family of results, with a particular focus on closing the gap between theory and practice by relaxing or removing assumptions made in the traditional distributed optimal control framework. Our contributions are to the foundational theory of distributed optimal control, and fall under three broad categories, namely controller synthesis, architecture design and system identification.

We begin by providing two novel controller synthesis algorithms. The first is a solution to the distributed H-infinity optimal control problem subject to delay constraints, and provides the only known exact characterization of delay-constrained distributed controllers satisfying an H-infinity norm bound. The second is an explicit dynamic programming solution to a two player LQR state-feedback problem with varying delays. Accommodating varying delays represents an important first step in combining distributed optimal control theory with the area of Networked Control Systems that considers lossy channels in the feedback loop. Our next set of results are concerned with controller architecture design. When designing controllers for large-scale systems, the architectural aspects of the controller such as the placement of actuators, sensors, and the communication links between them can no longer be taken as given -- indeed the task of designing this architecture is now as important as the design of the control laws themselves. To address this task, we formulate the Regularization for Design (RFD) framework, which is a unifying computationally tractable approach, based on the model matching framework and atomic norm regularization, for the simultaneous co-design of a structured optimal controller and the architecture needed to implement it. Our final result is a contribution to distributed system identification. Traditional system identification techniques such as subspace identification are not computationally scalable, and destroy rather than leverage any a priori information about the system's interconnection structure. We argue that in the context of system identification, an essential building block of any scalable algorithm is the ability to estimate local dynamics within a large interconnected system. To that end we propose a promising heuristic for identifying the dynamics of a subsystem that is still connected to a large system. We exploit the fact that the transfer function of the local dynamics is low-order, but full-rank, while the transfer function of the global dynamics is high-order, but low-rank, to formulate this separation task as a nuclear norm minimization problem. Finally, we conclude with a brief discussion of future research directions, with a particular emphasis on how to incorporate the results of this thesis, and those of optimal control theory in general, into a broader theory of dynamics, control and optimization in layered architectures.