Efficient methods for stochastic optimal control

The Hamilton Jacobi Bellman (HJB) equation is central to stochastic optimal control (SOC) theory, yielding the optimal solution to general problems specified by known dynamics and a specified cost functional. Given the assumption of quadratic cost on the control input, it is well known that the HJB reduces to a particular partial differential equation (PDE). While powerful, this reduction is not commonly used as the PDE is of second order, is nonlinear, and examples exist where the problem may not have a solution in a classical sense. Furthermore, each state of the system appears as another dimension of the PDE, giving rise to the curse of dimensionality. Since the number of degrees of freedom required to solve the optimal control problem grows exponentially with dimension, the problem becomes intractable for systems with all but modest dimension.

In the last decade researchers have found that under certain, fairly non-restrictive structural assumptions, the HJB may be transformed into a linear PDE, with an interesting analogue in the discretized domain of Markov Decision Processes (MDP). The work presented in this thesis uses the linearity of this particular form of the HJB PDE to push the computational boundaries of stochastic optimal control.

This is done by crafting together previously disjoint lines of research in computation. The first of these is the use of Sum of Squares (SOS) techniques for synthesis of control policies. A candidate polynomial with variable coefficients is proposed as the solution to the stochastic optimal control problem. An SOS relaxation is then taken to the partial differential constraints, leading to a hierarchy of semidefinite relaxations with improving sub-optimality gap. The resulting approximate solutions are shown to be guaranteed over- and under-approximations for the optimal value function. It is shown that these results extend to arbitrary parabolic and elliptic PDEs, yielding a novel method for Uncertainty Quantification (UQ) of systems governed by partial differential constraints. Domain decomposition techniques are also made available, allowing for such problems to be solved via parallelization and low-order polynomials.

The optimization-based SOS technique is then contrasted with the Separated Representation (SR) approach from the applied mathematics community. The technique allows for systems of equations to be solved through a low-rank decomposition that results in algorithms that scale linearly with dimensionality. Its application in stochastic optimal control allows for previously uncomputable problems to be solved quickly, scaling to such complex systems as the Quadcopter and VTOL aircraft. This technique may be combined with the SOS approach, yielding not only a numerical technique, but also an analytical one that allows for entirely new classes of systems to be studied and for stability properties to be guaranteed.

The analysis of the linear HJB is completed by the study of its implications in application. It is shown that the HJB and a popular technique in robotics, the use of navigation functions, sit on opposite ends of a spectrum of optimization problems, upon which tradeoffs may be made in problem complexity. Analytical solutions to the HJB in these settings are available in simplified domains, yielding guidance towards optimality for approximation schemes. Finally, the use of HJB equations in temporal multi-task planning problems is investigated. It is demonstrated that such problems are reducible to a sequence of SOC problems linked via boundary conditions. The linearity of the PDE allows us to pre-compute control policy primitives and then compose them, at essentially zero cost, to satisfy a complex temporal logic specification.

I. Foehn winds of southern California. II. Foehn wind cyclo-genesis. III. Weather conditions associated with the Akron disaster. IV. The Los Angeles storm of December 30, 1933 to January 1, 1934

I. Foehn winds of southern California.
An investigation of the hot, dry and dust laden winds occurring in the late fall and early winter in the Los Angeles Basin and attributed in the past to the influences of the desert regions to the north revealed that these currents were of a foehn nature. Their properties were found to be entirely due to dynamical heating produced in the descent from the high level areas in the interior to the lower Los Angeles Basin. Any dust associated with the phenomenon was found to be acquired from the Los Angeles area rather than transported from the desert. It was found that the frequency of occurrence of a mild type foehn of this nature during this season was sufficient to warrant its classification as a winter monsoon. This results from the topography of the Los Angeles region which allows an easy entrance to the air from the interior by virtue of the low level mountain passes north of the area. This monsoon provides the mild winter climate of southern California since temperatures associated with the foehn currents are far higher than those experienced when maritime air from the adjacent Pacific Ocean occupies the region.

II. Foehn wind cyclo-genesis.
Intense anticyclones frequently build up over the high level regions of the Great Basin and Columbia Plateau which lie between the Sierra Nevada and Cascade Mountains to the west and the Rocky Mountains to the east. The outflow from these anticyclones produce extensive foehns east of the Rockies in the comparatively low level areas of the middle west and the Canadian provinces of Alberta and Saskatchewan. Normally at this season of the year very cold polar continental air masses are present over this territory and with the occurrence of these foehns marked discontinuity surfaces arise between the warm foehn current, which is obliged to slide over a colder mass, and the Pc air to the east. Cyclones are easily produced from this phenomenon and take the form of unstable waves which propagate along the discontinuity surface between the two dissimilar masses. A continual series of such cyclones was found to occur as long as the Great Basin anticyclone is maintained with undiminished intensity.

III. Weather conditions associated with the Akron disaster.
This situation illustrates the speedy development and propagation of young disturbances in the eastern United States during the spring of the year under the influence of the conditionally unstable tropical maritime air masses which characterise the region. It also furnishes an excellent example of the superiority of air mass and frontal methods of weather prediction for aircraft operation over the older methods based upon pressure distribution.

IV. The Los Angeles storm of December 30, 1933 to January 1, 1934.
This discussion points out some of the fundamental interactions occurring between air masses of the North Pacific Ocean in connection with Pacific Coast storms and the value of topographic and aerological considerations in predicting them. Estimates of rainfall intensity and duration from analyses of this type may be made and would prove very valuable in the Los Angeles area in connection with flood control problems.

Interrelated contributions to freshwater science over 78 years: A guide to published work from the Freshwater Biological Association, Institute of Freshwater Ecology and Centre for Ecology and Hydrology, 1929-2006

This article introduces a new listing of published scientific contributions from the Freshwater Biological Association (FBA) and its later Research Council associates – the Institute of Freshwater Ecology (1989–2000) and the Centre for Ecology and Hydrology (2000+). The period 1929–2006 is covered. The authors offer also information on specific features of the listing; also an outline of influences that underlay the research, and its scientific scope.

Symmetric representation of an integral domain over a subdomain

Let F(θ) be a separable extension of degree n of a field F. Let Δ and D be integral domains with quotient fields F(θ) and F respectively. Assume that Δ ᴝ D. A mapping φ of Δ into the n x n D matrices is called a Δ/D rep if (i) it is a ring isomorphism and (ii) it maps d onto dI_n whenever d ϵ D. If the matrices are also symmetric, φ is a Δ/D symrep.

Every Δ/D rep can be extended uniquely to an F(θ)/F rep. This extension is completely determined by the image of θ. Two Δ/D reps are called equivalent if the images of θ differ by a D unimodular similarity. There is a one-to-one correspondence between classes of Δ/D reps and classes of Δ ideals having an n element basis over D.

The condition that a given Δ/D rep class contain a Δ/D symrep can be phrased in various ways. Using these formulations it is possible to (i) bound the number of symreps in a given class, (ii) count the number of symreps if F is finite, (iii) establish the existence of an F(θ)/F symrep when n is odd, F is an algebraic number field, and F(θ) is totally real if F is formally real (for n = 3 see Sapiro, “Characteristic polynomials of symmetric matrices” Sibirsk. Mat. Ž. 3 (1962) pp. 280-291), and (iv) study the case D = Z, the integers (see Taussky, “On matrix classes corresponding to an ideal and its inverse” Illinois J. Math. 1 (1957) pp. 108-113 and Faddeev, “On the characteristic equations of rational symmetric matrices” Dokl. Akad. Nauk SSSR 58 (1947) pp. 753-754).

The case D = Z and n = 2 is studied in detail. Let Δ’ be an integral domain also having quotient field F(θ) and such that Δ’ ᴝ Δ. Let φ be a Δ/Z symrep. A method is given for finding a Δ’/Z symrep ʘ such that the Δ’ ideal class corresponding to the class of ʘ is an extension to Δ’ of the Δ ideal class corresponding to the class of φ. The problem of finding all Δ/Z symreps equivalent to a given one is studied.

Theoretical investigation of turbulent boundary layer over a wavy surface

The important features of the two-dimensional incompressible turbulent flow over a wavy surface of wavelength comparable with the boundary layer thickness are analyzed.

A turbulent field method using model equation for turbulent shear stress similar to the scheme of Bradshaw, Ferriss and Atwell (1967) is employed with suitable modification to cover the viscous sublayer. The governing differential equations are linearized based on the small but finite amplitude to wavelength ratio. An orthogonal wavy coordinate system, accurate to the second order in the amplitude ratio, is adopted to avoid the severe restriction to the validity of linearization due to the large mean velocity gradient near the wall. Analytic solution up to the second order is obtained by using the method of matched-asymptotic-expansion based on the large Reynolds number and hence the small skin friction coefficient.

In the outer part of the layer, the perturbed flow is practically "inviscid." Solutions for the velocity, Reynolds stress and also the wall pressure distributions agree well with the experimental measurement. In the wall region where the perturbed Reynolds stress plays an important role in the process of momentum transport, only a qualitative agreement is obtained. The results also show that the nonlinear second-order effect is negligible for amplitude ratio of 0.03. The discrepancies in the detailed structure of the velocity, shear stress, and skin friction distributions near the wall suggest modifications to the model are required to describe the present problem.

New measurements on the absolute cosmic ray ionization from sea level to 1540 kilometers altitude

An air filled ionization chamber has been constructed with a volume of 552 liters and a wall consisting of 12.7 mg/cm² of plastic wrapped over a rigid, lightweight aluminum frame. A calibration in absolute units, independent of previous Caltech ion chamber calibrations, was applied to a sealed Neher electrometer for use in this chamber. The new chamber was flown along with an older, argon filled, balloon type chamber in a C-135 aircraft from 1,000 to 40,000 feet altitude, and other measurements of sea level cosmic ray ionization were made, resulting in the value of 2.60 ± .03 ion pairs/cm³ sec atm) at sea level. The calibrations of the two instruments were found to agree within 1 percent, and the airplane data were consistent with previous balloon measurements in the upper atmosphere. Ionization due to radon gas in the atmosphere was investigated. Absolute ionization data in the lower atmosphere have been compared with results of other observers, and discrepancies have been discussed.

Data from a polar orbiting ion chamber on the OGO-II, IV spacecraft have been analyzed. The problem of radioactivity produced on the spacecraft during passes through high fluxes of trapped protons has been investigated, and some corrections determined. Quiet time ionization averages over the polar regions have been plotted as function of altitude, and an analytical fit is made to the data that gives a value of 10.4 ± 2.3 percent for the fractional part of the ionization at the top of the atmosphere due to splash albedo particles, although this result is shown to depend on an assumed angular distribution for the albedo particles. Comparisons with other albedo measurements are made. The data are shown to be consistent with balloon and interplanetary ionization measurements. The position of the cosmic ray knee is found to exhibit an altitude dependence, a North-South effect, and a small local time variation.