Efficient methods for stochastic optimal control

The Hamilton Jacobi Bellman (HJB) equation is central to stochastic optimal control (SOC) theory, yielding the optimal solution to general problems specified by known dynamics and a specified cost functional. Given the assumption of quadratic cost on the control input, it is well known that the HJB reduces to a particular partial differential equation (PDE). While powerful, this reduction is not commonly used as the PDE is of second order, is nonlinear, and examples exist where the problem may not have a solution in a classical sense. Furthermore, each state of the system appears as another dimension of the PDE, giving rise to the curse of dimensionality. Since the number of degrees of freedom required to solve the optimal control problem grows exponentially with dimension, the problem becomes intractable for systems with all but modest dimension.

In the last decade researchers have found that under certain, fairly non-restrictive structural assumptions, the HJB may be transformed into a linear PDE, with an interesting analogue in the discretized domain of Markov Decision Processes (MDP). The work presented in this thesis uses the linearity of this particular form of the HJB PDE to push the computational boundaries of stochastic optimal control.

This is done by crafting together previously disjoint lines of research in computation. The first of these is the use of Sum of Squares (SOS) techniques for synthesis of control policies. A candidate polynomial with variable coefficients is proposed as the solution to the stochastic optimal control problem. An SOS relaxation is then taken to the partial differential constraints, leading to a hierarchy of semidefinite relaxations with improving sub-optimality gap. The resulting approximate solutions are shown to be guaranteed over- and under-approximations for the optimal value function. It is shown that these results extend to arbitrary parabolic and elliptic PDEs, yielding a novel method for Uncertainty Quantification (UQ) of systems governed by partial differential constraints. Domain decomposition techniques are also made available, allowing for such problems to be solved via parallelization and low-order polynomials.

The optimization-based SOS technique is then contrasted with the Separated Representation (SR) approach from the applied mathematics community. The technique allows for systems of equations to be solved through a low-rank decomposition that results in algorithms that scale linearly with dimensionality. Its application in stochastic optimal control allows for previously uncomputable problems to be solved quickly, scaling to such complex systems as the Quadcopter and VTOL aircraft. This technique may be combined with the SOS approach, yielding not only a numerical technique, but also an analytical one that allows for entirely new classes of systems to be studied and for stability properties to be guaranteed.

The analysis of the linear HJB is completed by the study of its implications in application. It is shown that the HJB and a popular technique in robotics, the use of navigation functions, sit on opposite ends of a spectrum of optimization problems, upon which tradeoffs may be made in problem complexity. Analytical solutions to the HJB in these settings are available in simplified domains, yielding guidance towards optimality for approximation schemes. Finally, the use of HJB equations in temporal multi-task planning problems is investigated. It is demonstrated that such problems are reducible to a sequence of SOC problems linked via boundary conditions. The linearity of the PDE allows us to pre-compute control policy primitives and then compose them, at essentially zero cost, to satisfy a complex temporal logic specification.

Fluid mechanics-non-linear waves studies on korteweg-de vries equations

In this thesis the author has presented qualitative studies of certain Kdv equations with variable coefficients. The well-known KdV equation is a model for waves propagating on the surface of shallow water of constant depth. This model is considered as fitting into waves reaching the shore. Renewed attempts have led to the derivation of KdV type equations in which the coefficients are not constants. Johnson's equation is one such equation. The researcher has used this model to study the interaction of waves. It has been found that three-wave interaction is possible, there is transfer of energy between the waves and the energy is not conserved during interaction.

An analytical model of mountain wave drag for wind profiles with shear and curvature

An analytical model is developed to predict the surface drag exerted by internal gravity waves on an isolated axisymmetric mountain over which there is a stratified flow with a velocity profile that varies relatively slowly with height. The model is linear with respect to the perturbations induced by the mountain, and solves the Taylor–Goldstein equation with variable coefficients using a Wentzel–Kramers–Brillouin (WKB) approximation, formally valid for high Richardson numbers, Ri. The WKB solution is extended to a higher order than in previous studies, enabling a rigorous treatment of the effects of shear and curvature of the wind profile on the surface drag. In the hydrostatic approximation, closed formulas for the drag are derived for generic wind profiles, where the relative magnitude of the corrections to the leading-order drag (valid for a constant wind profile) does not depend on the detailed shape of the orography. The drag is found to vary proportionally to Ri21, decreasing as Ri decreases for a wind that varies linearly with height, and increasing as Ri decreases for a wind that rotates with height maintaining its magnitude. In these two cases the surface drag is predicted to be aligned with the surface wind. When one of the wind components varies linearly with height and the other is constant, the surface drag is misaligned with the surface wind, especially for relatively small Ri. All these results are shown to be in fairly good agreement with numerical simulations of mesoscale nonhydrostatic models, for high and even moderate values of Ri.

An exact equation for the free surface of a fluid in a porous medium

We study the problem of the evolution of the free surface of a fluid in a saturated porous medium, bounded from below by a. at impermeable bottom, and described by the Laplace equation with moving-boundary conditions. By making use of a convenient conformal transformation, we show that the solution to this problem is equivalent to the solution of the Laplace equation on a fixed domain, with new variable coefficients, the boundary conditions. We use a kernel of the Laplace equation which allows us to write the Dirichlet-to-Neumann operator, and in this way we are able to find an exact differential-integral equation for the evolution of the free surface in one space dimension. Although not amenable to direct analytical solutions, this equation turns out to allow an easy numerical implementation. We give an explicit illustrative case at the end of the article.

Modulated entry /

"The technique of modulation, or variable coefficients, is discussed and the analytical formulation is reviewed. Representative numerical results of the use of modulation are shown for the lifting and nonlifting cases. These results include the effects of modulation on peak acceleration, entry corridor, and heat absorption. Results are given for entry at satellite speed and escape speed. The indications are that coefficient modulation on a vehicle with good lifting capability offers the possibility of sizable loading reductions or, alternatively, wider corridors; thus, steep entries become practical from the loading standpoint. The amount of steepness depends on the acceptable heating penalty. The price of sizable fractions of the possible gains does not appear to be excessive."

On functionally-fitted Runge-Kutta methods

Functionally-fitted methods are generalizations of collocation techniques to integrate an equation exactly if its solution is a linear combination of a chosen set of basis functions. When these basis functions are chosen as the power functions, we recover classical algebraic collocation methods. This paper shows that functionally-fitted methods can be derived with less restrictive conditions than previously stated in the literature, and that other related results can be derived in a much more elegant way. The novelty in our approach is to fully retain the collocation framework without reverting back into derivations based on cumbersome Taylor series expansions.

Analysis of trigonometric implicit Runge-Kutta methods

Using generalized collocation techniques based on fitting functions that are trigonometric (rather than algebraic as in classical integrators), we develop a new class of multistage, one-step, variable stepsize, and variable coefficients implicit Runge-Kutta methods to solve oscillatory ODE problems. The coefficients of the methods are functions of the frequency and the stepsize. We refer to this class as trigonometric implicit Runge-Kutta (TIRK) methods. They integrate an equation exactly if its solution is a trigonometric polynomial with a known frequency. We characterize the order and A-stability of the methods and establish results similar to that of classical algebraic collocation RK methods. (c) 2006 Elsevier B.V. All rights reserved.

Diffusion modeling of percutaneous absorption kinetics: 3. Variable diffusion and partition coefficients, consequences for stratum corneum depth profiles and desorption kinetics

Stratum corneum (SC) desorption experiments have yielded higher calculated steady-state fluxes than those obtained by epidermal penetration studies. A possible explanation of this result is a variable diffusion or partition coefficient across the SC. We therefore developed the diffusion model for percutaneous penetration and desorption to study the effects of either a variable diffusion coefficient or variable partition coefficient in the SC over the diffusion path length. Steady-state flux, lag time, and mean desorption time were obtained from Laplace domain solutions. Numerical inversion of the Laplace domain solutions was used for simulations of solute concentration-distance and amount penetrated (desorbed)-time profiles. Diffusion and partition coefficients heterogeneity were examined using six different models. The effect of heterogeneity on predicted flux from desorption studies was compared with that obtained in permeation studies. Partition coefficient heterogeneity had a more profound effect on predicted fluxes than diffusion coefficient heterogeneity. Concentration-distance profiles show even larger dependence on heterogeneity, which is consistent with experimental tape-stripping data reported for clobetasol propionate and other solutes. The clobetasol propionate tape-stripping data were most consistent with the partition coefficient decreasing exponentially for half the SC and then becoming a constant for the remaining SC. (C) 2004 Wiley-Liss, Inc.

Oscillatory flow in an elastic tube of variable cross-section

An oscillatory flow of a viscous incompressible fluid in an elastic tube of variable cross section has been investigated at low Reynolds number. The equations governing, the flow are derived under the assumption that the variation of the cross-section is slow in the axial direction for a tethered tube. The problem is then reduced to that of solving for the excess pressure from a second order ordinary differential equation with complex valued Bessel functions as the coefficients. This equation has been solved numerically for geometries of physiological interest and a comparison is made with some of the known theoretical and experimental results.

Vibrations of a Beam in Variable Contact With a Flat Surface

We study small vibrations of cantilever beams contacting a rigid surface. We study two cases: the first is a beam that sags onto the ground due to gravity, and the second is a beam that sticks to the ground through reversible adhesion. In both cases, the noncontacting length varies dynamically. We first obtain the governing equations and boundary conditions, including a transversality condition involving an end moment, using Hamilton's principle. Rescaling the variable length to a constant value, we obtain partial differential equations with time varying coefficients, which, upon linearization, give the natural frequencies of vibration. The natural frequencies for the first case (gravity without adhesion) match that of a clamped-clamped beam of the same nominal length; frequencies for the second case, however, show no such match. We develop simple, if atypical, single degree of freedom approximations for the first modes of these two systems, which provide insights into the role of the static deflection profile, as well as the end moment condition, in determining the first natural frequencies of these systems. Finally, we consider small transverse sinusoidal forcing of the first case and find that the governing equation contains both parametric and external forcing terms. For forcing at resonance, w find that either the internal or the external forcing may dominate.

Conservation law with the flux function discontinuous in the space variable- II - Convex-concave type fluxes and generalized entropy solutions

We deal with a single conservation law with discontinuous convex-concave type fluxes which arise while considering sign changing flux coefficients. The main difficulty is that a weak solution may not exist as the Rankine-Hugoniot condition at the interface may not be satisfied for certain choice of the initial data. We develop the concept of generalized entropy solutions for such equations by replacing the Rankine-Hugoniot condition by a generalized Rankine-Hugoniot condition. The uniqueness of solutions is shown by proving that the generalized entropy solutions form a contractive semi-group in L-1. Existence follows by showing that a Godunov type finite difference scheme converges to the generalized entropy solution. The scheme is based on solutions of the associated Riemann problem and is neither consistent nor conservative. The analysis developed here enables to treat the cases of fluxes having at most one extrema in the domain of definition completely. Numerical results reporting the performance of the scheme are presented. (C) 2006 Elsevier B.V. All rights reserved.

Latent Variable Models for Stochastic Discount Factors.

Latent variable models in finance originate both from asset pricing theory and time series analysis. These two strands of literature appeal to two different concepts of latent structures, which are both useful to reduce the dimension of a statistical model specified for a multivariate time series of asset prices. In the CAPM or APT beta pricing models, the dimension reduction is cross-sectional in nature, while in time-series state-space models, dimension is reduced longitudinally by assuming conditional independence between consecutive returns, given a small number of state variables. In this paper, we use the concept of Stochastic Discount Factor (SDF) or pricing kernel as a unifying principle to integrate these two concepts of latent variables. Beta pricing relations amount to characterize the factors as a basis of a vectorial space for the SDF. The coefficients of the SDF with respect to the factors are specified as deterministic functions of some state variables which summarize their dynamics. In beta pricing models, it is often said that only the factorial risk is compensated since the remaining idiosyncratic risk is diversifiable. Implicitly, this argument can be interpreted as a conditional cross-sectional factor structure, that is, a conditional independence between contemporaneous returns of a large number of assets, given a small number of factors, like in standard Factor Analysis. We provide this unifying analysis in the context of conditional equilibrium beta pricing as well as asset pricing with stochastic volatility, stochastic interest rates and other state variables. We address the general issue of econometric specifications of dynamic asset pricing models, which cover the modern literature on conditionally heteroskedastic factor models as well as equilibrium-based asset pricing models with an intertemporal specification of preferences and market fundamentals. We interpret various instantaneous causality relationships between state variables and market fundamentals as leverage effects and discuss their central role relative to the validity of standard CAPM-like stock pricing and preference-free option pricing.

On Connection, Linearization and Duplication Coefficients of Classical Orthogonal Polynomials

In this work, we have mainly achieved the following: 1. we provide a review of the main methods used for the computation of the connection and linearization coefficients between orthogonal polynomials of a continuous variable, moreover using a new approach, the duplication problem of these polynomial families is solved; 2. we review the main methods used for the computation of the connection and linearization coefficients of orthogonal polynomials of a discrete variable, we solve the duplication and linearization problem of all orthogonal polynomials of a discrete variable; 3. we propose a method to generate the connection, linearization and duplication coefficients for q-orthogonal polynomials; 4. we propose a unified method to obtain these coefficients in a generic way for orthogonal polynomials on quadratic and q-quadratic lattices. Our algorithmic approach to compute linearization, connection and duplication coefficients is based on the one used by Koepf and Schmersau and on the NaViMa algorithm. Our main technique is to use explicit formulas for structural identities of classical orthogonal polynomial systems. We find our results by an application of computer algebra. The major algorithmic tools for our development are Zeilberger’s algorithm, q-Zeilberger’s algorithm, the Petkovšek-van-Hoeij algorithm, the q-Petkovšek-van-Hoeij algorithm, and Algorithm 2.2, p. 20 of Koepf's book "Hypergeometric Summation" and it q-analogue.

Impact of variable atmospheric and oceanic form drag on simulations of Arctic sea ice

Over Arctic sea ice, pressure ridges and floe andmelt pond edges all introduce discrete obstructions to the flow of air or water past the ice and are a source of form drag. In current climate models form drag is only accounted for by tuning the air–ice and ice–ocean drag coefficients, that is, by effectively altering the roughness length in a surface drag parameterization. The existing approach of the skin drag parameter tuning is poorly constrained by observations and fails to describe correctly the physics associated with the air–ice and ocean–ice drag. Here, the authors combine recent theoretical developments to deduce the total neutral form drag coefficients from properties of the ice cover such as ice concentration, vertical extent and area of the ridges, freeboard and floe draft, and the size of floes and melt ponds. The drag coefficients are incorporated into the Los Alamos Sea Ice Model (CICE) and show the influence of the new drag parameterization on the motion and state of the ice cover, with the most noticeable being a depletion of sea ice over the west boundary of the Arctic Ocean and over the Beaufort Sea. The new parameterization allows the drag coefficients to be coupled to the sea ice state and therefore to evolve spatially and temporally. It is found that the range of values predicted for the drag coefficients agree with the range of values measured in several regions of the Arctic. Finally, the implications of the new form drag formulation for the spinup or spindown of the Arctic Ocean are discussed.

The impact of variable sea ice roughness on changes in Arctic Ocean surface stress: a model study

The Arctic sea ice cover is thinning and retreating, causing changes in surface roughness that in turn modify the momentum flux from the atmosphere through the ice into the ocean. New model simulations comprising variable sea ice drag coefficients for both the air and water interface demonstrate that the heterogeneity in sea ice surface roughness significantly impacts the spatial distribution and trends of ocean surface stress during the last decades. Simulations with constant sea ice drag coefficients as used in most climate models show an increase in annual mean ocean surface stress (0.003 N/m2 per decade, 4.6%) due to the reduction of ice thickness leading to a weakening of the ice and accelerated ice drift. In contrast, with variable drag coefficients our simulations show annual mean ocean surface stress is declining at a rate of -0.002 N/m2 per decade (3.1%) over the period 1980-2013 because of a significant reduction in surface roughness associated with an increasingly thinner and younger sea ice cover. The effectiveness of sea ice in transferring momentum does not only depend on its resistive strength against the wind forcing but is also set by its top and bottom surface roughness varying with ice types and ice conditions. This reveals the need to account for sea ice surface roughness variations in climate simulations in order to correctly represent the implications of sea ice loss under global warming.