Optimal bounded control of first-passage failure of quasi-integrable Hamiltonian systems with wide-band random excitation

The optimal bounded control of quasi-integrable Hamiltonian systems with wide-band random excitation for minimizing their first-passage failure is investigated. First, a stochastic averaging method for multi-degrees-of-freedom (MDOF) strongly nonlinear quasi-integrable Hamiltonian systems with wide-band stationary random excitations using generalized harmonic functions is proposed. Then, the dynamical programming equations and their associated boundary and final time conditions for the control problems of maximizinig reliability and maximizing mean first-passage time are formulated based on the averaged It$\ddot{\rm o}$ equations by applying the dynamical programming principle. The optimal control law is derived from the dynamical programming equations and control constraints. The relationship between the dynamical programming equations and the backward Kolmogorov equation for the conditional reliability function and the Pontryagin equation for the conditional mean first-passage time of optimally controlled system is discussed. Finally, the conditional reliability function, the conditional probability density and mean of first-passage time of an optimally controlled system are obtained by solving the backward Kolmogorov equation and Pontryagin equation. The application of the proposed procedure and effectiveness of control strategy are illustrated with an example.

Optimal bounded control of first-passage failure of strongly non-linear oscillators under combined harmonic and white-noise excitations

A procedure for designing the optimal bounded control of strongly non-linear oscillators under combined harmonic and white-noise excitations for minimizing their first-passage failure is proposed. First, a stochastic averaging method for strongly non-linear oscillators under combined harmonic and white-noise excitations using generalized harmonic functions is introduced. Then, the dynamical programming equations and their boundary and final time conditions for the control problems of maximizing reliability and of maximizing mean first-passage time are formulated from the averaged Ito equations by using the dynamical programming principle. The optimal control law is derived from the dynamical programming equations and control constraint. Finally, the conditional reliability function, the conditional probability density and mean of the first-passage time of the optimally controlled system are obtained from solving the backward Kolmogorov equation and Pontryagin equation. An example is given to illustrate the proposed procedure and the results obtained are verified by using those from digital simulation. (C) 2003 Elsevier Ltd. All rights reserved.

Hamilton?Jacobi?Bellman equations for quantum optimal feedback control

Gough, John; Belavkin, V.P.; Smolianov, O.G., (2005) 'Hamilton?Jacobi?Bellman equations for quantum optimal feedback control', Journal of Optics B: Quantum and Semiclassical Optics 7 pp.S237-S244 RAE2008

A unified framework for linear function approximation of value functions in stochastic control

This paper contributes with a unified formulation that merges previ- ous analysis on the prediction of the performance ( value function ) of certain sequence of actions ( policy ) when an agent operates a Markov decision process with large state-space. When the states are represented by features and the value function is linearly approxi- mated, our analysis reveals a new relationship between two common cost functions used to obtain the optimal approximation. In addition, this analysis allows us to propose an efficient adaptive algorithm that provides an unbiased linear estimate. The performance of the pro- posed algorithm is illustrated by simulation, showing competitive results when compared with the state-of-the-art solutions.

A Stochastic Control Approach to a Parabolic Equation, Reciprocal Processes

2000 Mathematics Subject Classi cation: 49L60, 60J60, 93E20.

Optimal investment under stochastic volatility and power type utility function

2000 Mathematics Subject Classification: 37F21, 70H20, 37L40, 37C40, 91G80, 93E20.

Optimal traffic control at railway junctions

Conflict occurs when two or more trains approach the same junction within a specified time. Such conflicts result in delays. Current practices to assign the right of way at junctions achieve orderly and safe passage of the trains, but do not attempt to reduce the delays. A traffic controller developed in the paper assigns right of way to impose minimum total weighted delay on the trains. The traffic flow model and the optimisation technique used in this controller are described. Simulation studies of the performance of the controller are given.

Optimal-stopping control for airborne collision avoidance and return-to-course flight

This paper considers an aircraft collision avoidance design problem that also incorporates design of the aircraft’s return-to-course flight. This control design problem is formulated as a non-linear optimal-stopping control problem; a formulation that does not require a prior knowledge of time taken to perform the avoidance and return-to-course manoeuvre. A dynamic programming solution to the avoidance and return-to-course problem is presented, before a Markov chain numerical approximation technique is described. Simulation results are presented that illustrate the proposed collision avoidance and return-to-course flight approach.

Motion planning and stochastic control with experimental validation on a planetary rover

Motion planning for planetary rovers must consider control uncertainty in order to maintain the safety of the platform during navigation. Modelling such control uncertainty is difficult due to the complex interaction between the platform and its environment. In this paper, we propose a motion planning approach whereby the outcome of control actions is learned from experience and represented statistically using a Gaussian process regression model. This model is used to construct a control policy for navigation to a goal region in a terrain map built using an on-board RGB-D camera. The terrain includes flat ground, small rocks, and non-traversable rocks. We report the results of 200 simulated and 35 experimental trials that validate the approach and demonstrate the value of considering control uncertainty in maintaining platform safety.

Jointly optimal power control and routing for a single cell, dense, ad hoc wireless network

We consider a dense, ad hoc wireless network confined to a small region, such that direct communication is possible between any pair of nodes. The physical communication model is that a receiver decodes the signal from a single transmitter, while treating all other signals as interference. Data packets are sent between source-destination pairs by multihop relaying. We assume that nodes self-organise into a multihop network such that all hops are of length d meters, where d is a design parameter. There is a contention based multiaccess scheme, and it is assumed that every node always has data to send, either originated from it or a transit packet (saturation assumption). In this scenario, we seek to maximize a measure of the transport capacity of the network (measured in bit-meters per second) over power controls (in a fading environment) and over the hop distance d, subject to an average power constraint. We first argue that for a dense collection of nodes confined to a small region, single cell operation is efficient for single user decoding transceivers. Then, operating the dense ad hoc network (described above) as a single cell, we study the optimal hop length and power control that maximizes the transport capacity for a given network power constraint. More specifically, for a fading channel and for a fixed transmission time strategy (akin to the IEEE 802.11 TXOP), we find that there exists an intrinsic aggregate bit rate (Theta(opt) bits per second, depending on the contention mechanism and the channel fading characteristics) carried by the network, when operating at the optimal hop length and power control. The optimal transport capacity is of the form d(opt)((P) over bar (t)) x Theta(opt) with d(opt) scaling as (P) over bar (1/eta)(t), where (P) over bar (t) is the available time average transmit power and eta is the path loss exponent. Under certain conditions on the fading distribution, we then provide a simple characterisation of the optimal operating point.

Optimal Nonlinear Control and Estimation for a Reusable Launch Vehicle During Reentry Phase

In this paper a nonlinear optimal controller has been designed for aerodynamic control during the reentry phase of the Reusable Launch Vehicle (RLV). The controller has been designed based on a recently developed technique Optimal Dynamic Inversion (ODI). For full state feedback the controller has required full information about the system states. In this work an Extended Kalman filter (EKF) is developed to estimate the states. The vehicle (RLV) has been has been consider as a nonlinear Six-Degree-Of-Freedom (6-DOF) model. The simulation results shows that EKF gives a very good estimation of the states and it is working well with ODI. The resultant trajectories are very similar to those obtained by perfect state feedback using ODI only.

On extremal solutions to stochastic control problems, II

The set of attainable laws of the joint state-control process of a controlled diffusion is analyzed from a convex analytic viewpoint. Various equivalence relations depending on one-dimensional marginals thereof are defined on this set and the corresponding equivalence classes are studied.

Jointly Optimal Power Control and Hop Length for a Single Cell, Dense, Ad hoc Wireless Network

We consider a dense, ad hoc wireless network confined to a small region, such that direct communication is possible between any pair of nodes. The physical communication model is that a receiver decodes the signal from a single transmitter, while treating all other signals as interference. Data packets are sent between source-destination pairs by multihop relaying. We assume that nodes self-organise into a multihop network such that all hops are of length d meters, where d is a design parameter. There is a contention based multiaccess scheme, and it is assumed that every node always has data to send, either originated from it or a transit packet (saturation assumption). In this scenario, we seek to maximize a measure of the transport capacity of the network (measured in bit-meters per second) over power controls (in a fading environment) and over the hop distance d, subject to an average power constraint. We first argue that for a dense collection of nodes confined to a small region, single cell operation is efficient for single user decoding transceivers. Then, operating the dense ad hoc network (described above) as a single cell, we study the optimal hop length and power control that maximizes the transport capacity for a given network power constraint. More specifically, for a fading channel and for a fixed transmission time strategy (akin to the IEEE 802.11 TXOP), we find that there exists an intrinsic aggregate bit rate (Thetaopt bits per second, depending on the contention mechanism and the channel fading characteristics) carried by the network, when operating at the optimal hop length and power control. The optimal transport capacity is of the form dopt(Pmacrt) x Thetaopt with dopt scaling as Pmacrt 1 /eta, where Pmacrt is the available time average transmit power and eta is the path loss exponent. Under certain conditions on the fading distribution, we then pro- - vide a simple characterisation of the optimal operating point.