Reinforcement learning with reference tracking control in continuous state spaces

The contribution described in this paper is an algorithm for learning nonlinear, reference tracking, control policies given no prior knowledge of the dynamical system and limited interaction with the system through the learning process. Concepts from the field of reinforcement learning, Bayesian statistics and classical control have been brought together in the formulation of this algorithm which can be viewed as a form of indirect self tuning regulator. On the task of reference tracking using a simulated inverted pendulum it was shown to yield generally improved performance on the best controller derived from the standard linear quadratic method using only 30 s of total interaction with the system. Finally, the algorithm was shown to work on the simulated double pendulum proving its ability to solve nontrivial control tasks. © 2011 IEEE.

Corner separation effects for normal shock wave/turbulent boundary layer interactions in rectangular channels

Experiments are conducted to examine the mechanisms behind the coupling between corner separation and separation away from the corner when holding a high-Machnumber M∞ = 1.5 normal shock in a rectangular channel. The ensuing shock wave interaction with the boundary layer on the wind tunnel floor and in the corners was studied using laser Doppler anemometry, Pitot probe traverses, pressure sensitive paint and flow visualization. The primary mechanism explaining the link between the corner separation size and the other areas of separation appears to be the generation of compression waves at the corner, which act to smear the adverse pressure gradient imposed upon other parts of the flow. Experimental results indicate that the alteration of the -region, which occurs in the supersonic portion of the shock wave/boundary layer interaction (SBLI), is more important than the generation of any blockage in the subsonic region downstream of the shock wave. © Copyright 2012 Cambridge University Press.

Thermal stratification produced by plumes and jets in enclosed spaces

The airflow and thermal stratification produced by a localised heat source located at floor level in a closed room is of considerable practical interest and is commonly referred to as a 'filling box'. In rooms with low aspect ratios H/R ≲ 1 (room height H to characteristic horizontal dimension R) the thermal plume spreads laterally on reaching the ceiling and a descending horizontal 'front' forms separating a stably stratified, warm upper region from cooler air below. The stratification is well predicted for H/R ≲ 1 by the original filling box model of Baines and Turner (J. Fluid. Mech. 37 (1968) 51). This model represents a somewhat idealised situation of a plume rising from a point source of buoyancy alone-in particular the momentum flux at the source is zero. In practical situations, real sources of heating and cooling in a ventilation system often include initial fluxes of both buoyancy and momentum, e.g. where a heating system vents warm air into a space. This paper describes laboratory experiments to determine the dependence of the 'front' formation and stratification on the source momentum and buoyancy fluxes of a single source, and on the location and relative strengths of two sources from which momentum and buoyancy fluxes were supplied separately. For a single source with a non-zero input of momentum, the rate of descent of the front is more rapid than for the case of zero source momentum flux and increases with increasing momentum input. Increasing the source momentum flux effectively increases the height of the enclosure, and leads to enhanced overturning motions and finally to complete mixing for highly momentum-driven flows. Stratified flows may be maintained by reducing the aspect ratio of the enclosure. At these low aspect ratios different long-time behaviour is observed depending on the nature of the heat input. A constant heat flux always produces a stratified interior at large times. On the other hand, a constant temperature supply ultimately produces a well-mixed space at the supply temperature. For separate sources of momentum and buoyancy, the developing stratification is shown to be strongly dependent on the separation of the sources and their relative strengths. Even at small separation distances the stratification initially exhibits horizontal inhomogeneity with localised regions of warm fluid (from the buoyancy source) and cool fluid. This inhomogeneity is less pronounced as the strength of one source is increased relative to the other. Regardless of the strengths of the sources, a constant buoyancy flux source dominates after sufficiently large times, although the strength of the momentum source determines whether the enclosure is initially well mixed (strong momentum source) or stably stratified (weak momentum source). © 2001 Elsevier Science Ltd. All rights reserved.

Consensus in non-commutative spaces

Convergence analysis of consensus algorithms is revisited in the light of the Hilbert distance. The Lyapunov function used in the early analysis by Tsitsiklis is shown to be the Hilbert distance to consensus in log coordinates. Birkhoff theorem, which proves contraction of the Hilbert metric for any positive homogeneous monotone map, provides an early yet general convergence result for consensus algorithms. Because Birkhoff theorem holds in arbitrary cones, we extend consensus algorithms to the cone of positive definite matrices. The proposed generalization finds applications in the convergence analysis of quantum stochastic maps, which are a generalization of stochastic maps to non-commutative probability spaces. ©2010 IEEE.

Corner effects in reflecting oblique shock-wave/boundarylayer interactions

A separated oblique shock reflection on the floor of a rectangular cross-section wind tunnel has been investigated at M=2.5. The study aims to determine if and how separations occurring in the corners influence the main interaction as observed around the centreline of the floor. By changing the size of the corner separations through localised suction and small corner obstructions it was shown that the shape of the separated region in the centre was altered considerably. The separation length along the floor centreline was also modified by changes to the corner separation. A simple physical model has been proposed to explain the coupling between these separated regions based on the existence of compression or shock waves caused by the displacement effect of corner separation. These corner shocks alter the adverse pressure gradient imposed on the boundary-layer elsewhere which can lead to local reductions or increases of separation length. It is suggested that a typical oblique shock wave/boundary-layer interaction in rectangular channels features several zones depending on the relative position of the corner shocks and the main incident shock wave. Based on these findings the dependence of centre-line separation length on effective wind tunnel width is hypothesised. This requires further verification through experiments or computation. © 2013 by H. Babinsky.

Efficient decoding with generative score-spaces using the expectation semiring

State-of-the-art speech recognisers are usually based on hidden Markov models (HMMs). They model a hidden symbol sequence with a Markov process, with the observations independent given that sequence. These assumptions yield efficient algorithms, but limit the power of the model. An alternative model that allows a wide range of features, including word- and phone-level features, is a log-linear model. To handle, for example, word-level variable-length features, the original feature vectors must be segmented into words. Thus, decoding must find the optimal combination of segmentation of the utterance into words and word sequence. Features must therefore be extracted for each possible segment of audio. For many types of features, this becomes slow. In this paper, long-span features are derived from the likelihoods of word HMMs. Derivatives of the log-likelihoods, which break the Markov assumption, are appended. Previously, decoding with this model took cubic time in the length of the sequence, and longer for higher-order derivatives. This paper shows how to decode in quadratic time. © 2013 IEEE.

Weighted least squares with expectation-maximization algorithm for burst detection in U.K. water distribution systems

Flow measurement data at the district meter area (DMA) level has the potential for burst detection in the water distribution systems. This work investigates using a polynomial function fitted to the historic flow measurements based on a weighted least-squares method for automatic burst detection in the U.K. water distribution networks. This approach, when used in conjunction with an expectationmaximization (EM) algorithm, can automatically select useful data from the historic flow measurements, which may contain normal and abnormal operating conditions in the distribution network, e.g., water burst. Thus, the model can estimate the normal water flow (nonburst condition), and hence the burst size on the water distribution system can be calculated from the difference between the measured flow and the estimated flow. The distinguishing feature of this method is that the burst detection is fully unsupervised, and the burst events that have occurred in the historic data do not affect the procedure and bias the burst detection algorithm. Experimental validation of the method has been carried out using a series of flushing events that simulate burst conditions to confirm that the simulated burst sizes are capable of being estimated correctly. This method was also applied to eight DMAs with known real burst events, and the results of burst detections are shown to relate to the water company's records of pipeline reparation work. © 2014 American Society of Civil Engineers.