Optimistic linear programming gives logarithmic regret for irreducible MDPs

We present an algorithm called Optimistic Linear Programming (OLP) for learning to optimize average reward in an irreducible but otherwise unknown Markov decision process (MDP). OLP uses its experience so far to estimate the MDP. It chooses actions by optimistically maximizing estimated future rewards over a set of next-state transition probabilities that are close to the estimates, a computation that corresponds to solving linear programs. We show that the total expected reward obtained by OLP up to time T is within C(P) log T of the reward obtained by the optimal policy, where C(P) is an explicit, MDP-dependent constant. OLP is closely related to an algorithm proposed by Burnetas and Katehakis with four key differences: OLP is simpler, it does not require knowledge of the supports of transition probabilities, the proof of the regret bound is simpler, but our regret bound is a constant factor larger than the regret of their algorithm. OLP is also similar in flavor to an algorithm recently proposed by Auer and Ortner. But OLP is simpler and its regret bound has a better dependence on the size of the MDP.

High-probability regret bounds for bandit online linear optimization

We present a modification of the algorithm of Dani et al. [8] for the online linear optimization problem in the bandit setting, which with high probability has regret at most O ∗ ( √ T) against an adaptive adversary. This improves on the previous algorithm [8] whose regret is bounded in expectation against an oblivious adversary. We obtain the same dependence on the dimension (n 3/2) as that exhibited by Dani et al. The results of this paper rest firmly on those of [8] and the remarkable technique of Auer et al. [2] for obtaining high probability bounds via optimistic estimates. This paper answers an open question: it eliminates the gap between the high-probability bounds obtained in the full-information vs bandit settings.

The differential recognition of children’s cultural practices in middle primary literacy classrooms

This paper argues that teachers’ recognition of children’s cultural practices is an important positive step in helping socio-economically disadvantaged children engage with school literacies. Based on twenty-one longitudinal case studies of children’s literacy development over a three-year period, the authors demonstrate that when children’s knowledges and practices assembled in home and community spheres are treated as valuable material for school learning, children are more likely to invest in the work of acquiring school literacies. However they show also that whilst some children benefit greatly from being allowed to draw on their knowledge of popular culture, sports and the outdoors, other children’s interests may be ignored or excluded. Some differences in teachers’ valuing of home and community cultures appeared to relate to gender dimensions.

Supplement : Efficient weak second-order stochastic Runge-Kutta methods for non-commutative Stratonvich stochastic differential equations

This paper gives a modification of a class of stochastic Runge–Kutta methods proposed in a paper by Komori (2007). The slight modification can reduce the computational costs of the methods significantly.

Compensation of unmodeled aircraft dynamics in airborne inspection of linear infrastructure assets

Fixed-wing aircraft equipped with downward pointing cameras and/or LiDAR can be used for inspecting approximately piecewise linear assets such as oil-gas pipelines, roads and power-lines. Automatic control of such aircraft is important from a productivity and safety point of view (long periods of precision manual flight at low-altitude is not considered reasonable from a safety perspective). This paper investigates the effect of any unwanted coupling between guidance and autopilot loops (typically caused by unmodeled delays in the aircraft’s response), and the specific impact of any unwanted dynamics on the performance of aircraft undertaking inspection of piecewise linear corridor assets (such as powerlines). Simulation studies and experimental flight tests are used to demonstrate the benefits of a simple compensator in mitigating the unwanted lateral oscillatory behaviour (or coupling) that is caused by unmodeled time constants in the aircraft dynamics.

Vision based control for fixed wing UAVs inspecting locally linear infrastructure using skid-to-turn maneuvers

The following paper proposes a novel application of Skid-to-Turn maneuvers for fixed wing Unmanned Aerial Vehicles (UAVs) inspecting locally linear infrastructure. Fixed wing UAVs, following the design of manned aircraft, traditionally employ Bank-to-Turn maneuvers to change heading and thus direction of travel. Commonly overlooked is the effect these maneuvers have on downward facing body fixed sensors, which as a result of bank, point away from the feature during turns. By adopting Skid-to-Turn maneuvers, the aircraft is able change heading whilst maintaining wings level flight, thus allowing body fixed sensors to maintain a downward facing orientation. Eliminating roll also helps to improve data quality, as sensors are no longer subjected to the swinging motion induced as they pivot about an axis perpendicular to their line of sight. Traditional tracking controllers that apply an indirect approach of capturing ground based data by flying directly overhead can also see the feature off center due to steady state pitch and roll required to stay on course. An Image Based Visual Servo controller is developed to address this issue, allowing features to be directly tracked within the image plane. Performance of the proposed controller is tested against that of a Bank-to-Turn tracking controller driven by GPS derived cross track error in a simulation environment developed to simulate the field of view of a body fixed camera.

Protection of microgrids using differential relays

A microgrid provides economical and reliable power to customers by integrating distributed resources more effectively. Islanded operation enables a continuous power supply for loads during a major grid disturbance. Reliability of a microgrid can be further increased by forming a mesh configuration. However, the protection of mesh microgrids is a challenging task. In this paper, protection schemes are discussed using current differential protection of a microgrid. The protection challenges associated with bi-directional power flow, meshed configuration, changing fault current level due to intermittent nature of DGs and reduced fault current level in an islanded mode are considered in proposing the protection solutions. Relay setting criterion and current transformer (CT) selection guidelines are also discussed. The results are verified using MATLAB calculations and PSCAD simulations.

Models of collective cell motion for cell populations with different aspect ratio : diffusion, proliferation and travelling waves

Continuum, partial differential equation models are often used to describe the collective motion of cell populations, with various types of motility represented by the choice of diffusion coefficient, and cell proliferation captured by the source terms. Previously, the choice of diffusion coefficient has been largely arbitrary, with the decision to choose a particular linear or nonlinear form generally based on calibration arguments rather than making any physical connection with the underlying individual-level properties of the cell motility mechanism. In this work we provide a new link between individual-level models, which account for important cell properties such as varying cell shape and volume exclusion, and population-level partial differential equation models. We work in an exclusion process framework, considering aligned, elongated cells that may occupy more than one lattice site, in order to represent populations of agents with different sizes. Three different idealizations of the individual-level mechanism are proposed, and these are connected to three different partial differential equations, each with a different diffusion coefficient; one linear, one nonlinear and degenerate and one nonlinear and nondegenerate. We test the ability of these three models to predict the population level response of a cell spreading problem for both proliferative and nonproliferative cases. We also explore the potential of our models to predict long time travelling wave invasion rates and extend our results to two dimensional spreading and invasion. Our results show that each model can accurately predict density data for nonproliferative systems, but that only one does so for proliferative systems. Hence great care must be taken to predict density data for with varying cell shape.

Interactive axial shortening of columns and walls in high rise buildings

Concrete is commonly used as a primary construction material for tall building construction. Load bearing components such as columns and walls in concrete buildings are subjected to instantaneous and long term axial shortening caused by the time dependent effects of "shrinkage", "creep" and "elastic" deformations. Reinforcing steel content, variable concrete modulus, volume to surface area ratio of the elements and environmental conditions govern axial shortening. The impact of differential axial shortening among columns and core shear walls escalate with increasing building height. Differential axial shortening of gravity loaded elements in geometrically complex and irregular buildings result in permanent distortion and deflection of the structural frame which have a significant impact on building envelopes, building services, secondary systems and the life time serviceability and performance of a building. Existing numerical methods commonly used in design to quantify axial shortening are mainly based on elastic analytical techniques and therefore unable to capture the complexity of non-linear time dependent effect. Ambient measurements of axial shortening using vibrating wire, external mechanical strain, and electronic strain gauges are methods that are available to verify pre-estimated values from the design stage. Installing these gauges permanently embedded in or on the surface of concrete components for continuous measurements during and after construction with adequate protection is uneconomical, inconvenient and unreliable. Therefore such methods are rarely if ever used in actual practice of building construction. This research project has developed a rigorous numerical procedure that encompasses linear and non-linear time dependent phenomena for prediction of axial shortening of reinforced concrete structural components at design stage. This procedure takes into consideration (i) construction sequence, (ii) time varying values of Young's Modulus of reinforced concrete and (iii) creep and shrinkage models that account for variability resulting from environmental effects. The capabilities of the procedure are illustrated through examples. In order to update previous predictions of axial shortening during the construction and service stages of the building, this research has also developed a vibration based procedure using ambient measurements. This procedure takes into consideration the changes in vibration characteristic of structure during and after construction. The application of this procedure is illustrated through numerical examples which also highlight the features. The vibration based procedure can also be used as a tool to assess structural health/performance of key structural components in the building during construction and service life.

MHD natural convection flow from an isothermal horizontal circular cylinder under consideration of temperature dependent viscosity

Magnetohydrodynamic (MHD) natural convection laminar flow from an iso-thermal horizontal circular cylinder immersed in a fluid with viscosity proportional to a linear function of temperature will be discussed with numerical simulations. The governing boundary layer equations are transformed into a non-dimensional form and the resulting nonlinear system of partial differential equa-tions are reduced to convenient form, which are solved numerically by two very efficient methods, namely, (i) Implicit finite difference method together with Keller box scheme and (ii) Direct numerical scheme. Numerical results are presented by velocity and temperature distributions of the fluid as well as heat transfer characteristics, namely the shearing stress and the local heat transfer rate in terms of the local skin-friction coefficient and the local Nusselt number for a wide range of magnetohydrodynamic parameter, viscosity-variation parameter and viscous dissipation parameter. MHD flow in this geometry with temperature dependent viscosity is absent in the literature. The results obtained from the numerical simulations have been veri-fied by two methodologies.

Renewal decision support for linear assets

Linear (or continuous) assets are engineering infrastructure that usually spans long distances and can be divided into different segments, all of which perform the same function but may be subject to different loads and environmental factors. Typical linear assets include railway lines, roads, pipelines and cables. How and when to renew such assets are critical decisions for asset owners as they normally involves significant capital investment. Through investigating the characteristics of linear asset renewal decisions and identifying the critical requirements that are associated with renewal decisions, we present a multi-criteria decision support method to help optimise renewal decisions. A case study that concerns renewal of an economiser's tubing system is a coal-fired power station is adopted to demonstrate the application of this method. Although the paper concerns a particular linear asset decision type, the approach has broad applicability for linear asset management.

PySSM : a Python module for Bayesian inference of linear Gaussian state space models

PySSM is a Python package that has been developed for the analysis of time series using linear Gaussian state space models (SSM). PySSM is easy to use; models can be set up quickly and efficiently and a variety of different settings are available to the user. It also takes advantage of scientific libraries Numpy and Scipy and other high level features of the Python language. PySSM is also used as a platform for interfacing between optimised and parallelised Fortran routines. These Fortran routines heavily utilise Basic Linear Algebra (BLAS) and Linear Algebra Package (LAPACK) functions for maximum performance. PySSM contains classes for filtering, classical smoothing as well as simulation smoothing.

Optimal coordination of overcurrent relays in distribution systems with distributed generators based on differential evolution algorithm

Distributed generators (DGs) are defined as generators that are connected to a distribution network. The direction of the power flow and short-circuit current in a network could be changed compared with one without DGs. The conventional protective relay scheme does not meet the requirement in this emerging situation. As the number and capacity of DGs in the distribution network increase, the problem of coordinating protective relays becomes more challenging. Given this background, the protective relay coordination problem in distribution systems is investigated, with directional overcurrent relays taken as an example, and formulated as a mixed integer nonlinear programming problem. A mathematical model describing this problem is first developed, and the well-developed differential evolution algorithm is then used to solve it. Finally, a sample system is used to demonstrate the feasiblity and efficiency of the developed method.