Optimistic linear programming gives logarithmic regret for irreducible MDPs

We present an algorithm called Optimistic Linear Programming (OLP) for learning to optimize average reward in an irreducible but otherwise unknown Markov decision process (MDP). OLP uses its experience so far to estimate the MDP. It chooses actions by optimistically maximizing estimated future rewards over a set of next-state transition probabilities that are close to the estimates, a computation that corresponds to solving linear programs. We show that the total expected reward obtained by OLP up to time T is within C(P) log T of the reward obtained by the optimal policy, where C(P) is an explicit, MDP-dependent constant. OLP is closely related to an algorithm proposed by Burnetas and Katehakis with four key differences: OLP is simpler, it does not require knowledge of the supports of transition probabilities, the proof of the regret bound is simpler, but our regret bound is a constant factor larger than the regret of their algorithm. OLP is also similar in flavor to an algorithm recently proposed by Auer and Ortner. But OLP is simpler and its regret bound has a better dependence on the size of the MDP.

High-probability regret bounds for bandit online linear optimization

We present a modification of the algorithm of Dani et al. [8] for the online linear optimization problem in the bandit setting, which with high probability has regret at most O ∗ ( √ T) against an adaptive adversary. This improves on the previous algorithm [8] whose regret is bounded in expectation against an oblivious adversary. We obtain the same dependence on the dimension (n 3/2) as that exhibited by Dani et al. The results of this paper rest firmly on those of [8] and the remarkable technique of Auer et al. [2] for obtaining high probability bounds via optimistic estimates. This paper answers an open question: it eliminates the gap between the high-probability bounds obtained in the full-information vs bandit settings.

The differential recognition of children’s cultural practices in middle primary literacy classrooms

This paper argues that teachers’ recognition of children’s cultural practices is an important positive step in helping socio-economically disadvantaged children engage with school literacies. Based on twenty-one longitudinal case studies of children’s literacy development over a three-year period, the authors demonstrate that when children’s knowledges and practices assembled in home and community spheres are treated as valuable material for school learning, children are more likely to invest in the work of acquiring school literacies. However they show also that whilst some children benefit greatly from being allowed to draw on their knowledge of popular culture, sports and the outdoors, other children’s interests may be ignored or excluded. Some differences in teachers’ valuing of home and community cultures appeared to relate to gender dimensions.

Supplement : Efficient weak second-order stochastic Runge-Kutta methods for non-commutative Stratonvich stochastic differential equations

This paper gives a modification of a class of stochastic Runge–Kutta methods proposed in a paper by Komori (2007). The slight modification can reduce the computational costs of the methods significantly.

Compensation of unmodeled aircraft dynamics in airborne inspection of linear infrastructure assets

Fixed-wing aircraft equipped with downward pointing cameras and/or LiDAR can be used for inspecting approximately piecewise linear assets such as oil-gas pipelines, roads and power-lines. Automatic control of such aircraft is important from a productivity and safety point of view (long periods of precision manual flight at low-altitude is not considered reasonable from a safety perspective). This paper investigates the effect of any unwanted coupling between guidance and autopilot loops (typically caused by unmodeled delays in the aircraft’s response), and the specific impact of any unwanted dynamics on the performance of aircraft undertaking inspection of piecewise linear corridor assets (such as powerlines). Simulation studies and experimental flight tests are used to demonstrate the benefits of a simple compensator in mitigating the unwanted lateral oscillatory behaviour (or coupling) that is caused by unmodeled time constants in the aircraft dynamics.

Vision based control for fixed wing UAVs inspecting locally linear infrastructure using skid-to-turn maneuvers

The following paper proposes a novel application of Skid-to-Turn maneuvers for fixed wing Unmanned Aerial Vehicles (UAVs) inspecting locally linear infrastructure. Fixed wing UAVs, following the design of manned aircraft, traditionally employ Bank-to-Turn maneuvers to change heading and thus direction of travel. Commonly overlooked is the effect these maneuvers have on downward facing body fixed sensors, which as a result of bank, point away from the feature during turns. By adopting Skid-to-Turn maneuvers, the aircraft is able change heading whilst maintaining wings level flight, thus allowing body fixed sensors to maintain a downward facing orientation. Eliminating roll also helps to improve data quality, as sensors are no longer subjected to the swinging motion induced as they pivot about an axis perpendicular to their line of sight. Traditional tracking controllers that apply an indirect approach of capturing ground based data by flying directly overhead can also see the feature off center due to steady state pitch and roll required to stay on course. An Image Based Visual Servo controller is developed to address this issue, allowing features to be directly tracked within the image plane. Performance of the proposed controller is tested against that of a Bank-to-Turn tracking controller driven by GPS derived cross track error in a simulation environment developed to simulate the field of view of a body fixed camera.

Protection of microgrids using differential relays

A microgrid provides economical and reliable power to customers by integrating distributed resources more effectively. Islanded operation enables a continuous power supply for loads during a major grid disturbance. Reliability of a microgrid can be further increased by forming a mesh configuration. However, the protection of mesh microgrids is a challenging task. In this paper, protection schemes are discussed using current differential protection of a microgrid. The protection challenges associated with bi-directional power flow, meshed configuration, changing fault current level due to intermittent nature of DGs and reduced fault current level in an islanded mode are considered in proposing the protection solutions. Relay setting criterion and current transformer (CT) selection guidelines are also discussed. The results are verified using MATLAB calculations and PSCAD simulations.

Models of collective cell motion for cell populations with different aspect ratio : diffusion, proliferation and travelling waves

Continuum, partial differential equation models are often used to describe the collective motion of cell populations, with various types of motility represented by the choice of diffusion coefficient, and cell proliferation captured by the source terms. Previously, the choice of diffusion coefficient has been largely arbitrary, with the decision to choose a particular linear or nonlinear form generally based on calibration arguments rather than making any physical connection with the underlying individual-level properties of the cell motility mechanism. In this work we provide a new link between individual-level models, which account for important cell properties such as varying cell shape and volume exclusion, and population-level partial differential equation models. We work in an exclusion process framework, considering aligned, elongated cells that may occupy more than one lattice site, in order to represent populations of agents with different sizes. Three different idealizations of the individual-level mechanism are proposed, and these are connected to three different partial differential equations, each with a different diffusion coefficient; one linear, one nonlinear and degenerate and one nonlinear and nondegenerate. We test the ability of these three models to predict the population level response of a cell spreading problem for both proliferative and nonproliferative cases. We also explore the potential of our models to predict long time travelling wave invasion rates and extend our results to two dimensional spreading and invasion. Our results show that each model can accurately predict density data for nonproliferative systems, but that only one does so for proliferative systems. Hence great care must be taken to predict density data for with varying cell shape.

Interactive axial shortening of columns and walls in high rise buildings

Concrete is commonly used as a primary construction material for tall building construction. Load bearing components such as columns and walls in concrete buildings are subjected to instantaneous and long term axial shortening caused by the time dependent effects of "shrinkage", "creep" and "elastic" deformations. Reinforcing steel content, variable concrete modulus, volume to surface area ratio of the elements and environmental conditions govern axial shortening. The impact of differential axial shortening among columns and core shear walls escalate with increasing building height. Differential axial shortening of gravity loaded elements in geometrically complex and irregular buildings result in permanent distortion and deflection of the structural frame which have a significant impact on building envelopes, building services, secondary systems and the life time serviceability and performance of a building. Existing numerical methods commonly used in design to quantify axial shortening are mainly based on elastic analytical techniques and therefore unable to capture the complexity of non-linear time dependent effect. Ambient measurements of axial shortening using vibrating wire, external mechanical strain, and electronic strain gauges are methods that are available to verify pre-estimated values from the design stage. Installing these gauges permanently embedded in or on the surface of concrete components for continuous measurements during and after construction with adequate protection is uneconomical, inconvenient and unreliable. Therefore such methods are rarely if ever used in actual practice of building construction. This research project has developed a rigorous numerical procedure that encompasses linear and non-linear time dependent phenomena for prediction of axial shortening of reinforced concrete structural components at design stage. This procedure takes into consideration (i) construction sequence, (ii) time varying values of Young's Modulus of reinforced concrete and (iii) creep and shrinkage models that account for variability resulting from environmental effects. The capabilities of the procedure are illustrated through examples. In order to update previous predictions of axial shortening during the construction and service stages of the building, this research has also developed a vibration based procedure using ambient measurements. This procedure takes into consideration the changes in vibration characteristic of structure during and after construction. The application of this procedure is illustrated through numerical examples which also highlight the features. The vibration based procedure can also be used as a tool to assess structural health/performance of key structural components in the building during construction and service life.

MHD natural convection flow from an isothermal horizontal circular cylinder under consideration of temperature dependent viscosity

Magnetohydrodynamic (MHD) natural convection laminar flow from an iso-thermal horizontal circular cylinder immersed in a fluid with viscosity proportional to a linear function of temperature will be discussed with numerical simulations. The governing boundary layer equations are transformed into a non-dimensional form and the resulting nonlinear system of partial differential equa-tions are reduced to convenient form, which are solved numerically by two very efficient methods, namely, (i) Implicit finite difference method together with Keller box scheme and (ii) Direct numerical scheme. Numerical results are presented by velocity and temperature distributions of the fluid as well as heat transfer characteristics, namely the shearing stress and the local heat transfer rate in terms of the local skin-friction coefficient and the local Nusselt number for a wide range of magnetohydrodynamic parameter, viscosity-variation parameter and viscous dissipation parameter. MHD flow in this geometry with temperature dependent viscosity is absent in the literature. The results obtained from the numerical simulations have been veri-fied by two methodologies.