Bayesian learning of noisy Markov decision processes

This work addresses the problem of estimating the optimal value function in a Markov Decision Process from observed state-action pairs. We adopt a Bayesian approach to inference, which allows both the model to be estimated and predictions about actions to be made in a unified framework, providing a principled approach to mimicry of a controller on the basis of observed data. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from theposterior distribution over the optimal value function. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Designing and testing an advanced pneumatic braking system for heavy vehicles

Heavy goods vehicles exhibit poor braking performance in emergency situations when compared to other vehicles. Part of the problem is caused by sluggish pneumatic brake actuators, which limit the control bandwidth of their antilock braking systems. In addition, heuristic control algorithms are used that do not achieve the maximum braking force throughout the stop. In this article, a novel braking system is introduced for pneumatically braked heavy goods vehicles. The conventional brake actuators are improved by placing high-bandwidth, binary-actuated valves directly on the brake chambers. A made-for-purpose valve is described. It achieves a switching delay of 3-4 ms in tests, which is an order of magnitude faster than solenoids in conventional anti-lock braking systems. The heuristic braking control algorithms are replaced with a wheel slip regulator based on sliding mode control. The combined actuator and slip controller are shown to reduce stopping distances on smooth and rough, high friction (μ = 0.9) surfaces by 10% and 27% respectively in hardware-in-the-loop tests compared with conventional ABS. On smooth and rough, low friction (μ = 0.2) surfaces, stopping distances are reduced by 23% and 25%, respectively. Moreover, the overall air reservoir size required on a heavy goods vehicle is governed by its air usage during an anti-lock braking stop on a low friction, smooth surface. The 37% reduction in air usage observed in hardware-in-the-loop tests on this surface therefore represents the potential reduction in reservoir size that could be achieved by the new system. © 2012 IMechE.

Manopt, a matlab toolbox for optimization on manifolds

Optimization on manifolds is a rapidly developing branch of nonlinear optimization. Its focus is on problems where the smooth geometry of the search space can be leveraged to design effcient numerical algorithms. In particular, optimization on manifolds is well-suited to deal with rank and orthogonality constraints. Such structured constraints appear pervasively in machine learning applications, including low-rank matrix completion, sensor network localization, camera network registration, independent component analysis, metric learning, dimensionality reduction and so on. The Manopt toolbox, available at www.manopt.org, is a user-friendly, documented piece of software dedicated to simplify experimenting with state of the art Riemannian optimization algorithms. By dealing internally with most of the differential geometry, the package aims particularly at lowering the entrance barrier. © 2014 Nicolas Boumal.