Information theory, inference, and learning algorithms

Fun and exciting textbook on the mathematics underpinning the most dynamic areas of modern science and engineering.

Learning the Structure of Deep Sparse Graphical Models

Deep belief networks are a powerful way to model complex probability distributions. However, learning the structure of a belief network, particularly one with hidden units, is difficult. The Indian buffet process has been used as a nonparametric Bayesian prior on the directed structure of a belief network with a single infinitely wide hidden layer. In this paper, we introduce the cascading Indian buffet process (CIBP), which provides a nonparametric prior on the structure of a layered, directed belief network that is unbounded in both depth and width, yet allows tractable inference. We use the CIBP prior with the nonlinear Gaussian belief network so each unit can additionally vary its behavior between discrete and continuous representations. We provide Markov chain Monte Carlo algorithms for inference in these belief networks and explore the structures learned on several image data sets.

Rebuilding the limit order book: Sequential Bayesian inference on hidden states

The limit order book of an exchange represents an information store of market participants' future aims and for many traders the information held in this store is of interest. However, information loss occurs between orders being entered into the exchange and limit order book data being sent out. We present an online algorithm which carries out Bayesian inference to replace information lost at the level of the exchange server and apply our proof of concept algorithm to real historical data from some of the world's most liquid futures contracts as traded on CME GLOBEX, EUREX and NYSE Liffe exchanges. © 2013 © 2013 Taylor & Francis.

Control limitations from distributed sensing: Theory and Extremely Large Telescope application

We investigate performance bounds for feedback control of distributed plants where the controller can be centralized (i.e. it has access to measurements from the whole plant), but sensors only measure differences between neighboring subsystem outputs. Such "distributed sensing" can be a technological necessity in applications where system size exceeds accuracy requirements by many orders of magnitude. We formulate how distributed sensing generally limits feedback performance robust to measurement noise and to model uncertainty, without assuming any controller restrictions (among others, no "distributed control" restriction). A major practical consequence is the necessity to cut down integral action on some modes. We particularize the results to spatially invariant systems and finally illustrate implications of our developments for stabilizing the segmented primary mirror of the European Extremely Large Telescope. © 2013 Elsevier Ltd. All rights reserved.