Information theory, inference, and learning algorithms

Fun and exciting textbook on the mathematics underpinning the most dynamic areas of modern science and engineering.

Automated 3D Structure Inference of Civil Infrastructure Using a Stereo Camera Set

The commercial far-range (>10m) infrastructure spatial data collection methods are not completely automated. They need significant amount of manual post-processing work and in some cases, the equipment costs are significant. This paper presents a method that is the first step of a stereo videogrammetric framework and holds the promise to address these issues. Under this method, video streams are initially collected from a calibrated set of two video cameras. For each pair of simultaneous video frames, visual feature points are detected and their spatial coordinates are then computed. The result, in the form of a sparse 3D point cloud, is the basis for the next steps in the framework (i.e., camera motion estimation and dense 3D reconstruction). A set of data, collected from an ongoing infrastructure project, is used to show the merits of the method. Comparison with existing tools is also shown, to indicate the performance differences of the proposed method in the level of automation and the accuracy of results.

Linear gaussian computations for near-exact Bayesian Monte Carlo inference in skewed alpha-stable time series models

In this paper we study parameter estimation for time series with asymmetric α-stable innovations. The proposed methods use a Poisson sum series representation (PSSR) for the asymmetric α-stable noise to express the process in a conditionally Gaussian framework. That allows us to implement Bayesian parameter estimation using Markov chain Monte Carlo (MCMC) methods. We further enhance the series representation by introducing a novel approximation of the series residual terms in which we are able to characterise the mean and variance of the approximation. Simulations illustrate the proposed framework applied to linear time series, estimating the model parameter values and model order P for an autoregressive (AR(P)) model driven by asymmetric α-stable innovations. © 2012 IEEE.

Decomposing signals into a sum of amplitude and frequency modulated sinusoids using probabilistic inference

There are many methods for decomposing signals into a sum of amplitude and frequency modulated sinusoids. In this paper we take a new estimation based approach. Identifying the problem as ill-posed, we show how to regularize the solution by imposing soft constraints on the amplitude and phase variables of the sinusoids. Estimation proceeds using a version of Kalman smoothing. We evaluate the method on synthetic and natural, clean and noisy signals, showing that it outperforms previous decompositions, but at a higher computational cost. © 2012 IEEE.

Optimal inference of sameness.

Deciding whether a set of objects are the same or different is a cornerstone of perception and cognition. Surprisingly, no principled quantitative model of sameness judgment exists. We tested whether human sameness judgment under sensory noise can be modeled as a form of probabilistically optimal inference. An optimal observer would compare the reliability-weighted variance of the sensory measurements with a set size-dependent criterion. We conducted two experiments, in which we varied set size and individual stimulus reliabilities. We found that the optimal-observer model accurately describes human behavior, outperforms plausible alternatives in a rigorous model comparison, and accounts for three key findings in the animal cognition literature. Our results provide a normative footing for the study of sameness judgment and indicate that the notion of perception as near-optimal inference extends to abstract relations.

Statistical inference for single- and multi-band probabilistic amplitude demodulation.

Amplitude demodulation is an ill-posed problem and so it is natural to treat it from a Bayesian viewpoint, inferring the most likely carrier and envelope under probabilistic constraints. One such treatment is Probabilistic Amplitude Demodulation (PAD), which, whilst computationally more intensive than traditional approaches, offers several advantages. Here we provide methods for estimating the uncertainty in the PAD-derived envelopes and carriers, and for learning free-parameters like the time-scale of the envelope. We show how the probabilistic approach can naturally handle noisy and missing data. Finally, we indicate how to extend the model to signals which contain multiple modulators and carriers.

Model reductions for inference: generality of pairwise, binary, and planar factor graphs.

We offer a solution to the problem of efficiently translating algorithms between different types of discrete statistical model. We investigate the expressive power of three classes of model-those with binary variables, with pairwise factors, and with planar topology-as well as their four intersections. We formalize a notion of "simple reduction" for the problem of inferring marginal probabilities and consider whether it is possible to "simply reduce" marginal inference from general discrete factor graphs to factor graphs in each of these seven subclasses. We characterize the reducibility of each class, showing in particular that the class of binary pairwise factor graphs is able to simply reduce only positive models. We also exhibit a continuous "spectral reduction" based on polynomial interpolation, which overcomes this limitation. Experiments assess the performance of standard approximate inference algorithms on the outputs of our reductions.

Characterization and control of a pneumatic microactuator with an integrated inductive position sensor

As the intelligence and the functionality of microrobots increase, there is a growing need to incorporate sensors into these robots. In order to limit the outer dimensions of these microsystems, this research investigates sensors that can be integrated efficiently into microactuators. Here, a pneumatic piston-cylinder microactuator with an integrated inductive position sensor was developed. The main advantage of pneumatic actuators is their high force and power density at microscale. The outside diameter of the actuator is 1.3 mm and the length is 15 mm. The stroke of the actuator is 12 mm, and the actuation force is 1 N at a supply pressure of 1.5 MPa. The position sensor consists of two coils wound around the cylinder of the actuator. The measurement principle is based on the change in coupling factor between the coils as the piston moves in the actuator. The sensor is extremely small since one layer of 25 μm copper wire is sufficient to achieve an accuracy of 10 μm over the total stroke. Position tests with a PI controller and a sliding mode controller showed that the actuator is able to position with an accuracy up to 30 μm. Such positioning systems offer great opportunities for all devices that need to control a large number of degrees of freedom in a restricted volume. © 2007 Elsevier B.V. All rights reserved.

Neural mechanisms of belief inference during cooperative games.

Humans have the arguably unique ability to understand the mental representations of others. For success in both competitive and cooperative interactions, however, this ability must be extended to include representations of others' belief about our intentions, their model about our belief about their intentions, and so on. We developed a "stag hunt" game in which human subjects interacted with a computerized agent using different degrees of sophistication (recursive inferences) and applied an ecologically valid computational model of dynamic belief inference. We show that rostral medial prefrontal (paracingulate) cortex, a brain region consistently identified in psychological tasks requiring mentalizing, has a specific role in encoding the uncertainty of inference about the other's strategy. In contrast, dorsolateral prefrontal cortex encodes the depth of recursion of the strategy being used, an index of executive sophistication. These findings reveal putative computational representations within prefrontal cortex regions, supporting the maintenance of cooperation in complex social decision making.

Rebuilding the limit order book: Sequential Bayesian inference on hidden states

The limit order book of an exchange represents an information store of market participants' future aims and for many traders the information held in this store is of interest. However, information loss occurs between orders being entered into the exchange and limit order book data being sent out. We present an online algorithm which carries out Bayesian inference to replace information lost at the level of the exchange server and apply our proof of concept algorithm to real historical data from some of the world's most liquid futures contracts as traded on CME GLOBEX, EUREX and NYSE Liffe exchanges. © 2013 © 2013 Taylor & Francis.