The dynamics of matrix momentum

We analyse the matrix momentum algorithm, which provides an efficient approximation to on-line Newton's method, by extending a recent statistical mechanics framework to include second order algorithms. We study the efficacy of this method when the Hessian is available and also consider a practical implementation which uses a single example estimate of the Hessian. The method is shown to provide excellent asymptotic performance, although the single example implementation is sensitive to the choice of training parameters. We conjecture that matrix momentum could provide efficient matrix inversion for other second order algorithms.

Graphical modelling for brain connectivity via partial coherence

Spectral and coherence methodologies are ubiquitous for the analysis of multiple time series. Partial coherence analysis may be used to try to determine graphical models for brain functional connectivity. The outcome of such an analysis may be considerably influenced by factors such as the degree of spectral smoothing, line and interference removal, matrix inversion stabilization and the suppression of effects caused by side-lobe leakage, the combination of results from different epochs and people, and multiple hypothesis testing. This paper examines each of these steps in turn and provides a possible path which produces relatively ‘clean’ connectivity plots. In particular we show how spectral matrix diagonal up-weighting can simultaneously stabilize spectral matrix inversion and reduce effects caused by side-lobe leakage, and use the stepdown multiple hypothesis test procedure to help formulate an interaction strength.

Simultaneous measurement of strain and temperature using the first- and second-order diffraction wavelengths of Bragg gratings

A method of discriminating between temperature and strain effects in fibre sensing using a conventionally written, in-fibre Bragg grating is presented. The technique uses wavelength information from the first and second diffraction orders of the grating element to determine the wavelength dependent strain and temperature coefficients, from which independent temperature and strain measurements can be made. The authors present results that validate this matrix inversion technique and quantify the strain and temperature errors which can arise for a given uncertainty in the measurement of the reflected wavelength.

Simultaneous measurement of strain and temperature using the first- and second-order diffraction wavelengths of Bragg gratings

A method of discriminating between temperature and strain effects in fibre sensing using a conventionally written, in-fibre Bragg grating is presented. The technique uses wavelength information from the first and second diffraction orders of the grating element to determine the wavelength dependent strain and temperature coefficients, from which independent temperature and strain measurements can be made. The authors present results that validate this matrix inversion technique and quantify the strain and temperature errors which can arise for a given uncertainty in the measurement of the reflected wavelength.

Natural gradient matrix momentum

Natural gradient learning is an efficient and principled method for improving on-line learning. In practical applications there will be an increased cost required in estimating and inverting the Fisher information matrix. We propose to use the matrix momentum algorithm in order to carry out efficient inversion and study the efficacy of a single step estimation of the Fisher information matrix. We analyse the proposed algorithm in a two-layer network, using a statistical mechanics framework which allows us to describe analytically the learning dynamics, and compare performance with true natural gradient learning and standard gradient descent.