Gaussian processes for classification: mean field algorithms:mean-field algorithms

We derive a mean field algorithm for binary classification with Gaussian processes which is based on the TAP approach originally proposed in Statistical Physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler 'naive' mean field theory and support vector machines (SVM) as limiting cases. For both mean field algorithms and support vectors machines, simulation results for three small benchmark data sets are presented. They show 1. that one may get state of the art performance by using the leave-one-out estimator for model selection and 2. the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The latter result is a taken as a strong support for the internal consistency of the mean field approach.

Gaussian processes and SVM: Mean field results and leave-one-out

In this chapter, we elaborate on the well-known relationship between Gaussian processes (GP) and Support Vector Machines (SVM). Secondly, we present approximate solutions for two computational problems arising in GP and SVM. The first one is the calculation of the posterior mean for GP classifiers using a `naive' mean field approach. The second one is a leave-one-out estimator for the generalization error of SVM based on a linear response method. Simulation results on a benchmark dataset show similar performances for the GP mean field algorithm and the SVM algorithm. The approximate leave-one-out estimator is found to be in very good agreement with the exact leave-one-out error.

TAP for parity check error correcting codes

We employ the methods presented in the previous chapter for decoding corrupted codewords, encoded using sparse parity check error correcting codes. We show the similarity between the equations derived from the TAP approach and those obtained from belief propagation, and examine their performance as practical decoding methods.

Error-correcting codes on a Bethe-like lattice

We analyse Gallager codes by employing a simple mean-field approximation that distorts the model geometry and preserves important interactions between sites. The method naturally recovers the probability propagation decoding algorithm as a minimization of a proper free-energy. We find a thermodynamical phase transition that coincides with information theoretical upper-bounds and explain the practical code performance in terms of the free-energy landscape.

A mean field theory of coded CDMA systems

We present a mean field theory of code-division multiple access (CDMA) systems with error-control coding. On the basis of the relation between the free energy and mutual information, we obtain an analytical expression of the maximum spectral efficiency of the coded CDMA system, from which a mean field description of the coded CDMA system is provided in terms of a bank of scalar Gaussian channels whose variances in general vary at different code symbol positions. Regular low-density parity-check (LDPC)-coded CDMA systems are also discussed as an example of the coded CDMA systems.

Data methods in optometry. Part 6: fitting a regression line to data

1. Fitting a linear regression to data provides much more information about the relationship between two variables than a simple correlation test. A goodness of fit test of the line should always be carried out. Hence, r squared estimates the strength of the relationship between Y and X, ANOVA whether a statistically significant line is present, and the ‘t’ test whether the slope of the line is significantly different from zero. 2. Always check whether the data collected fit the assumptions for regression analysis and, if not, whether a transformation of the Y and/or X variables is necessary. 3. If the regression line is to be used for prediction, it is important to determine whether the prediction involves an individual y value or a mean. Care should be taken if predictions are made close to the extremities of the data and are subject to considerable error if x falls beyond the range of the data. Multiple predictions require correction of the P values. 3. If several individual regression lines have been calculated from a number of similar sets of data, consider whether they should be combined to form a single regression line. 4. If the data exhibit a degree of curvature, then fitting a higher-order polynomial curve may provide a better fit than a straight line. In this case, a test of whether the data depart significantly from a linear regression should be carried out.

Bayesian error bars for regression

Regression problems are concerned with predicting the values of one or more continuous quantities, given the values of a number of input variables. For virtually every application of regression, however, it is also important to have an indication of the uncertainty in the predictions. Such uncertainties are expressed in terms of the error bars, which specify the standard deviation of the distribution of predictions about the mean. Accurate estimate of error bars is of practical importance especially when safety and reliability is an issue. The Bayesian view of regression leads naturally to two contributions to the error bars. The first arises from the intrinsic noise on the target data, while the second comes from the uncertainty in the values of the model parameters which manifests itself in the finite width of the posterior distribution over the space of these parameters. The Hessian matrix which involves the second derivatives of the error function with respect to the weights is needed for implementing the Bayesian formalism in general and estimating the error bars in particular. A study of different methods for evaluating this matrix is given with special emphasis on the outer product approximation method. The contribution of the uncertainty in model parameters to the error bars is a finite data size effect, which becomes negligible as the number of data points in the training set increases. A study of this contribution is given in relation to the distribution of data in input space. It is shown that the addition of data points to the training set can only reduce the local magnitude of the error bars or leave it unchanged. Using the asymptotic limit of an infinite data set, it is shown that the error bars have an approximate relation to the density of data in input space.

Mean sea surface modelling from multi-satellite altimetry utilising frequency domain techniques

A technique is presented for the development of a high precision and resolution Mean Sea Surface (MSS) model. The model utilises Radar altimetric sea surface heights extracted from the geodetic phase of the ESA ERS-1 mission. The methodology uses a modified Le Traon et al. (1995) cubic-spline fit of dual ERS-1 and TOPEX/Poseidon crossovers for the minimisation of radial orbit error. The procedure then uses Fourier domain processing techniques for spectral optimal interpolation of the mean sea surface in order to reduce residual errors within the model. Additionally, a multi-satellite mean sea surface integration technique is investigated to supplement the first model with additional enhanced data from the GEOSAT geodetic mission.The methodology employs a novel technique that combines the Stokes' and Vening-Meinsz' transformations, again in the spectral domain. This allows the presentation of a new enhanced GEOSAT gravity anomaly field.

Evaluation of the measurement of refractive error by the PowerRefractor:a remote, continuous and binocular measurement system of oculomotor function

Background/aim: The technique of photoretinoscopy is unique in being able to measure the dynamics of the oculomotor system (ocular accommodation, vergence, and pupil size) remotely (working distance typically 1 metre) and objectively in both eyes simultaneously. The aim af this study was to evaluate clinically the measurement of refractive error by a recent commercial photoretinoscopic device, the PowerRefractor (PlusOptiX, Germany). Method: The validity and repeatability of the PowerRefractor was compared to: subjective (non-cycloplegic) refraction on 100 adult subjects (mean age 23.8 (SD 5.7) years) and objective autarefractian (Shin-Nippon SRW-5000, Japan) on 150 subjects (20.1 (4.2) years). Repeatability was assessed by examining the differences between autorefractor readings taken from each eye and by re-measuring the objective prescription of 100 eyes at a subsequent session. Results: On average the PowerRefractor prescription was not significantly different from the subjective refraction, although quite variable (difference -0.05 (0.63) D, p = 0.41) and more negative than the SRW-5000 prescription (by -0.20 (0.72) D, p<0.001). There was no significant bias in the accuracy of the instrument with regard to the type or magnitude of refractive error. The PowerRefractor was found to be repeatable over the prescription range of -8.75D to +4.00D (mean spherical equivalent) examined. Conclusion: The PowerRefractor is a useful objective screening instrument and because of its remote and rapid measurement of both eyes simultaneously is able to assess the oculomotor response in a variety of unrestricted viewing conditions and patient types.

Refractive error, cognitive demand and nearwork-induced transient myopia

Purpose. Whereas many previous studies have identified the association between sustained near work and myopia, few have assessed the influence of concomitant levels of cognitive effort. This study investigates the effect of cognitive effort on near-work induced transient myopia (NITM). Methods. Subjects comprised of six early onset myopes (EOM; mean age 23.7 yrs; mean onset 10.8 yrs), six late-onset myopes (LOM; mean age 23.2 yrs; mean onset 20.0 yrs) and six emmetropes (EMM; mean age 23.8 yrs). Dynamic, monocular, ocular accommodation was measured with the Shin-Nippon SRW-5000 autorefractor. Subjects engaged passively or actively in a 5 minute arithmetic sum checking task presented monocularly on an LCD monitor via a Badal optical system. In all conditions the task was initially located at near (4.50 D) and immediately following the task instantaneously changed to far (0.00 D) for a further 5 minutes. The combinations of active (A) and passive (P) cognition were randomly allocated as P:P; A:P; A:A; P:A. Results. For the initial near task, LOMs were shown to have a significantly less accurate accommodative response than either EOMs or EMMs (p < 0.001). For the far task, post hoc analyses for refraction identified EOMs as demonstrating significant NITM compared to LOMs (p < 0.05), who in turn showed greater NITM than EMMs (p < 0.001). The data show that for EOMs the level of cognitive activity operating during the near and far tasks determines the persistence of NITM; persistence being maximal when active cognition at near is followed by passive cognition at far. Conclusions. Compared with EMMs, EOMs and LOMs are particularly susceptible to NITM such that sustained near vision reduces subsequent accommodative accuracy for far vision. It is speculated that the marked NITM found in EOM may be a consequence of the crystalline lens thinning shown to be a developmental feature of EOM. Whereas the role of small amounts of retinal defocus in myopigenesis remains equivocal, the results show that account needs to be taken of cognitive demand in assessing phenomena such as NITM.

The influence on unaided vision of age, pupil diameter and spherocylindrical refractive error

Background - The aim was to derive equations for the relationship between unaided vision and age, pupil diameter, iris colour and sphero-cylindrical refractive error. Methods - Data were collected from 663 healthy right eyes of white subjects aged 20 to 70 years. Subjective sphero-cylindrical refractive errors ranged from -6.8 to +9.4 D (mean spherical equivalent), -1.5 to +1.9 D (orthogonal component, J0) and -0.8 to 1.0 D (oblique component, J45). Cylinder axis orientation was orthogonal in 46 per cent of the eyes and oblique in 18 per cent. Unaided vision (-0.3 to +1.3 logMAR), pupil diameter (2.3 to 7.5 mm) and iris colour (67 per cent light/blue irides) was recorded. The sample included mostly females (60 per cent) and many contact lens wearers (42 per cent) and so the influences of these parameters were also investigated. Results - Decision tree analysis showed that sex, iris colour, contact lens wear and cylinder axis orientation did not influence the relationship between unaided vision and refractive error. New equations for the dependence of the minimum angle of resolution on age and pupil diameter arose from step backwards multiple linear regressions carried out separately on the myopes (2.91.scalar vector +0.51.pupil diameter -3.14 ) and hyperopes (1.55.scalar vector + 0.06.age – 3.45 ). Conclusion - The new equations may be useful in simulators designed for teaching purposes as they accounted for 81 per cent (for myopes) and 53 per cent (for hyperopes) of the variance in measured data. In comparison, previously published equations accounted for not more than 76 per cent (for myopes) and 24 per cent (for hyperopes) of the variance depending on whether they included pupil size. The new equations are, as far as is known to the authors, the first to include age. The age-related decline in accommodation is reflected in the equation for hyperopes.

Intraoperative prediction of refractive error using a prototype surgical aberrometer

Purpose: To ascertain the agreement level between intra-operative refraction using a prototype surgical Hartmann-Shack aberrometer and subjective refraction a month later. Methods: Fifty-four consecutive patients had their pseudophakic refractive measured with the aberrometer intra-operatively at the end of their cataract surgery. A masked optometrist performed subjective refraction 4 weeks later. The two sets of data were then analysed for correlation. Results: The mean spherical equivalent was −0.14 ± 0.37 D (Range: −1.41 to +1.72 D) with the prototype aberrometer and −0.34 ± 0.32 (−1.64 to +1.88 D) with subjective refraction. The measurements positively correlated to a very high degree (r =+0.81, p < 0.01). In 84.3% of cases the two measurements were within 0.50D of each other. Conclusion: The aberrometer can verify the aimed refractive status of the eye intraoperatively to avoid a refractive surprise. The aberrometer is a useful tool for real time assessment of the ocular refractive status.