Dynamics of on-line gradient descent learning for multilayer neural networks

We consider the problem of on-line gradient descent learning for general two-layer neural networks. An analytic solution is presented and used to investigate the role of the learning rate in controlling the evolution and convergence of the learning process.

Transients and asymptotics of natural gradient learning

We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.

Natural gradient matrix momentum

Natural gradient learning is an efficient and principled method for improving on-line learning. In practical applications there will be an increased cost required in estimating and inverting the Fisher information matrix. We propose to use the matrix momentum algorithm in order to carry out efficient inversion and study the efficacy of a single step estimation of the Fisher information matrix. We analyse the proposed algorithm in a two-layer network, using a statistical mechanics framework which allows us to describe analytically the learning dynamics, and compare performance with true natural gradient learning and standard gradient descent.

Phakometric measurement of ocular surface radii of curvature, axial separations and alignment in relaxed and accommodated human eyes

Measurements (autokeratometry, A-scan ultrasonography and video ophthalmophakometry) of ocular surface radii, axial separations and alignment were made in the horizontal meridian of nine emmetropes (aged 20-38 years) with relaxed (cycloplegia) and active accommodation (mean ± 95% confidence interval: 3.7 ± 1.1 D). The anterior chamber depth (-1.5 ± 0.3 D) and both crystalline lens surfaces (front 3.1 ± 0.8 D; rear 2.1 ± 0.6 D) contributed to dioptric vergence changes that accompany accommodation. Accommodation did not alter ocular surface alignment. Ocular misalignment in relaxed eyes is mainly because of eye rotation (5.7 ± 1.6° temporally) with small amounts of lens tilt (0.2 ± 0.8° temporally) and decentration (0.1 ± 0.1 mm nasally) but these results must be viewed with caution as we did not account for corneal asymmetry. Comparison of calculated and empirically derived coefficients (upon which ocular surface alignment calculations depend) revealed that negligible inherent errors arose from neglect of ocular surface asphericity, lens gradient refractive index properties, surface astigmatism, effects of pupil size and centration, assumed eye rotation axis position and use of linear equations for analysing Purkinje image shifts. © 2004 The College of Optometrists.

Blurred edges look faint, and faint edges look sharp:The effect of a gradient threshold in a multi-scale edge coding model

A multi-scale model of edge coding based on normalized Gaussian derivative filters successfully predicts perceived scale (blur) for a wide variety of edge profiles [Georgeson, M. A., May, K. A., Freeman, T. C. A., & Hesse, G. S. (in press). From filters to features: Scale-space analysis of edge and blur coding in human vision. Journal of Vision]. Our model spatially differentiates the luminance profile, half-wave rectifies the 1st derivative, and then differentiates twice more, to give the 3rd derivative of all regions with a positive gradient. This process is implemented by a set of Gaussian derivative filters with a range of scales. Peaks in the inverted normalized 3rd derivative across space and scale indicate the positions and scales of the edges. The edge contrast can be estimated from the height of the peak. The model provides a veridical estimate of the scale and contrast of edges that have a Gaussian integral profile. Therefore, since scale and contrast are independent stimulus parameters, the model predicts that the perceived value of either of these parameters should be unaffected by changes in the other. This prediction was found to be incorrect: reducing the contrast of an edge made it look sharper, and increasing its scale led to a decrease in the perceived contrast. Our model can account for these effects when the simple half-wave rectifier after the 1st derivative is replaced by a smoothed threshold function described by two parameters. For each subject, one pair of parameters provided a satisfactory fit to the data from all the experiments presented here and in the accompanying paper [May, K. A. & Georgeson, M. A. (2007). Added luminance ramp alters perceived edge blur and contrast: A critical test for derivative-based models of edge coding. Vision Research, 47, 1721-1731]. Thus, when we allow for the visual system's insensitivity to very shallow luminance gradients, our multi-scale model can be extended to edge coding over a wide range of contrasts and blurs. © 2007 Elsevier Ltd. All rights reserved.

Resonant gradient force on atom interacting with laser field

The gradient force, as a function of position and velocity, is derived for a two-level atom interacting with a standing-wave laser field. Basing on optical Bloch equations, the numerical solutions for the gradient force f_(|_;n) (n = 0, 1, 2, 3, 4, ...) pointing in the direction of the transverse of the laser beam are given. It is shown the higher order gradient force plays important role at strong intensity (G = 64), the contribution of them can not be neglected.

The investigation of the sensitivity gradient of the visual field, as a function of stimulus dynamic range, in the normal and abnormal eye

The study utilized the advanced technology provided by automated perimeters to investigate the hypothesis that patients with retinitis pigmentosa behave atypically over the dynamic range and to concurrently determine the influence of extraneous factors on the format of the normal perimetric sensitivity profile. The perimetric processing of some patients with retinitis pigmentosa was considered to be abnormal in either the temporal and/or the spatial domain. The standard size III stimulus saturated the central regions and was thus ineffective in detecting early depressions in sensitivity in these areas. When stimulus size was scaled in inverse proportion to the square root of ganglion cell receptive field density (M-scaled), isosensitive profiles did not result, although cortical representation was theoretically equivalent across the visual field. It was conjectured that this was due to variations in the ganglion cell characteristics with increasing peripheral angle, most notably spatial summation. It was concluded that the development of perimetric routines incorporating stimulus sizes adjusted in proportion to the coverage factor of retinal ganglion cells would enhance the diagnostic capacity of perimetry. Good general and local correspondence was found between perimetric sensitivity and the available retinal cell counts. Intraocular light scatter arising both from simulations and media opacities depressed perimetric sensitivity. Attenuation was greater centrally for the smaller LED stimuli, whereas the reverse was true for the larger projected stimuli. Prior perimetric experience and pupil size also demonstrated eccentricity-dependent effect on sensitivity. Practice improved perimetric sensitivity for projected stimuli at eccentricities greater than or equal to 30o; particularly in the superior region. Increase in pupil size for LED stimuli enhanced sensitivity at eccentricities greater than 10o. Conversely, microfluctuation in the accommodative response during perimetric examination and the correction of peripheral refractive error had no significant influence on perimetric sensitivity.

Comparison of techniques for measuring anterior chamber depth:Orbscan imaging, Smith's technique, and van Herick's method

Background Evaluation of anterior chamber depth (ACD) can potentially identify those patients at risk of angle-closure glaucoma. We aimed to: compare van Herick’s limbal chamber depth (LCDvh) grades with LCDorb grades calculated from the Orbscan anterior chamber angle values; determine Smith’s technique ACD and compare to Orbscan ACD; and calculate a constant for Smith’s technique using Orbscan ACD. Methods Eighty participants free from eye disease underwent LCDvh grading, Smith’s technique ACD, and Orbscan anterior chamber angle and ACD measurement. Results LCDvh overestimated grades by a mean of 0.25 (coefficient of repeatability [CR] 1.59) compared to LCDorb. Smith’s technique (constant 1.40 and 1.31) overestimated ACD by a mean of 0.33 mm (CR 0.82) and 0.12 mm (CR 0.79) respectively, compared to Orbscan. Using linear regression, we determined a constant of 1.22 for Smith’s slit-length method. Conclusions Smith’s technique (constant 1.31) provided an ACD that is closer to that found with Orbscan compared to a constant of 1.40 or LCDvh. Our findings also suggest that Smith’s technique would produce values closer to that obtained with Orbscan by using a constant of 1.22.

Shape of the posterior vitreous chamber in human emmetropia and myopia

PURPOSE - To compare posterior vitreous chamber shape in myopia to that in emmetropia. METHODS - Both eyes of 55 adult subjects were studied, 27 with emmetropia (MSE =-0.55; <+0.75D; mean +0.09 ±0.36D) and 28 with myopia (MSE -5.87 ±2.31D). Cycloplegic refraction was measured with a Shin Nippon autorefractor and anterior chamber depth and axial length with a Zeiss IOLMaster. Posterior vitreous chamber shapes were determined from T2-weighted MRI (3-Tesla) using procedures previously reported by our laboratory. 3-D surface model coordinates were assigned to nasal, temporal, superior and inferior quadrants and plotted in 2-D to illustrate the composite shape of respective quadrants posterior to the second nodal point. Spherical analogues of chamber shape were constructed to compare relative sphericity between refractive groups and quadrants. RESULTS - Differences in shape occurred in the region posterior to points of maximum globe width and were thus in general accord with an equatorial model of myopic expansion. Shape in emmetropia is categorised distinctly as that of an oblate ellipse and in myopia as an oblate ellipse of significantly less degree such that it approximates to a sphere. There was concordance between shape and retinotopic projection of respective quadrants into right, left, superior and inferior visual fields. CONCLUSIONS - The transition in shape from oblate ellipse to sphere with axial elongation supports the hypothesis that myopia may be a consequence of equatorial restriction associated with biomechanical anomalies of the ciliary apparatus. The synchronisation of quadrant shapes with retinotopic projection suggests that binocular growth is coordinated by processes that operate beyond the optic chiasm.

Computationally efficient natural gradient descent

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT