GTM: the generative topographic mapping

Latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. A familiar example is factor analysis which is based on a linear transformations between the latent space and the data space. In this paper we introduce a form of non-linear latent variable model called the Generative Topographic Mapping, for which the parameters of the model can be determined using the EM algorithm. GTM provides a principled alternative to the widely used Self-Organizing Map (SOM) of Kohonen (1982), and overcomes most of the significant limitations of the SOM. We demonstrate the performance of the GTM algorithm on a toy problem and on simulated data from flow diagnostics for a multi-phase oil pipeline.

Topographic mappings and feed-forward neural networks

This thesis is a study of the generation of topographic mappings - dimension reducing transformations of data that preserve some element of geometric structure - with feed-forward neural networks. As an alternative to established methods, a transformational variant of Sammon's method is proposed, where the projection is effected by a radial basis function neural network. This approach is related to the statistical field of multidimensional scaling, and from that the concept of a 'subjective metric' is defined, which permits the exploitation of additional prior knowledge concerning the data in the mapping process. This then enables the generation of more appropriate feature spaces for the purposes of enhanced visualisation or subsequent classification. A comparison with established methods for feature extraction is given for data taken from the 1992 Research Assessment Exercise for higher educational institutions in the United Kingdom. This is a difficult high-dimensional dataset, and illustrates well the benefit of the new topographic technique. A generalisation of the proposed model is considered for implementation of the classical multidimensional scaling (¸mds}) routine. This is related to Oja's principal subspace neural network, whose learning rule is shown to descend the error surface of the proposed ¸mds model. Some of the technical issues concerning the design and training of topographic neural networks are investigated. It is shown that neural network models can be less sensitive to entrapment in the sub-optimal global minima that badly affect the standard Sammon algorithm, and tend to exhibit good generalisation as a result of implicit weight decay in the training process. It is further argued that for ideal structure retention, the network transformation should be perfectly smooth for all inter-data directions in input space. Finally, there is a critique of optimisation techniques for topographic mappings, and a new training algorithm is proposed. A convergence proof is given, and the method is shown to produce lower-error mappings more rapidly than previous algorithms.

GTM: the generative topographic mapping

Latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. A familiar example is factor analysis which is based on a linear transformations between the latent space and the data space. In this paper we introduce a form of non-linear latent variable model called the Generative Topographic Mapping, for which the parameters of the model can be determined using the EM algorithm. GTM provides a principled alternative to the widely used Self-Organizing Map (SOM) of Kohonen (1982), and overcomes most of the significant limitations of the SOM. We demonstrate the performance of the GTM algorithm on a toy problem and on simulated data from flow diagnostics for a multi-phase oil pipeline.

Developments of the generative topographic mapping

The generative topographic mapping (GTM) model was introduced by Bishop et al. (1998, Neural Comput. 10(1), 215-234) as a probabilistic re- formulation of the self-organizing map (SOM). It offers a number of advantages compared with the standard SOM, and has already been used in a variety of applications. In this paper we report on several extensions of the GTM, including an incremental version of the EM algorithm for estimating the model parameters, the use of local subspace models, extensions to mixed discrete and continuous data, semi-linear models which permit the use of high-dimensional manifolds whilst avoiding computational intractability, Bayesian inference applied to hyper-parameters, and an alternative framework for the GTM based on Gaussian processes. All of these developments directly exploit the probabilistic structure of the GTM, thereby allowing the underlying modelling assumptions to be made explicit. They also highlight the advantages of adopting a consistent probabilistic framework for the formulation of pattern recognition algorithms.

GTM: the generative topographic mapping

This thesis describes the Generative Topographic Mapping (GTM) --- a non-linear latent variable model, intended for modelling continuous, intrinsically low-dimensional probability distributions, embedded in high-dimensional spaces. It can be seen as a non-linear form of principal component analysis or factor analysis. It also provides a principled alternative to the self-organizing map --- a widely established neural network model for unsupervised learning --- resolving many of its associated theoretical problems. An important, potential application of the GTM is visualization of high-dimensional data. Since the GTM is non-linear, the relationship between data and its visual representation may be far from trivial, but a better understanding of this relationship can be gained by computing the so-called magnification factor. In essence, the magnification factor relates the distances between data points, as they appear when visualized, to the actual distances between those data points. There are two principal limitations of the basic GTM model. The computational effort required will grow exponentially with the intrinsic dimensionality of the density model. However, if the intended application is visualization, this will typically not be a problem. The other limitation is the inherent structure of the GTM, which makes it most suitable for modelling moderately curved probability distributions of approximately rectangular shape. When the target distribution is very different to that, theaim of maintaining an `interpretable' structure, suitable for visualizing data, may come in conflict with the aim of providing a good density model. The fact that the GTM is a probabilistic model means that results from probability theory and statistics can be used to address problems such as model complexity. Furthermore, this framework provides solid ground for extending the GTM to wider contexts than that of this thesis.

The topographic distribution of the magnetic P100M to full- and half-field stimulation

Visual evoked magnetic responses were recorded to full-field and left and right half-field stimulation with three check sizes (70′, 34′ and 22′) in five normal subjects. Recordings were made sequentially on a 20-position grid (4 × 5) based on the inion, by means of a single-channel direct current-Superconducting Quantum Interference Device second-order gradiometer. The topographic maps were consistent on the same subjects recorded 2 months apart. The half-field responses produced the strongest signals in the contralateral hemisphere and were consistent with the cruciform model of the calcarine fissure. Right half fields produced upper-left-quadrant outgoing fields and lower-left-quadrant ingoing fields, while the left half field produced the opposite response. The topographic maps also varied with check size, with the larger checks producing positive or negative maximum position more anteriorly than small checks. In addition, with large checks the full-field responses could be explained as the summation of the two half fields, whereas full-field responses to smaller checks were more unpredictable and may be due to sources located at the occipital pole or lateral surface. In addition, dipole sources were located as appropriate with the use of inverse problem solutions. Topographic data will be vital to the clinical use of the visual evoked field but, in addition, provides complementary information to visual evoked potentials, allowing detailed studies of the visual cortex. © 1992 Kluwer Academic Publishers.

Multiple system atrophy (MSA): Topographic distribution of the alpha-synuclein-associated pathological changes

To study the topographic distribution of the pathology in multiple system atrophy (MSA). Pattern analysis was carried out using a-synuclein immunohistochemistry in 10 MSA cases. The glial cytoplasmic inclusions (GCI) were distributed randomly or in large clusters. The neuronal inclusions (NI) and abnormal neurons were distributed in regular clusters. Clusters of the NI and abnormal neurons were spatially correlated whereas the GCI were not spatially correlated with either the NI or the abnormal neurons. The data suggest that the GCI represent the primary change in MSA and the neuronal pathology develops secondary to the glial pathology.

Constructing localized non-linear projection manifolds in a principled way:hierarchical generative topographic mapping

It has been argued that a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets, and therefore a hierarchical visualization system is desirable. In this paper we extend an existing locally linear hierarchical visualization system PhiVis (Bishop98a) in several directions: 1. We allow for em non-linear projection manifolds. The basic building block is the Generative Topographic Mapping. 2. We introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree. General training equations are derived, regardless of the position of the model in the tree. 3. Using tools from differential geometry we derive expressions for local directionalcurvatures of the projection manifold. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. It enables the user to interactively highlight those data in the parent visualization plot which are captured by a child model.We also incorporate into our system a hierarchical, locally selective representation of magnification factors and directional curvatures of the projection manifolds. Such information is important for further refinement of the hierarchical visualization plot, as well as for controlling the amount of regularization imposed on the local models. We demonstrate the principle of the approach on a toy data set andapply our system to two more complex 12- and 19-dimensional data sets.

Dynamics and topographic organization of recursive self-organizing maps

Recently there has been an outburst of interest in extending topographic maps of vectorial data to more general data structures, such as sequences or trees. However, there is no general consensus as to how best to process sequences using topographicmaps, and this topic remains an active focus of neurocomputational research. The representational capabilities and internal representations of the models are not well understood. Here, we rigorously analyze a generalization of the self-organizingmap (SOM) for processing sequential data, recursive SOM (RecSOM) (Voegtlin, 2002), as a nonautonomous dynamical system consisting of a set of fixed input maps. We argue that contractive fixed-input maps are likely to produce Markovian organizations of receptive fields on the RecSOM map. We derive bounds on parameter β (weighting the importance of importing past information when processing sequences) under which contractiveness of the fixed-input maps is guaranteed. Some generalizations of SOM contain a dynamic module responsible for processing temporal contexts as an integral part of the model. We show that Markovian topographic maps of sequential data can be produced using a simple fixed (nonadaptable) dynamic module externally feeding a standard topographic model designed to process static vectorial data of fixed dimensionality (e.g., SOM). However, by allowing trainable feedback connections, one can obtain Markovian maps with superior memory depth and topography preservation. We elaborate on the importance of non-Markovian organizations in topographic maps of sequential data. © 2006 Massachusetts Institute of Technology.

Topographic mapping and source localization of the pattern onset visual evoked magnetic response

The topography of the visual evoked magnetic response (VEMR) to a pattern onset stimulus was investigated using 4 check sizes and 3 contrast levels. The pattern onset response consists of three early components within the first 200ms, CIm, CIIm and CIIIm. The CIIm is usually of high amplitude and is very consistent in latency within a subject. Half field (HF) stimuli produce their strongest response over the contralateral hemisphere; the RHF stimulus exhibiting a lower positivity (outgoing field) and an upper negativity (ingoing field), rotated towards the midline. LHF stimulation produced the opposite response, a lower negative and an upper positive. Larger check sizes produce a single area of ingoing and outgoing field while smaller checks produce on area of ingoing and outgoing field over each hemisphere. Latency did not appear to vary with change in contrast but amplitudes increased with increasing contrast. A more detailed topographic study incorporating source localisation procedures suggested a source for CIIm - 4cm below the scalp, close to the midline with current flowing towards the lateral surface. Similar depth and position estimates but with opposite polarity were obtained for the pattern shift P100m previously. Hence, the P100m and the CIIm may originate in similar areas of visual cortex but reveal different aspects of visual processing. © 1992 Human Sciences Press, Inc.

Topographic mapping and source localization of the pattern reversal visual evoked magnetic response

The topography of the visual evoked magnetic response (VEMR) to pattern reversal stimulation was studied in four normal subjects using a single channel BTI magnetometer. VEMRs were recorded from 20 locations over the occipital scalp and the topographic distribution of the most consistent component (P100M) studied. A single dipole in a sphere model was fitted to the data. Topographic maps were similar when recorded two months apart on the same subject to the same stimulus. Half field (HF) stimulation elicited responses from sources on the medial surface of the calcarine fissure mainly in the contralateral hemisphere as predicted by the cruciform model. The full field (FF) responses to large checks were approximately the sum of the HF responses. However, with small checks, FF stimulation appeared to activate a different combination of sources than the two HFs. In addition, HF topography was more consistent between subjects than FF for small check sizes. Topographic studies of the VEMR may help to explain the analogous visual evoked electrical response and will be essential to define optimal recording positions for clinical applications.

Topographic study of the visual evoked magnetic response

Since the visual evoked potential to pattern reversal stimulation produces a paradoxical lateralisation of the major positive P100 component and since this paradoxical lateralisation is dependent on the stimulus parameters including check and field size, we have therefore, carried out a study of the magnetic response (VEMR) to a pattern reversal stimulus in four normal subjects using both full field and half field stimulation and two different check sizes.

Orbit and altimetric corrections for the ERS satellites through analysis of single and dual satellite crossovers

Due to the failure of PRARE the orbital accuracy of ERS-1 is typically 10-15 cm radially as compared to 3-4cm for TOPEX/Poseidon. To gain the most from these simultaneous datasets it is necessary to improve the orbital accuracy of ERS-1 so that it is commensurate with that of TOPEX/Poseidon. For the integration of these two datasets it is also necessary to determine the altimeter and sea state biases for each of the satellites. Several models for the sea state bias of ERS-1 are considered by analysis of the ERS-1 single satellite crossovers. The model adopted consists of the sea state bias as a percentage of the significant wave height, namely 5.95%. The removal of ERS-1 orbit error and recovery of an ERS-1 - TOPEX/Poseidon relative bias are both achieved by analysis of dual crossover residuals. The gravitational field based radial orbit error is modelled by a finite Fourier expansion series with the dominant frequencies determined by analysis of the JGM-2 co-variance matrix. Periodic and secular terms to model the errors due to atmospheric density, solar radiation pressure and initial state vector mis-modelling are also solved for. Validation of the dataset unification consists of comparing the mean sea surface topographies and annual variabilities derived from both the corrected and uncorrected ERS-1 orbits with those derived from TOPEX/Poseidon. The global and regional geographically fixed/variable orbit errors are also analysed pre and post correction, and a significant reduction is noted. Finally the use of dual/single satellite crossovers and repeat pass data, for the calibration of ERS-2 with respect to ERS-1 and TOPEX/Poseidon is shown by calculating the ERS-1/2 sea state and relative biases.

Topographic, haemodynamic and psychophysical investigation of glaucomatous optic neuropathy

Loss of optic nerve head (ONH) axons in primary open angle glaucoma (POAG) has been attributed to both mechanical and vascular factors. Confocal scanning laser ophthalmoscopy (cSLO) provides a promising tool for the topographic follow-up of the ONH in glaucoma, while scanning laser Doppler flowmetry (SLDF) facilitates the rapid non-invasive assessment of retinal capillary blood flow. The purposes of these investigations were to optimise the techniques and explore their potential to classify and monitor disease. Preliminary investigations explored the reproducibility and validity of cSLO and SLDF and showed that: For cSLO: In a model eye, measurements are accurate over a range of axial lengths. For best reproducibility, seven images per visit are required, with a contour line located on Elschnig's scleral ring and transferred automatically between images. For SLDF: Three perfusion images are required for optimum reproducibility. Physiological changes induced by gas perturbation can be measured. Cross-sectional comparison of groups of normal subjects and early POAG patients showed that: cSLO parameters differentiate the early POAG group. Blood volume measured by SLDF showed group differences in superior nasal retina only. Longitudinal investigation of ONH topography, haemodynamic and visual field indices in normal subjects and POAG patients showed that: cSLO detects topographical change over time more frequently in the POAG group. Important parameters include: C:D area ratio, cup and rim area, mean depth in contour, volumes above and below reference and surface. Factor analysis identified "cup" and "rim" factors that can be used to detect change over time in individual patients. Blood flow changes were most apparent in the inferior nasal peripapillary retina of the POAG group. Perimetry is of clinical value for the identification of glaucoma but is less sensitive than cSLO for monitoring glaucomatous change.

Topographic studies of scalp potentials evoked by pattern presentation

The waveform and scalp distribution of the visual evoked potentials elicited by stimuli in the foveal and parafoveal regions have been investigated in a group of normal humans using a 16-channel `brain mapping' system. The waveform and topography of the responses to pattern onset and pattern reversal stimulation were investigated, using 4 x 4o full field and 4 x 2o lateral and altitudinal half-field stimuli. The responses were composed of several successive peaks which are in some respects consistent with those demonstrated by other workers using larger field sizes. The differences in the behaviour of these components with respect to the position of the stimulus in the visual field were suggestive of origins in different areas of the visual cortex and/or different visual mechanism. Of particular interest were the major early positive components `P90' and `P95' of the responses to pattern onset and pattern reversal stimulation respectively. More detailed exploration of the behaviour of these major early positive components was carried out using `M-scaled' stimuli selected to activate one square centimetre patches of striate cortex and associated extrastriate re-projections, positioned at different points in the foveal and parafoveal area of the visual field. The inter- and intra-subject variability in amplitude and localisation of the signals elicited by these targets was considered to be a reflection of the individual variations in relationship of visual field projections with the pattern of gyri and fissures on the proximal surface of the occipital lobe. The behaviour of component P90 of the onset response is consistent with a lateral origin in extrastriate visual cortex; that of P95 of the pattern reversal response is consistent in some respects with a striate cortical origin, but in others with a partial origin in extrastriate cortex.