A novel neural network technique for exploratory data analysis

This paper surveys the context of feature extraction by neural network approaches, and compares and contrasts their behaviour as prospective data visualisation tools in a real world problem. We also introduce and discuss a hybrid approach which allows us to control the degree of discriminatory and topographic information in the extracted feature space.

Topographic mappings and feed-forward neural networks

This thesis is a study of the generation of topographic mappings - dimension reducing transformations of data that preserve some element of geometric structure - with feed-forward neural networks. As an alternative to established methods, a transformational variant of Sammon's method is proposed, where the projection is effected by a radial basis function neural network. This approach is related to the statistical field of multidimensional scaling, and from that the concept of a 'subjective metric' is defined, which permits the exploitation of additional prior knowledge concerning the data in the mapping process. This then enables the generation of more appropriate feature spaces for the purposes of enhanced visualisation or subsequent classification. A comparison with established methods for feature extraction is given for data taken from the 1992 Research Assessment Exercise for higher educational institutions in the United Kingdom. This is a difficult high-dimensional dataset, and illustrates well the benefit of the new topographic technique. A generalisation of the proposed model is considered for implementation of the classical multidimensional scaling (¸mds}) routine. This is related to Oja's principal subspace neural network, whose learning rule is shown to descend the error surface of the proposed ¸mds model. Some of the technical issues concerning the design and training of topographic neural networks are investigated. It is shown that neural network models can be less sensitive to entrapment in the sub-optimal global minima that badly affect the standard Sammon algorithm, and tend to exhibit good generalisation as a result of implicit weight decay in the training process. It is further argued that for ideal structure retention, the network transformation should be perfectly smooth for all inter-data directions in input space. Finally, there is a critique of optimisation techniques for topographic mappings, and a new training algorithm is proposed. A convergence proof is given, and the method is shown to produce lower-error mappings more rapidly than previous algorithms.

A hierarchical latent variable model for data visualization

Visualization has proven to be a powerful and widely-applicable tool the analysis and interpretation of data. Most visualization algorithms aim to find a projection from the data space down to a two-dimensional visualization space. However, for complex data sets living in a high-dimensional space it is unlikely that a single two-dimensional projection can reveal all of the interesting structure. We therefore introduce a hierarchical visualization algorithm which allows the complete data set to be visualized at the top level, with clusters and sub-clusters of data points visualized at deeper levels. The algorithm is based on a hierarchical mixture of latent variable models, whose parameters are estimated using the expectation-maximization algorithm. We demonstrate the principle of the approach first on a toy data set, and then apply the algorithm to the visualization of a synthetic data set in 12 dimensions obtained from a simulation of multi-phase flows in oil pipelines and to data in 36 dimensions derived from satellite images.

Mixtures of probabilistic principal component analysers

Principal component analysis (PCA) is one of the most popular techniques for processing, compressing and visualising data, although its effectiveness is limited by its global linearity. While nonlinear variants of PCA have been proposed, an alternative paradigm is to capture data complexity by a combination of local linear PCA projections. However, conventional PCA does not correspond to a probability density, and so there is no unique way to combine PCA models. Previous attempts to formulate mixture models for PCA have therefore to some extent been ad hoc. In this paper, PCA is formulated within a maximum-likelihood framework, based on a specific form of Gaussian latent variable model. This leads to a well-defined mixture model for probabilistic principal component analysers, whose parameters can be determined using an EM algorithm. We discuss the advantages of this model in the context of clustering, density modelling and local dimensionality reduction, and we demonstrate its application to image compression and handwritten digit recognition.

Probabilistic principal component analysis

Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based upon a probability model. In this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum-likelihood estimation of parameters in a latent variable model closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace iteratively, and discuss the advantages conveyed by the definition of a probability density function for PCA.

Probabilistic principal component analysis

Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based upon a probability model. In this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum-likelihood estimation of parameters in a latent variable model closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace iteratively, and discuss the advantages conveyed by the definition of a probability density function for PCA.

Governance in contemporary Germany:the semisovereign State revisited

Throughout the 1970s and 1980s, West Germany was considered to be one of the world’s most successful economic and political systems. In his seminal 1987 analysis of West Germany’s ‘semisovereign’ system of governance, Peter Katzenstein attributed this success to a combination of a fragmented polity, consensus politics and incremental policy changes. However, unification in 1990 has both changed Germany’s institutional configuration and created economic and social challenges on a huge scale. This volume therefore asks whether semisovereignty still exists in contemporary Germany and, crucially, whether it remains an asset in terms of addressing these challenges. By shadowing and building on the original study, an eminent team of British, German and American scholars analyses institutional changes and the resulting policy developments in key sectors, with Peter Katzenstein himself providing the conclusion. Together, the chapters provide a landmark assessment of the outcomes produced by one of the world’s most important countries. Contents: 1. Introduction: semisovereignty challenged Simon Green and William E. Paterson; 2. Institutional transfer: can semisovereignty be transferred? The political economy of Eastern Germany Wade Jacoby; 3. Political parties Thomas Saalfeld; 4. Federalism: the new territorialism Charlie Jeffery; 5. Shock-absorbers under stress. Parapublic institutions and the double challenges of German unification and European integration Andreas Busch; 6. Economic policy management: catastrophic equilibrium, tipping points and crisis interventions Kenneth Dyson; 7. Industrial relations: from state weakness as strength to state weakness as weakness. Welfare corporatism and the private use of the public interest Wolfgang Streeck; 8. Social policy: crisis and transformation Roland Czada; 9. Immigration and integration policy: between incrementalism and non-decisions Simon Green; 10. Environmental policy: the law of diminishing returns? Charles Lees; 11. Administrative reform Kluas H. Goetz; 12. European policy-making: between associated sovereignty and semisovereignty William E. Paterson; 13. Conclusion: semisovereignty in United Germany Peter J. Katzenstein.

Fundamental mechanisms affecting service performance of electroplated plastics

Ten grades of ABS and four grades of polypropylene have been plated with various copper + nickel + chromium coatings and subjected to a variety of tests. In corrosion studies the pre-electroplating sequence and plastics type have been shown to influence performance. One ABS pre-electroplating sequence was consistently associated with better corrosion performance; two factors were responsible for this, namely the more severe nature of the etch and the relatively more noble electroless nickel. Statistical analysis has indicated that order of severity of the corrosion tests was static-mobile-CASS, the latter being the least severe. In mechanical tests two properties of ABS and polypropJylene, ductility and impact strength, have been shown to be adversely affected when electrodeposited layers were applied. The cause of this is due to a complex of factors, the most important of which is the notch sensitivity of the plastics. Peel adhesion has been studied on flat panels and also on ones which had a ridge and a valley moulded into one face. High adhesion peaks occurred on the flat face at regions associated with the ridge and valley. The local moulding conditions induced by the features were responsible for this phenonemon. In the main programme the thermal cycling test was shown to be more likely than the peel adhesion test to give an indication of the service performance of electroplated plastics.

Feed-forward neural networks and topographic mappings for exploratory data analysis

A recent novel approach to the visualisation and analysis of datasets, and one which is particularly applicable to those of a high dimension, is discussed in the context of real applications. A feed-forward neural network is utilised to effect a topographic, structure-preserving, dimension-reducing transformation of the data, with an additional facility to incorporate different degrees of associated subjective information. The properties of this transformation are illustrated on synthetic and real datasets, including the 1992 UK Research Assessment Exercise for funding in higher education. The method is compared and contrasted to established techniques for feature extraction, and related to topographic mappings, the Sammon projection and the statistical field of multidimensional scaling.