Hierarchical GTM: Constructing localized non-linear projection manifolds in a principled way

It has been argued that a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets, and therefore a hierarchical visualization system is desirable. In this paper we extend an existing locally linear hierarchical visualization system PhiVis ¸iteBishop98a in several directions: bf(1) We allow for em non-linear projection manifolds. The basic building block is the Generative Topographic Mapping. bf(2) We introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree. General training equations are derived, regardless of the position of the model in the tree. bf(3) Using tools from differential geometry we derive expressions for local directional curvatures of the projection manifold. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. It enables the user to interactively highlight those data in the parent visualization plot which are captured by a child model. We also incorporate into our system a hierarchical, locally selective representation of magnification factors and directional curvatures of the projection manifolds. Such information is important for further refinement of the hierarchical visualization plot, as well as for controlling the amount of regularization imposed on the local models. We demonstrate the principle of the approach on a toy data set and apply our system to two more complex 12- and 19-dimensional data sets.

Using directional curvatures to visualize folding patterns of the GTM projection manifolds

In data visualization, characterizing local geometric properties of non-linear projection manifolds provides the user with valuable additional information that can influence further steps in the data analysis. We take advantage of the smooth character of GTM projection manifold and analytically calculate its local directional curvatures. Curvature plots are useful for detecting regions where geometry is distorted, for changing the amount of regularization in non-linear projection manifolds, and for choosing regions of interest when constructing detailed lower-level visualization plots.

Hierarchical GTM: constructing localized non-linear projection manifolds in a principled way

It has been argued that a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets, and therefore a hierarchical visualization system is desirable. In this paper we extend an existing locally linear hierarchical visualization system PhiVis ¸iteBishop98a in several directions: bf(1) We allow for em non-linear projection manifolds. The basic building block is the Generative Topographic Mapping (GTM). bf(2) We introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree. General training equations are derived, regardless of the position of the model in the tree. bf(3) Using tools from differential geometry we derive expressions for local directional curvatures of the projection manifold. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. It enables the user to interactively highlight those data in the ancestor visualization plots which are captured by a child model. We also incorporate into our system a hierarchical, locally selective representation of magnification factors and directional curvatures of the projection manifolds. Such information is important for further refinement of the hierarchical visualization plot, as well as for controlling the amount of regularization imposed on the local models. We demonstrate the principle of the approach on a toy data set and apply our system to two more complex 12- and 18-dimensional data sets.

Constructing localized non-linear projection manifolds in a principled way:hierarchical generative topographic mapping

It has been argued that a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets, and therefore a hierarchical visualization system is desirable. In this paper we extend an existing locally linear hierarchical visualization system PhiVis (Bishop98a) in several directions: 1. We allow for em non-linear projection manifolds. The basic building block is the Generative Topographic Mapping. 2. We introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree. General training equations are derived, regardless of the position of the model in the tree. 3. Using tools from differential geometry we derive expressions for local directionalcurvatures of the projection manifold. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. It enables the user to interactively highlight those data in the parent visualization plot which are captured by a child model.We also incorporate into our system a hierarchical, locally selective representation of magnification factors and directional curvatures of the projection manifolds. Such information is important for further refinement of the hierarchical visualization plot, as well as for controlling the amount of regularization imposed on the local models. We demonstrate the principle of the approach on a toy data set andapply our system to two more complex 12- and 19-dimensional data sets.

Information geometric measurements of generalisation

Neural networks can be regarded as statistical models, and can be analysed in a Bayesian framework. Generalisation is measured by the performance on independent test data drawn from the same distribution as the training data. Such performance can be quantified by the posterior average of the information divergence between the true and the model distributions. Averaging over the Bayesian posterior guarantees internal coherence; Using information divergence guarantees invariance with respect to representation. The theory generalises the least mean squares theory for linear Gaussian models to general problems of statistical estimation. The main results are: (1)~the ideal optimal estimate is always given by average over the posterior; (2)~the optimal estimate within a computational model is given by the projection of the ideal estimate to the model. This incidentally shows some currently popular methods dealing with hyperpriors are in general unnecessary and misleading. The extension of information divergence to positive normalisable measures reveals a remarkable relation between the dlt dual affine geometry of statistical manifolds and the geometry of the dual pair of Banach spaces Ld and Ldd. It therefore offers conceptual simplification to information geometry. The general conclusion on the issue of evaluating neural network learning rules and other statistical inference methods is that such evaluations are only meaningful under three assumptions: The prior P(p), describing the environment of all the problems; the divergence Dd, specifying the requirement of the task; and the model Q, specifying available computing resources.

Developments of the generative topographic mapping

The generative topographic mapping (GTM) model was introduced by Bishop et al. (1998, Neural Comput. 10(1), 215-234) as a probabilistic re- formulation of the self-organizing map (SOM). It offers a number of advantages compared with the standard SOM, and has already been used in a variety of applications. In this paper we report on several extensions of the GTM, including an incremental version of the EM algorithm for estimating the model parameters, the use of local subspace models, extensions to mixed discrete and continuous data, semi-linear models which permit the use of high-dimensional manifolds whilst avoiding computational intractability, Bayesian inference applied to hyper-parameters, and an alternative framework for the GTM based on Gaussian processes. All of these developments directly exploit the probabilistic structure of the GTM, thereby allowing the underlying modelling assumptions to be made explicit. They also highlight the advantages of adopting a consistent probabilistic framework for the formulation of pattern recognition algorithms.

A principled approach to interactive hierarchical non-linear visualization of high-dimensional data

Hierarchical visualization systems are desirable because a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex high-dimensional data sets. We extend an existing locally linear hierarchical visualization system PhiVis [1] in several directions: bf(1) we allow for em non-linear projection manifolds (the basic building block is the Generative Topographic Mapping -- GTM), bf(2) we introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree, bf(3) we describe folding patterns of low-dimensional projection manifold in high-dimensional data space by computing and visualizing the manifold's local directional curvatures. Quantities such as magnification factors [3] and directional curvatures are helpful for understanding the layout of the nonlinear projection manifold in the data space and for further refinement of the hierarchical visualization plot. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. We demonstrate the visualization system principle of the approach on a complex 12-dimensional data set and mention possible applications in the pharmaceutical industry.

Approximating differentiable relationships between delay embedded dynamical systems with radical basis functions

This thesis is about the study of relationships between experimental dynamical systems. The basic approach is to fit radial basis function maps between time delay embeddings of manifolds. We have shown that under certain conditions these maps are generically diffeomorphisms, and can be analysed to determine whether or not the manifolds in question are diffeomorphically related to each other. If not, a study of the distribution of errors may provide information about the lack of equivalence between the two. The method has applications wherever two or more sensors are used to measure a single system, or where a single sensor can respond on more than one time scale: their respective time series can be tested to determine whether or not they are coupled, and to what degree. One application which we have explored is the determination of a minimum embedding dimension for dynamical system reconstruction. In this special case the diffeomorphism in question is closely related to the predictor for the time series itself. Linear transformations of delay embedded manifolds can also be shown to have nonlinear inverses under the right conditions, and we have used radial basis functions to approximate these inverse maps in a variety of contexts. This method is particularly useful when the linear transformation corresponds to the delay embedding of a finite impulse response filtered time series. One application of fitting an inverse to this linear map is the detection of periodic orbits in chaotic attractors, using suitably tuned filters. This method has also been used to separate signals with known bandwidths from deterministic noise, by tuning a filter to stop the signal and then recovering the chaos with the nonlinear inverse. The method may have applications to the cancellation of noise generated by mechanical or electrical systems. In the course of this research a sophisticated piece of software has been developed. The program allows the construction of a hierarchy of delay embeddings from scalar and multi-valued time series. The embedded objects can be analysed graphically, and radial basis function maps can be fitted between them asynchronously, in parallel, on a multi-processor machine. In addition to a graphical user interface, the program can be driven by a batch mode command language, incorporating the concept of parallel and sequential instruction groups and enabling complex sequences of experiments to be performed in parallel in a resource-efficient manner.

Deformation of sand in plane strain and oxisymmetric compression

Small scale laboratory experiments, in which the specimen is considered to represent an element of soil in the soil mass, are essential to the evolution of fundamental theories of mechanical behaviour. In this thesis, plane strain and axisymmetric compression tests, performed on a fine sand, are reported and the results are compared with various theoretical predictions. A new apparatus is described in which cuboidal samples can be tested in either axisymmetric compression or plane strain. The plane strain condition is simulated either by rigid side platens, in the conventional manner, or by flexible side platens which also measure the intermediate principal stress. Close control of the initial porosity of the specimens is achieved by a vibratory method of sample preparation. The strength of sand is higher in plane strain than in axisymmetric compression, and the strains required to mobilize peak strength are much smaller. The difference between plane strain and axisymmetric compression behaviour is attributed to the restrictions on particle movement enforced by the plane strain condition; this results in an increase in the frictional component of shear strength. The stress conditions at failure in plane strain, including the intermediate principal stress, are accurately predicted by a theory based on the stress- dilatancy interpretation of Mohr's circles. Detailed observations of rupture modes are presented and measured rupture plane inclinations are predicted by the stress-dilatancy theory. Although good correlation with the stress-dilatancy theory is obtained during virgin loading, in both axisymmetric compression and plane strain, the stress-dilatancy rule is only obeyed during reloading if the specimen has been unloaded to approximate ambient stress conditions. The shape of the stress-strain curves during pre-peak deformation, in both plane strain and axisymmetric compression, is accurately described bv a combined parabolic-hyperbolic specification.

A study of the aerodynamic characteristics of captor hoods in local exhaust ventilation systems

The research objectives were:- 1.To review the literature to establish the factors which have traditionally been regarded as most crucial to the design of effectlve exhaust ventilation systems. 2. To design, construct, install and calibrate a wind tunnel. 3. To develop procedures for air velocity measurement followed by a comprehensive programme of aerodvnamic data collection and data analysis for a variety of conditions. The major research findings were:- a) The literature in the subject is inadequate. There is a particular need for a much greater understanding of the aerodynamics of the suction flow field. b) The discrepancies between the experimentally observed centre-line velocities and those predicted by conventional formulae are unacceptably large. c) There was little agreement between theoretically calculated and observed velocities in the suction zone of captor hoods. d) Improved empirical formulae for the prediction of centre-line velocity applicable to the classical geometrically shaped suction openings and the flanged condition could be (and were) derived. Further analysis of data revealed that: - i) Point velocity is directly proportional to the suction. flow rate and the ratio of the point velocity to the average face velocity is constant. ii) Both shape, and size of the suction opening are significant factors as the coordinates of their points govern the extent of the effect of the suction flow field. iii) The hypothetical ellipsoidal potential function and hyperbolic streamlines were found experimentally to be correct. iv) The effect of guide plates depends on the size, shape and the angle of fitting. The effect was to very approximately double the suction velocity but the exact effect is difficult to predict. v) The axially symmetric openings produce practically symmetric flow fields. Similarity of connection pieces between the suction opening and the main duct in each case is essential in order to induce a similar suction flow field. Additionally a pilot study was made in which an artificial extraneous air flow was created, measured and its interaction with the suction flow field measured and represented graphically.

Transition of planar Couette flow at infinite Reynolds numbers

An outline of the state space of planar Couette flow at high Reynolds numbers (Re<105) is investigated via a variety of efficient numerical techniques. It is verified from nonlinear analysis that the lower branch of the hairpin vortex state (HVS) asymptotically approaches the primary (laminar) state with increasing Re. It is also predicted that the lower branch of the HVS at high Re belongs to the stability boundary that initiates a transition to turbulence, and that one of the unstable manifolds of the lower branch of HVS lies on the boundary. These facts suggest HVS may provide a criterion to estimate a minimum perturbation arising transition to turbulent states at the infinite Re limit. © 2013 American Physical Society.

An overview of Total Factor Productivity estimations adjusted for pollutant outputs:an application to sugarcane farming

The conventional Total Factor Productivity (TFP) measurement does not incorporate the effects of undesirable outputs, which are harmful to the environment. Using sugarcane farming in Kenya, this paper illustrates the differences between the conventional Malmquist index measures where the environment variable is not adjusted and environment-adjusted measures using both hyperbolic and directional distance functions. The mean TFP change estimates for the conventional Malmquist index, adjusted hyperbolic index and Luenberger indicator were 3.13%, 0.11% and 2.21%, respectively. The conventional non-adjusted measure lies between the two adjusted measures of hyperbolic index and Luenberger indicator. © 2012 Inderscience Enterprises Ltd.

Fundamental Pd0/PdII redox steps in cross-coupling reactions:homogeneous, hybrid homogeneous-heterogeneous to heterogeneous mechanistic pathways for C-C couplings

C–C bond-forming, cross-coupling reactions of organohalides with nucleophilic compounds, catalysed by palladium, are amongst the most important chemical reactions available to the synthetic chemist. The intimate mechanisms of these reactions, involving Pd0/PdII redox steps, have been of great historical interest and continue to be so. The myriad of possible mechanisms is reviewed in this chapter. The interplay of mononuclear Pd species with higher order Pd species, e.g. nanoclusters/nanoparticles are considered as being equally important in cross-coupling reaction mechanisms. A focus is placed on trichotomic behaviour of cross-coupling catalytic manifolds, from homogeneous to hybrid homogeneous–heterogeneous to truly heterogeneous behaviour. For the latter, surface chemistry and metal atom leaching (and various experimental techniques) are broadly discussed. It is now clear that mechanism for general cross‐coupling reactions, that is as presented to undergraduate students studying Chemistry degrees across the world, is undoubtedly more complex than first thought. New opportunities for catalyst design have therefore emerged in the area of Pd nanoparticles and nanocatalysis, with some wonderful applications especially in chemical biology, providing a snapshot of what the future might hold.

Time-spatial drift of decelerating electromagnetic pulses

A time dependent electromagnetic pulse generated by a current running laterally to the direction of the pulse propagation is considered in paraxial approximation. It is shown that the pulse envelope moves in the time-spatial coordinates on the surface of a parabolic cylinder for the Airy pulse and a hyperbolic cylinder for the Gaussian. These pulses propagate in time with deceleration along the dominant propagation direction and drift uniformly in the lateral direction. The Airy pulse stops at infinity while the asymptotic velocity of the Gaussian is nonzero. © 2013 Optical Society of America.

Generation of drifting optical pulses

The radiation in the form of Airy and Gaussian optical pulses is investigated. It is shown that the pulse envelope moves decelerating in the time-spatial coordinates on the surface of a parabolic cylinder for the Airy pulse and a hyperbolic cylinder for the Gaussian. © 2013 IEEE.