Linear dimensionality reduction: Survey, insights, and generalizations

© 2015 John P. Cunningham and Zoubin Ghahramani. Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation between data sets, input-output relationships, and margin between data classes. Methods have been developed with a variety of names and motivations in many fields, and perhaps as a result the connections between all these methods have not been highlighted. Here we survey methods from this disparate literature as optimization programs over matrix manifolds. We discuss principal component analysis, factor analysis, linear multidimensional scaling, Fisher's linear discriminant analysis, canonical correlations analysis, maximum autocorrelation factors, slow feature analysis, sufficient dimensionality reduction, undercomplete independent component analysis, linear regression, distance metric learning, and more. This optimization framework gives insight to some rarely discussed shortcomings of well-known methods, such as the suboptimality of certain eigenvector solutions. Modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver, which accepts as input data and an objective to be optimized, and returns, as output, an optimal low-dimensional projection of the data. This simple optimization framework further allows straightforward generalizations and novel variants of classical methods, which we demonstrate here by creating an orthogonal-projection canonical correlations analysis. More broadly, this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox, objective-agnostic numerical technology.

Controlling magnetic ordering in coupled nanomagnet arrays

We have fabricated using high-resolution electron beam lithography circular magnetic particles (nanomagnets) of diameter 60 nm and thickness 7 nm out of the common magnetic alloy supermalloy. The nanomagnets were arranged on rectangular lattices of different periods. A high-sensitivity magneto-optical method was used to measure the magnetic properties of each lattice. We show experimentally how the magnetic properties of a lattice of nanomagnets can be profoundly changed by the magnetostatic interactions between nanomagnets within the lattice. We find that simply reducing the lattice spacing in one direction from 180 nm down to 80 nm (leaving a gap of only 20 nm between edges) causes the lattice to change from a magnetically disordered state to an ordered state. The change in state is accompanied by a peak in the magnetic susceptibility. We show that this is analogous to the paramagnetic-ferromagnetic phase transition which occurs in conventional magnetic materials, although low-dimensionality and kinetic effects must also be considered.

Convergence of the auxiliary particle implementation of the PHD filter

Optimal Bayesian multi-target filtering is in general computationally impractical owing to the high dimensionality of the multi-target state. The Probability Hypothesis Density (PHD) filter propagates the first moment of the multi-target posterior distribution. While this reduces the dimensionality of the problem, the PHD filter still involves intractable integrals in many cases of interest. Several authors have proposed Sequential Monte Carlo (SMC) implementations of the PHD filter. However, these implementations are the equivalent of the Bootstrap Particle Filter, and the latter is well known to be inefficient. Drawing on ideas from the Auxiliary Particle Filter (APF), a SMC implementation of the PHD filter which employs auxiliary variables to enhance its efficiency was proposed by Whiteley et. al. Numerical examples were presented for two scenarios, including a challenging nonlinear observation model, to support the claim. This paper studies the theoretical properties of this auxiliary particle implementation. $\mathbb{L}_p$ error bounds are established from which almost sure convergence follows.

Auxiliary particle implementation of the probability hypothesis density filter

Optimal Bayesian multi-target filtering is, in general, computationally impractical owing to the high dimensionality of the multi-target state. The Probability Hypothesis Density (PHD) filter propagates the first moment of the multi-target posterior distribution. While this reduces the dimensionality of the problem, the PHD filter still involves intractable integrals in many cases of interest. Several authors have proposed Sequential Monte Carlo (SMC) implementations of the PHD filter. However, these implementations are the equivalent of the Bootstrap Particle Filter, and the latter is well known to be inefficient. Drawing on ideas from the Auxiliary Particle Filter (APF), we present a SMC implementation of the PHD filter which employs auxiliary variables to enhance its efficiency. Numerical examples are presented for two scenarios, including a challenging nonlinear observation model.

First-principles prediction of doped graphane as a high-temperature electron-phonon superconductor.

We predict by first-principles calculations that p-doped graphane is an electron-phonon superconductor with a critical temperature above the boiling point of liquid nitrogen. The unique strength of the chemical bonds between carbon atoms and the large density of electronic states at the Fermi energy arising from the reduced dimensionality give rise to a giant Kohn anomaly in the optical phonon dispersions and push the superconducting critical temperature above 90 K. As evidence of graphane was recently reported, and doping of related materials such as graphene, diamond, and carbon nanostructures is well established, superconducting graphane may be feasible.

The modified Kanerva model for automatic speech recognition

A parallel processing network derived from Kanerva's associative memory theory Kanerva 1984 is shown to be able to train rapidly on connected speech data and recognize further speech data with a label error rate of 0·68%. This modified Kanerva model can be trained substantially faster than other networks with comparable pattern discrimination properties. Kanerva presented his theory of a self-propagating search in 1984, and showed theoretically that large-scale versions of his model would have powerful pattern matching properties. This paper describes how the design for the modified Kanerva model is derived from Kanerva's original theory. Several designs are tested to discover which form may be implemented fastest while still maintaining versatile recognition performance. A method is developed to deal with the time varying nature of the speech signal by recognizing static patterns together with a fixed quantity of contextual information. In order to recognize speech features in different contexts it is necessary for a network to be able to model disjoint pattern classes. This type of modelling cannot be performed by a single layer of links. Network research was once held back by the inability of single-layer networks to solve this sort of problem, and the lack of a training algorithm for multi-layer networks. Rumelhart, Hinton & Williams 1985 provided one solution by demonstrating the "back propagation" training algorithm for multi-layer networks. A second alternative is used in the modified Kanerva model. A non-linear fixed transformation maps the pattern space into a space of higher dimensionality in which the speech features are linearly separable. A single-layer network may then be used to perform the recognition. The advantage of this solution over the other using multi-layer networks lies in the greater power and speed of the single-layer network training algorithm. © 1989.

Multi-dimensional conditional moment closure modelling applied to a heavy-duty common-rail diesel engine

A multi-dimensional combustion code implementing the Conditional Moment Closure turbulent combustion model interfaced with a well-established RANS two- phase flow field solver has been employed to study a broad range of operating conditions for a heavy duty direct-injection common-rail Diesel engine. These conditions include different loads (25%, 50%, 75% and full load) and engine speeds (1250 and 1830 RPM) and, with respect to the fuel path, different injection timings and rail pressures. A total of nine cases have been simulated. Excellent agreement with experimental data has been found for the pressure traces and the heat release rates, without adjusting any model constants. The chemical mechanism used contains a detailed NOx sub-mechanism. The predicted emissions agree reasonably well with the experimental data considering the range of operating points and given no adjustments of any rate constants have been employed. In an effort to identify CPU cost reduction potential, various dimensionality reduction strategies have been assessed. Furthermore, the sensitivity of the predictions with respect to resolution in particular relating to the CMC grid has been investigated. Overall, the results suggest that the presented modelling strategy has considerable predictive capability concerning Diesel engine combustion without requiring model constant calibration based on experimental data. This is true particularly for the heat release rates predictions and, to a lesser extent, for NOx emissions where further progress is still necessary. © 2009 SAE International.

VGs for a normal SBLI with a downstream diffuser

New types of vortex generators for boundary layer control were investigated experimentally in a flow field which contains a Mach 1.4 normal Shockwave followed by a subsonic diffuser. A parametric study of device height and distance upstream of the normal shock was undertaken with two novel devices: ramped-vanes and split-ramps. Flowfield diagnostics included high-speed Schlieren, oil flow visualization, and pitot-static pressure measurements. A number of flowfield parameters including flow separation, pressure recovery, centerline incompressible boundary layer shape factor, and shock stability were analyzed and compared to the baseline. All configurations tested yielded an elimination of centerline flow separation with the presence of the vortex generators. However, the devices also tended to increase the three-dimensionality of the flow with increased side-wall interaction. When located 25δo upstream of the normal shock, the largest ramped-vane device (whose height was about 0.75 the incoming uncontrolled boundary layer thickness, δo) yielded the smallest centerline incompressible shape factor and the least streamwise oscillations of the normal shock. However, additional studies are needed to better understand the corner interaction effects, which are substantial. © 2010 by the American Institute of Aeronautics and Astronautics, Inc.

Three-dimensional effects on sliding and waving wings

Like large insects, micro air vehicles operate at low Reynolds numbers O(1; 000 - 10; 000) in a regime characterized by separated flow and strong vortices. The leading-edge vortex has been identified as a significant source of high lift on insect wings, but the conditions required for the formation of a stably attached leading-edge vortex are not yet known. The waving wing is designed to model the translational phase of an insect wing stroke by preserving the unsteady starting and stopping motion as well as three-dimensionality in both wing geometry (via a finite-span wing) and kinematics (via wing rotation). The current study examines the effect of the spanwise velocity gradient on the development of the leading-edge vortex along the wing as well as the effects of increasing threedimensionalityby decreasing wing aspect ratio from four to two. Dye flow visualization and particle image velocimetry reveal that the leading-edge vortices that form on a sliding or waving wing have a very high aspect ratio. The structure of the flow is largely two-dimensional on both sliding and waving wings and there is minimal interaction between the leading-edge vortices and the tip vortex. Significant spanwise flow was observed on the waving wing but not on the sliding wing. Despite the increased three-dimensionality on the aspect ratio 2 waving wing, there is no evidence of an attached leading-edge vortex and the structure of the flow is very similar to that on the higher-aspect-ratio wing and sliding wing. © Copyright 2010.

Microvortex generators applied to flowfield containing a normal shock wave and diffuser

The flow through a terminating shock wave and the subsequent subsonic diffuser typically found in supersonic inlets has been simulated using a small-scale wind tunnel. Experiments have been conducted at an inflow Mach number of 1.4 using a dual-channel working section to produce a steady near-normal shock wave. The setup was designed so that the location of the shock wave could be varied relative to the diffuser. As the near-normal shock wave was moved downstream and into the diffuser, an increasingly distorted, three-dimensional, and separated flow was observed. Compared with the interaction of a normal shock wave in a constant area duct, the addition of the diffuser resulted in more prominent corner interactions. Microvortex generators were added to determine their potential for removing flow separation. Although these devices were found to reduce the extent of separation, they significantly increased three-dimensionality and even led to a large degree of flow asymmetry in some configurations. Copyright © 2011 by Neil Titchener and Holger Babinsky.

Model-based approaches to handling additive noise in reverberant environments

Model-based approaches to handle additive and convolutional noise have been extensively investigated and used. However, the application of these schemes to handling reverberant noise has received less attention. This paper examines the extension of two standard additive/convolutional noise approaches to handling reverberant noise. The first is an extension of vector Taylor series (VTS) compensation, reverberant VTS, where a mismatch function including reverberant noise is used. The second scheme modifies constrained MLLR to allow a wide-span of frames to be taken into account and projected into the required dimensionality. To allow additive noise to be handled, both these schemes are combined with standard VTS. The approaches are evaluated and compared on two tasks, MC-WSJ-AV, and a reverberant simulated version of AURORA-4. © 2011 IEEE.

Optimal design of Functionally Graded Materials using a procedural model and Particle Swarm Optimization

A new method for the optimal design of Functionally Graded Materials (FGM) is proposed in this paper. Instead of using the widely used explicit functional models, a feature tree based procedural model is proposed to represent generic material heterogeneities. A procedural model of this sort allows more than one explicit function to be incorporated to describe versatile material gradations and the material composition at a given location is no longer computed by simple evaluation of an analytic function, but obtained by execution of customizable procedures. This enables generic and diverse types of material variations to be represented, and most importantly, by a reasonably small number of design variables. The descriptive flexibility in the material heterogeneity formulation as well as the low dimensionality of the design vectors help facilitate the optimal design of functionally graded materials. Using the nature-inspired Particle Swarm Optimization (PSO) method, functionally graded materials with generic distributions can be efficiently optimized. We demonstrate, for the first time, that a PSO based optimizer outperforms classical mathematical programming based methods, such as active set and trust region algorithms, in the optimal design of functionally graded materials. The underlying reason for this performance boost is also elucidated with the help of benchmarked examples. © 2011 Elsevier Ltd. All rights reserved.

A Study of Fan-Distortion Interaction Within the NASA Rotor 67 Transonic Stage

The performance of a transonic fan operating within nonuniform inlet flow remains a key concern for the design and operability of a turbofan engine. This paper applies computational methods to improve the understanding of the interaction between a transonic fan and an inlet total pressure distortion. The test case studied is the NASA rotor 67 stage operating with a total pressure distortion covering a 120-deg sector of the inlet flow field. Full-annulus, unsteady, three-dimensional CFD has been used to simulate the test rig installation and the full fan assembly operating with inlet distortion. Novel post-processing methods have been applied to extract the fan performance and features of the interaction between the fan and the nonuniform inflow. The results of the unsteady computations agree well with the measurement data. The local operating condition of the fan at different positions around the annulus has been tracked and analyzed, and this is shown to be highly dependent on the swirl and mass flow redistribution that the rotor induces ahead of it due to the incoming distortion. The upstream flow effects lead to a variation in work input that determines the distortion pattern seen downstream of the fan stage. In addition, the unsteady computations also reveal more complex flow features downstream of the fan stage, which arise due to the three dimensionality of the flow and unsteadiness. © 2012 American Society of Mechanical Engineers.

Multiwall nanotubes, multilayers, and hybrid nanostructures: new frontiers for technology and Raman spectroscopy.

Technological progress is determined, to a great extent, by developments in material science. Breakthroughs can happen when a new type of material or new combinations of known materials with different dimensionality and functionality are created. Multilayered structures, being planar or concentric, are now emerging as major players at the forefront of research. Raman spectroscopy is a well-established characterization technique for carbon nanomaterials and is being developed for layered materials. In this issue of ACS Nano, Hirschmann et al. investigate triple-wall carbon nanotubes via resonant Raman spectroscopy, showing how a wealth of information can be derived about these complex structures. The next challenge is to tackle hybrid heterostructures, consisting of different planar or concentric materials, arranged "on demand" to achieve targeted properties.