Linear dimensionality reduction: Survey, insights, and generalizations

© 2015 John P. Cunningham and Zoubin Ghahramani. Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation between data sets, input-output relationships, and margin between data classes. Methods have been developed with a variety of names and motivations in many fields, and perhaps as a result the connections between all these methods have not been highlighted. Here we survey methods from this disparate literature as optimization programs over matrix manifolds. We discuss principal component analysis, factor analysis, linear multidimensional scaling, Fisher's linear discriminant analysis, canonical correlations analysis, maximum autocorrelation factors, slow feature analysis, sufficient dimensionality reduction, undercomplete independent component analysis, linear regression, distance metric learning, and more. This optimization framework gives insight to some rarely discussed shortcomings of well-known methods, such as the suboptimality of certain eigenvector solutions. Modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver, which accepts as input data and an objective to be optimized, and returns, as output, an optimal low-dimensional projection of the data. This simple optimization framework further allows straightforward generalizations and novel variants of classical methods, which we demonstrate here by creating an orthogonal-projection canonical correlations analysis. More broadly, this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox, objective-agnostic numerical technology.

Comparison of relative flexoelectric coefficients measured by different techniques

While it is well known that it is possible to determine the effective flexoelectric coefficient of nematic liquid crystals using hybrid cells [1], this technique can be difficult due to the necessity of using a D.C. field. We have used a second method[2], requiring an A.C. field, to determine this parameter and here we compare the two techniques. The A.C. method employs the linear flexoelectrically induced linear electro-optic switching mechanism observed in chiral nematics. In order to use this second technique a chiral nematic phase is induced in an achiral nematic by the addition of a small amount of chiral additive (∼3% concentration w/w) to give helix pitch lengths of typically 0.5-1.0 μm. We note that the two methods can be used interchangeably, since they produce similar results, and we conclude with a discussion of their relative merits.

Sequential Monte Carlo computation of the score and observed information matrix in state-space models with application to parameter estimation

Sequential Monte Carlo (SMC) methods are popular computational tools for Bayesian inference in non-linear non-Gaussian state-space models. For this class of models, we propose SMC algorithms to compute the score vector and observed information matrix recursively in time. We propose two different SMC implementations, one with computational complexity $\mathcal{O}(N)$ and the other with complexity $\mathcal{O}(N^{2})$ where $N$ is the number of importance sampling draws. Although cheaper, the performance of the $\mathcal{O}(N)$ method degrades quickly in time as it inherently relies on the SMC approximation of a sequence of probability distributions whose dimension is increasing linearly with time. In particular, even under strong \textit{mixing} assumptions, the variance of the estimates computed with the $\mathcal{O}(N)$ method increases at least quadratically in time. The $\mathcal{O}(N^{2})$ is a non-standard SMC implementation that does not suffer from this rapid degrade. We then show how both methods can be used to perform batch and recursive parameter estimation.

CLADP - THE CAMBRIDGE LINEAR ANALYSIS AND DESIGN PROGRAMS USER REFERENCE MANUAL (4TH ISSUE: APRIL 1983).

CLADP is an engineering software program developed at Cambridge University for the interactive computer aided design of feedback control systems. CLADP contains a wide range of tools for the analysis of complex systems, and the assessment of their performance when feedback control is applied, thus enabling control systems to be designed to meet difficult performance objectives. The range of tools within CLADP include the latest techniques in the field whose central theme is the extension of classical frequency domain concepts (well known and well proven for single loop systems) to multivariable or multiloop systems, and by making extensive use of graphical presentation information is provided in a readily understood form.

Comparison of different scheduling techniques for H-infinity controllers

Several approaches to designing schedule H-infinity control systems are compared. These include a controller switching approach and also parameter scheduling of an observer representation of the controller. They are illustrated by application to a Generic VSTOI. Aircraft Model (GVAM) supplied by The Royal Aerospace Establishment (RAE) at Bedford. The switched design has been tested on the simulator at RAE Bedford. The linear H-infinity designs make use of a loop-shaping followed by robust stabilisation to additive perturbations of a normalised coprime factorisation of the shaped plans. The different scheduling approaches are compared with respect to achieved robust stability levels. performance and complexity of implementation.

The use of C-V techniques to investigate instability mechanisms in M-I-S structures

In the field of flat panel displays, the current leading technology is the Active Matrix liquid Crystal Display; this uses a-Si:H based thin film transistors (TFTs) as the switching element in each pixel. However, under gate bias a-Si:H TFTs suffer from instability, as is evidenced by a shift in the gate threshold voltage. The shift in the gate threshold voltage is generally measured from the gate transfer characteristics, after subjecting the TFT to prolonged gate bias. However, a major drawback of this measurement method is that it cannot distinguish whether the shift is caused by the change in the midgap states in the a-Si:H channel or by charge trapping in the gate insulator. In view of this, we have developed a capacitance-voltage (C-V) method to measure the shift in threshold voltage. We employ Metal-Insulator-Semiconductor (MIS) structures to investigate the threshold voltage shift as they are simpler to fabricate than TFTs. We have investigated a large of number Metal/a-Si:H/Si3N4/Si+n structures using our C-V technique. From, the C-V data for the MIS structures, we have found that the relationship between the thermal energy and threshold voltage shift is similar to that reported by Wehrspohn et. al in a-Si:H TFTs (J Appl. Phys, 144, 87, 2000). The a-Si:H and Si3N4 layers were grown using the radio-frequency plasma-enhanced chemical vapour deposition technique.

Incremental Linear Discriminant Analysis Using Sufficient Spanning Sets and Its Applications

This paper presents an incremental learning solution for Linear Discriminant Analysis (LDA) and its applications to object recognition problems. We apply the sufficient spanning set approximation in three steps i.e. update for the total scatter matrix, between-class scatter matrix and the projected data matrix, which leads an online solution which closely agrees with the batch solution in accuracy while significantly reducing the computational complexity. The algorithm yields an efficient solution to incremental LDA even when the number of classes as well as the set size is large. The incremental LDA method has been also shown useful for semi-supervised online learning. Label propagation is done by integrating the incremental LDA into an EM framework. The method has been demonstrated in the task of merging large datasets which were collected during MPEG standardization for face image retrieval, face authentication using the BANCA dataset, and object categorisation using the Caltech101 dataset. © 2010 Springer Science+Business Media, LLC.

Shape Recognition of Linear Construction Entities from Construction Site Images

The capability to automatically identify shapes, objects and materials from the image content through direct and indirect methodologies has enabled the development of several civil engineering related applications that assist in the design, construction and maintenance of construction projects. This capability is a product of the technological breakthroughs in the area of Image Processing that has allowed for the development of a large number of digital imaging applications in all industries. In this paper, an automated and content based shape recognition model is presented. This model was devised to enhance the recognition capabilities of our existing material based image retrieval model. The shape recognition model is based on clustering techniques, and specifically those related with material and object segmentation. The model detects the borders of each previously detected material depicted in the image, examines its linearity (length/width ratio) and detects its orientation (horizontal/vertical). The results emonstrate the suitability of this model for construction site image retrieval purposes and reveal the capability of existing clustering technologies to accurately identify the shape of a wealth of materials from construction site images.