Probabilistic principal component analysis

Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based upon a probability model. In this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum-likelihood estimation of parameters in a latent variable model closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace iteratively, and discuss the advantages conveyed by the definition of a probability density function for PCA.

Probabilistic principal component analysis

Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based upon a probability model. In this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum-likelihood estimation of parameters in a latent variable model closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace iteratively, and discuss the advantages conveyed by the definition of a probability density function for PCA.

Evaluation of catalytic pyrolysis of cassava rhizome by principal component analysis

Rhizome of cassava plants (Manihot esculenta Crantz) was catalytically pyrolysed at 500 °C using analytical pyrolysis–gas chromatography/mass spectrometry (Py–GC/MS) method in order to investigate the relative effect of various catalysts on pyrolysis products. Selected catalysts expected to affect bio-oil properties were used in this study. These include zeolites and related materials (ZSM-5, Al-MCM-41 and Al-MSU-F type), metal oxides (zinc oxide, zirconium (IV) oxide, cerium (IV) oxide and copper chromite) catalysts, proprietary commercial catalysts (Criterion-534 and alumina-stabilised ceria-MI-575) and natural catalysts (slate, char and ashes derived from char and biomass). The pyrolysis product distributions were monitored using models in principal components analysis (PCA) technique. The results showed that the zeolites, proprietary commercial catalysts, copper chromite and biomass-derived ash were selective to the reduction of most oxygenated lignin derivatives. The use of ZSM-5, Criterion-534 and Al-MSU-F catalysts enhanced the formation of aromatic hydrocarbons and phenols. No single catalyst was found to selectively reduce all carbonyl products. Instead, most of the carbonyl compounds containing hydroxyl group were reduced by zeolite and related materials, proprietary catalysts and copper chromite. The PCA model for carboxylic acids showed that zeolite ZSM-5 and Al-MSU-F tend to produce significant amounts of acetic and formic acids.

Mixtures of probabilistic principal component analysers

Principal component analysis (PCA) is one of the most popular techniques for processing, compressing and visualising data, although its effectiveness is limited by its global linearity. While nonlinear variants of PCA have been proposed, an alternative paradigm is to capture data complexity by a combination of local linear PCA projections. However, conventional PCA does not correspond to a probability density, and so there is no unique way to combine PCA models. Previous attempts to formulate mixture models for PCA have therefore to some extent been ad hoc. In this paper, PCA is formulated within a maximum-likelihood framework, based on a specific form of Gaussian latent variable model. This leads to a well-defined mixture model for probabilistic principal component analysers, whose parameters can be determined using an EM algorithm. We discuss the advantages of this model in the context of clustering, density modelling and local dimensionality reduction, and we demonstrate its application to image compression and handwritten digit recognition.

Independent component analysis for domain independent watermarking

A new principled domain independent watermarking framework is presented. The new approach is based on embedding the message in statistically independent sources of the covertext to mimimise covertext distortion, maximise the information embedding rate and improve the method's robustness against various attacks. Experiments comparing the performance of the new approach, on several standard attacks show the current proposed approach to be competitive with other state of the art domain-specific methods.

Watermarking of audio signals using independent component analysis

A novel approach to watermarking of audio signals using Independent Component Analysis (ICA) is proposed. It exploits the statistical independence of components obtained by practical ICA algorithms to provide a robust watermarking scheme with high information rate and low distortion. Numerical simulations have been performed on audio signals, showing good robustness of the watermark against common attacks with unnoticeable distortion, even for high information rates. An important aspect of the method is its domain independence: it can be used to hide information in other types of data, with minor technical adaptations.

Description and critical appraisal of principal components analysis (PCA) methodology applied to pulsed-field gel electrophoresis profiles of methicillin-resistant Staphylococcus aureus isolates

Principal components analysis (PCA) has been described for over 50 years; however, it is rarely applied to the analysis of epidemiological data. In this study PCA was critically appraised in its ability to reveal relationships between pulsed-field gel electrophoresis (PFGE) profiles of methicillin- resistant Staphylococcus aureus (MRSA) in comparison to the more commonly employed cluster analysis and representation by dendrograms. The PFGE type following SmaI chromosomal digest was determined for 44 multidrug-resistant hospital-acquired methicillin-resistant S. aureus (MR-HA-MRSA) isolates, two multidrug-resistant community-acquired MRSA (MR-CA-MRSA), 50 hospital-acquired MRSA (HA-MRSA) isolates (from the University Hospital Birmingham, NHS Trust, UK) and 34 community-acquired MRSA (CA-MRSA) isolates (from general practitioners in Birmingham, UK). Strain relatedness was determined using Dice band-matching with UPGMA clustering and PCA. The results indicated that PCA revealed relationships between MRSA strains, which were more strongly correlated with known epidemiology, most likely because, unlike cluster analysis, PCA does not have the constraint of generating a hierarchic classification. In addition, PCA provides the opportunity for further analysis to identify key polymorphic bands within complex genotypic profiles, which is not always possible with dendrograms. Here we provide a detailed description of a PCA method for the analysis of PFGE profiles to complement further the epidemiological study of infectious disease. © 2005 Elsevier B.V. All rights reserved.

Neuropathological heterogeneity in Alzheimer's disease:A study of 80 cases using principal components analysis

Three hypotheses have been proposed to explain neuropathological heterogeneity in Alzheimer's disease (AD): the presence of distinct subtypes ('subtype hypothesis'), variation in the stage of the disease ('phase hypothesis') and variation in the origin and progression of the disease ('compensation hypothesis'). To test these hypotheses, variation in the distribution and severity of senile plaques (SP) and neurofibrillary tangles (NFT) was studied in 80 cases of AD using principal components analysis (PCA). Principal components analysis using the cases as variables (Q-type analysis) suggested that individual differences between patients were continuously distributed rather than the cases being clustered into distinct subtypes. In addition, PCA using the abundances of SP and NFT as variables (R-type analysis) suggested that variations in the presence and abundance of lesions in the frontal and occipital lobes, the cingulate gyrus and the posterior parahippocampal gyrus were the most important sources of heterogeneity consistent with the presence of different stages of the disease. In addition, in a subgroup of patients, individual differences were related to apolipoprotein E (ApoE) genotype, the presence and severity of SP in the frontal and occipital cortex being significantly increased in patients expressing apolipoprotein (Apo)E allele ε4. It was concluded that some of the neuropathological heterogeneity in our AD cases may be consistent with the 'phase hypothesis'. A major factor determining this variation in late-onset cases was ApoE genotype with accelerated rates of spread of the pathology in patients expressing allele ε4.

Neuropathological changes in ten cases of neuronal intermediate filament inclusion disease (NIFID):A study using α-internexin immunohistochemistry and principal components analysis (PCA)

Ten cases of neuronal intermediate filament inclusion disease (NIFID) were studied quantitatively. The α-internexin positive neurofilament inclusions (NI) were most abundant in the motor cortex and CA sectors of the hippocampus. The densities of the NI and the swollen achromatic neurons (SN) were similar in laminae II/III and V/VI but glial cell density was greater in V/VI. The density of the NI was positively correlated with the SN and the glial cells. Principal components analysis (PCA) suggested that PC1 was associated with variation in neuronal loss in the frontal/temporal lobes and PC2 with neuronal loss in the frontal lobe and NI density in the parahippocampal gyrus. The data suggest: 1) frontal and temporal lobe degeneration in NIFID is associated with the widespread formation of NI and SN, 2) NI and SN affect cortical laminae II/III and V/VI, 3) the NI and SN affect closely related neuronal populations, and 4) variations in neuronal loss and in the density of NI were the most important sources of pathological heterogeneity. © Springer-Verlag 2005.

Statnote 27: Principal components analysis (PCA)

In Statnotes 24 and 25, multiple linear regression, a statistical method that examines the relationship between a single dependent variable (Y) and two or more independent variables (X), was described. The principle objective of such an analysis was to determine which of the X variables had a significant influence on Y and to construct an equation that predicts Y from the X variables. ‘Principal components analysis’ (PCA) and ‘factor analysis’ (FA) are also methods of examining the relationships between different variables but they differ from multiple regression in that no distinction is made between the dependent and independent variables, all variables being essentially treated the same. Originally, PCA and FA were regarded as distinct methods but in recent times they have been combined into a single analysis, PCA often being the first stage of a FA. The basic objective of a PCA/FA is to examine the relationships between the variables or the ‘structure’ of the variables and to determine whether these relationships can be explained by a smaller number of ‘factors’. This statnote describes the use of PCA/FA in the analysis of the differences between the DNA profiles of different MRSA strains introduced in Statnote 26.

Principal components analysis of Alzheimer’s disease based on neuropathological data: A study of 79 patients

A Principal Components Analysis of neuropathological data from 79 Alzheimer’s disease (AD) cases was performed to determine whether there was evidence for subtypes of the disease. Two principal components were extracted from the data which accounted for 72% and 12% of the total variance respectively. The results suggested that 1) AD was heterogeneous but subtypes could not be clearly defined; 2) the heterogeneity, in part, reflected disease onset; 3) familial cases did not constitute a distinct subtype of AD and 4) there were two forms of late onset AD, one of which was associated with less senile plaque and neurofibrillary tangle development but with a greater degree of brain atherosclerosis.

Characterisation of Aeromonas strains and species by pulsed field gel electrophoresis and principal components analysis

Aeromonas genomes were investigated by restriction digesting chromosomal DNA with the endonuclease XbaI, separation of restriction fragments by pulsed field gel electrophoresis (PFGE) and principal components analysis (PCA) of resulting separation patterns. A. salmonicida salmonicida were unique amongst the isolates investigated. Separation profiles of these isolates were similar and all characterised by a distinct absence of bands in the 250kb region. Principal components analysis represented these strains as a clearly defined homogeneous group separated by insignificant Euclidian distances. However, A. salmonicida achromogenes isolates in common with those of A. hydrophila and A. sobria were shown by principal components analysis to be more heterogeneous in nature. Fragments from these isolates were more uniform in size distribution but as demonstrated by the Euclidian distances attained through PCA potentially characteristic of each strain. Furthermore passaging of Aeromonas isolates through an appropriate host did not greatly modify fragment separation profiles, indicative of the genomic stability of test aeromonads and the potential of restriction digesting/PFGE/PCA in Aeromonas typing.

The relationship between senile plaques, neurofibrillary tangles and amyloid (A4) deposits in Alzheimer’s disease: A Principal Components Analysis

A Principal Components Analysis (PCA) was carried out on the density of lesions revealed by different stains in a total of 47 brain regions from six elderly patients with Alzheimer’s disease (AD). The aim was to determine the relationships between the density of senile plaques (SP) revealed by the Glees and Gallyas stains and A4 deposits and between the plaques and neurofibrillary tangles (NFT) in the same brain region. The analysis indicated that the populations of plaques revealed by the Glees and Gallyas stains were closely related to the A4 protein deposits but none of the lesions were related to NFT. The data suggest: 1) that neocortical regions differ from the hippocampus in the relative development of A4 and NFT; the former having more A4 deposits and the latter more NFT and 2) that the processes that lead to the formation of SP and NFT occur independently of each other in the same brain region.

Data analysis methods in optometry Part 8: Principal components analysis (PCA) and factor analysis (FA)

PCA/FA is a method of analyzing complex data sets in which there are no clearly defined X or Y variables. It has multiple uses including the study of the pattern of variation between individual entities such as patients with particular disorders and the detailed study of descriptive variables. In most applications, variables are related to a smaller number of ‘factors’ or PCs that account for the maximum variance in the data and hence, may explain important trends among the variables. An increasingly important application of the method is in the ‘validation’ of questionnaires that attempt to relate subjective aspects of a patients experience with more objective measures of vision.

Principal components analysis of Alzheimer's disease based on neuropathological data: a study of 79 patients

A principal components analysis was carried out on neuropathological data collected from 79 cases of Alzheimer's disease (AD) diagnosed in a single centre. The purpose of the study was to determine whether on neuropathological criteria there was evidence for clearly defined subtypes of the disease. Two principal components (PC1 and PC2) were extracted from the data. PC1 was considerable more important than PC2 accounting for 72% of the total variance. When plotted in relation to the first two principal components the majority of cases (65/79) were distributed in a single cluster within which subgroupings were not clearly evident. In addition, there were a number of individual, mainly early-onset cases, which were neither related to each other nor to the main cluster. The distribution of each neuropathological feature was examined in relation to PC1 and 2, Disease onset, rhe degree of gross brain atrophy, neuronal loss and the devlopment of senile plaques (SP) and neurofibrillary tangles (NFT) were negatively correlated with PC1. The devlopment of SP and NFT and the degree of brain athersclerosis were positively correlated with PC2. These results suggested: 1) that there were different forms of AD but no clear division of the cases into subclasses could be made based on the neuropathological criteria used; the cases showing a more continuous distribution from one form to another, 2) that disease onset was an important variable and was associated with a greater development of pathological changes, 3) familial cases were not a distinct subclass of AD; the cases being widely distributed in relation to PC1 and PC2 and 4) that there may be two forms of late-onset AD whic grade into each other, one of which was associated with less SP and NFT development but with a greater degree of brain atherosclerosis.