Visualization of learning in multilayer perceptron networks using principal component analysis

This paper is concerned with the use of scientific visualization methods for the analysis of feedforward neural networks (NNs). Inevitably, the kinds of data associated with the design and implementation of neural networks are of very high dimensionality, presenting a major challenge for visualization. A method is described using the well-known statistical technique of principal component analysis (PCA). This is found to be an effective and useful method of visualizing the learning trajectories of many learning algorithms such as back-propagation and can also be used to provide insight into the learning process and the nature of the error surface.

Supervisory control of wastewater treatment plants by combining principal component analysis and fuzzy c-means clustering

In this paper a methodology for integrated multivariate monitoring and control of biological wastewater treatment plants during extreme events is presented. To monitor the process, on-line dynamic principal component analysis (PCA) is performed on the process data to extract the principal components that represent the underlying mechanisms of the process. Fuzzy c-means (FCM) clustering is used to classify the operational state. Performing clustering on scores from PCA solves computational problems as well as increases robustness due to noise attenuation. The class-membership information from FCM is used to derive adequate control set points for the local control loops. The methodology is illustrated by a simulation study of a biological wastewater treatment plant, on which disturbances of various types are imposed. The results show that the methodology can be used to determine and co-ordinate control actions in order to shift the control objective and improve the effluent quality.

Drying process of microcrystalline cellulose studied by attenuated total reflection IR spectroscopy with two-dimensional correlation spectroscopy and principal component analysis

Molecular interactions between microcrystalline cellulose (MCC) and water were investigated by attenuated total reflection infrared (ATR/IR) spectroscopy. Moisture-content-dependent IR spectra during a drying process of wet MCC were measured. In order to distinguish overlapping O–H stretching bands arising from both cellulose and water, principal component analysis (PCA) and, generalized two-dimensional correlation spectroscopy (2DCOS) and second derivative analysis were applied to the obtained spectra. Four typical drying stages were clearly separated by PCA, and spectral variations in each stage were analyzed by 2DCOS. In the drying time range of 0–41 min, a decrease in the broad band around 3390 cm−1 was observed, indicating that bulk water was evaporated. In the drying time range of 49–195 min, decreases in the bands at 3412, 3344 and 3286 cm−1 assigned to the O6H6cdots, three dots, centeredO3′ interchain hydrogen bonds (H-bonds), the O3H3cdots, three dots, centeredO5 intrachain H-bonds and the H-bonds in Iβ phase in MCC, respectively, were observed. The result of the second derivative analysis suggests that water molecules mainly interact with the O6H6cdots, three dots, centeredO3′ interchain H-bonds. Thus, the H-bonding network in MCC is stabilized by H-bonds between OH groups constructing O6H6cdots, three dots, centeredO3′ interchain H-bonds and water, and the removal of the water molecules induces changes in the H-bonding network in MCC.

Non-linear principal components analysis: an alternative method for finding patterns in environmental data

The main purpose of this article is to gain an insight into the relationships between variables describing the environmental conditions of the Far Northern section of the Great Barrier Reef, Australia, Several of the variables describing these conditions had different measurement levels and often they had non-linear relationships. Using non-linear principal component analysis, it was possible to acquire an insight into these relationships. Furthermore. three geographical areas with unique environmental characteristics could be identified. Copyright (c) 2005 John Wiley & Sons, Ltd.

Adaptive multiscale principal components analysis for online monitoring of wastewater treatment

Fault detection and isolation (FDI) are important steps in the monitoring and supervision of industrial processes. Biological wastewater treatment (WWT) plants are difficult to model, and hence to monitor, because of the complexity of the biological reactions and because plant influent and disturbances are highly variable and/or unmeasured. Multivariate statistical models have been developed for a wide variety of situations over the past few decades, proving successful in many applications. In this paper we develop a new monitoring algorithm based on Principal Components Analysis (PCA). It can be seen equivalently as making Multiscale PCA (MSPCA) adaptive, or as a multiscale decomposition of adaptive PCA. Adaptive Multiscale PCA (AdMSPCA) exploits the changing multivariate relationships between variables at different time-scales. Adaptation of scale PCA models over time permits them to follow the evolution of the process, inputs or disturbances. Performance of AdMSPCA and adaptive PCA on a real WWT data set is compared and contrasted. The most significant difference observed was the ability of AdMSPCA to adapt to a much wider range of changes. This was mainly due to the flexibility afforded by allowing each scale model to adapt whenever it did not signal an abnormal event at that scale. Relative detection speeds were examined only summarily, but seemed to depend on the characteristics of the faults/disturbances. The results of the algorithms were similar for sudden changes, but AdMSPCA appeared more sensitive to slower changes.

Using multivariate analysis to predict the behaviour of soils under effluent irrigation

Onsite wastewater treatment systems aim to assimilate domestic effluent into the environment. Unfortunately failure of such systems is common and inadequate effluent treatment can have serious environmental implications. The capacity of a particular soil to treat wastewater will change over time. The physical properties influence the rate of effluent movement through the soil and its chemical properties dictate the ability to renovate effluent. A research project was undertaken to determine the role that physical and chemical soil properties play in predicting the long-term behaviour of soil under effluent irrigation and to determine if they have a potential function as early indicators of adverse effects of effluent irrigation on treatment sustainability. Principal Component Analysis (PCA) and Cluster Analysis grouped the soils independently of their soil classifications and allowed us to distinguish the most suitable soils for sustainable long term effluent irrigation and determine the most influential soil parameters to characterise them. Multivariate analysis allowed a clear distinction between soils based on the cation exchange capacities. This in turn correlated well with the soil mineralogy. Mixed mineralogy soils in particular sodium or magnesium dominant soils are the most susceptible to dispersion under effluent irrigation. The soil Exchangeable Sodium Percentage (ESP) was identified as a crucial parameter and was highly correlated with percentage clay, electrical conductivity, exchangeable sodium, exchangeable magnesium and low Ca:Mg ratios (less than 0.5).

Distribution of major, minor, and trace metals in lake environments of Antarctica

The concentrations of major, minor and trace metals were measured in water samples collected from five shallow Antarctic lakes (Carezza, Edmonson Point (No 14 and 15a), Inexpressible Island and Tarn Flat) found in Terra Nova Bay (northern Victoria Land, Antarctica) during the Italian Expeditions of 1993-2001. The total concentrations of a large suite of elements (Al, As, Ba, Ca, Cd, Ce, Co, Cr, Cs, Cu, Fe, Ga, Gd, K, La, Li, Mg, Mn, Mo, Na, Nd, Ni, Pb, Pr, Rb, Sc, Si, Sr, Ta, Ti, U, V, Y, W, Zn and Zr) were determined using spectroscopic techniques (ICP-AES, GF-AAS and ICP-MS). The results are similar to those obtained for the freshwater lakes of the Larsemann Hills, East Antarctica, and for the McMurdo Dry Valleys. Principal Component Analysis (PCA) and Cluster Analysis (CA) were performed to identify groups of samples with similar characteristics and to find correlations between the variables. The variability observed within the water samples is closely connected to the sea spray input; hence, it is primarily a consequence of geographical and meteorological factors, such as distance from the ocean and time of year. The trace element levels, in particular those of heavy metals, are very low, suggesting an origin from natural sources rather than from anthropogenic contamination.

Using biplots to interpret gene expression patterns in plants

Sum: Plant biologists in fields of ecology, evolution, genetics and breeding frequently use multivariate methods. This paper illustrates Principal Component Analysis (PCA) and Gabriel's biplot as applied to microarray expression data from plant pathology experiments. Availability: An example program in the publicly distributed statistical language R is available from the web site (www.tpp.uq.edu.au) and by e-mail from the contact. Contact: scott.chapman@csiro.au.

Electronic nose evaluation of onion headspace volatiles and bulb quality as affected by nitrogen, sulphur and soil type

Edaphic factors affect the quality of onions (Allium cepa). Two experiments were carried out in the field and glasshouse to investigate the effects of N (field: 0, 120 kg ha(-1); glasshouse: 0, 108 kg ha(-1)), S (field: 0, 20 kg ha(-1); glasshouse: 0, 4.35 kg ha(-1)) and soil type (clay, sandy loam) on onion quality. A conducting polymer sensor electronic nose (E-nose) was used to classify onion headspace volatiles. Relative changes in the E-nose sensor resistance ratio (%dR/R) were reduced following N and S fertilisation. A 2D Principal Component Analysis (PCA) of the E-nose data sets accounted for c. 100% of the variations in onion headspace volatiles in both experiments. For the field experiment, E-nose data set clusters for headspace volatiles for no N-added onions overlapped (D-2 = 1.0) irrespective of S treatment. Headspace volatiles of N-fertilised onions for the glasshouse sandy loam also overlapped (D-2 = 1.1) irrespective of S treatment as compared with distinct separations among clusters for the clay soil. N fertilisation significantly (P < 0.01) reduced onion bulb pyruvic acid concentration (flavour) in both experiments. S fertilisation increased pyruvic acid concentration significantly (P < 0.01) in the glasshouse experiment, especially for the clay soil, but had no effect on pyruvic acid concentration in the field. N and S fertilisation significantly (P < 0.01) increased lachrymatory potency (pungency), but reduced total soluble solids (TSS) content in the field experiment. In the glasshouse experiment, N and S had no effect on TSS. TSS content was increased on the clay by 1.2-fold as compared with the sandy loam. Onion tissue N:water-soluble SO42- ratios of between five and eight were associated with greater %dR/R and pyruvic acid concentration values. N did not affect inner bulb tissue microbial load. In contrast, S fertilisation reduced inner bulb tissue microbial load by 80% in the field experiment and between 27% (sandy loam) and 92% (clay) in the glasshouse experiment. Overall, onion bulb quality discriminated by the E-nose responded to N, S and soil type treatments, and reflected their interactions. However, the conventional analytical and sensory measures of onion quality did not correlate with %dR/R.

Multivariate monitoring of a biological wastewater treatment process: a case study at Melbourne Water's Western Treatment Plant

Biological wastewater treatment is a complex, multivariate process, in which a number of physical and biological processes occur simultaneously. In this study, principal component analysis (PCA) and parallel factor analysis (PARAFAC) were used to profile and characterise Lagoon 115E, a multistage biological lagoon treatment system at Melbourne Water's Western Treatment Plant (WTP) in Melbourne, Australia. In this study, the objective was to increase our understanding of the multivariate processes taking place in the lagoon. The data used in the study span a 7-year period during which samples were collected as often as weekly from the ponds of Lagoon 115E and subjected to analysis. The resulting database, involving 19 chemical and physical variables, was studied using the multivariate data analysis methods PCA and PARAFAC. With these methods, alterations in the state of the wastewater due to intrinsic and extrinsic factors could be discerned. The methods were effective in illustrating and visually representing the complex purification stages and cyclic changes occurring along the lagoon system. The two methods proved complementary, with each having its own beneficial features. (C) 2003 Elsevier B.V. All rights reserved.

Evaluation of eight spring onion genotypes, sulphur nutrition and soil-type effects with an electronic nose

Genotype, sulphur (S) nutrition and soil-type effects on spring onion quality were assessed using a 32-conducting polymer sensor E-nose. Relative changes in sensor resistance ratio (% dR/R) varied among eight spring onion genotypes. The % dR/R was reduced by S application in four of the eight genotypes. For the other four genotypes, S application gave no change in % dR/R in three, and increased % dR/R in the other. E-nose classification of headspace volatiles by a two-dimensional principal component analysis (PCA) plot for spring onion genotypes differed for S fertilisation vs. no S fertilisation. Headspace volatiles data set clusters for cv. 'White Lisbon' grown on clay or on sandy loam overlapped when 2.9 [Mahalanobis distance value (D2) = 1.6], or 5.8-(D2 = 0.3) kg S ha-1 was added. In contrast, clear separation (D2 = 7.5) was recorded for headspace volatile clusters for 0 kg S hd-1 on clay vs. sandy loam. Addition of 5.8 kg S ha-1 increased pyruvic acid content (mmole g-1 fresh weight) by 1.7-fold on average across the eight genotypes. However, increased S from 2.9 to 5.8 kg ha-1 did not significantly (P > 0.05) influence % dR/R, % dry matter (DM) or total soluble solids (TSS) contents, but significantly (P < 0.05) increased pyruvic acid content. TSS was significantly (P < 0.05) reduced by S addition, while % DM was unaffected. In conclusion, the 32-conducting polymer E-nose discerned differences in spring onion quality that were attributable to genotype and to variations in growing conditions as shown by the significant (P < 0.05) interaction effects for % dR/R.

Cluster analysis of high-dimensional data: A case study

Normal mixture models are often used to cluster continuous data. However, conventional approaches for fitting these models will have problems in producing nonsingular estimates of the component-covariance matrices when the dimension of the observations is large relative to the number of observations. In this case, methods such as principal components analysis (PCA) and the mixture of factor analyzers model can be adopted to avoid these estimation problems. We examine these approaches applied to the Cabernet wine data set of Ashenfelter (1999), considering the clustering of both the wines and the judges, and comparing our results with another analysis. The mixture of factor analyzers model proves particularly effective in clustering the wines, accurately classifying many of the wines by location.

Face recognition robust to head pose from one sample image

Most face recognition systems only work well under quite constrained environments. In particular, the illumination conditions, facial expressions and head pose must be tightly controlled for good recognition performance. In 2004, we proposed a new face recognition algorithm, Adaptive Principal Component Analysis (APCA) [4], which performs well against both lighting variation and expression change. But like other eigenface-derived face recognition algorithms, APCA only performs well with frontal face images. The work presented in this paper is an extension of our previous work to also accommodate variations in head pose. Following the approach of Cootes et al, we develop a face model and a rotation model which can be used to interpret facial features and synthesize realistic frontal face images when given a single novel face image. We use a Viola-Jones based face detector to detect the face in real-time and thus solve the initialization problem for our Active Appearance Model search. Experiments show that our approach can achieve good recognition rates on face images across a wide range of head poses. Indeed recognition rates are improved by up to a factor of 5 compared to standard PCA.

Limited ecoclinal variation found in Malva parviflora (small-flowered mallow) across the Mediterranean-climatic agricultural region of Western Australia

Malva parviflora L. populations were collected from 24 locations across the Mediterranean-climatic agricultural region of Western Australia and grown in Perth in a common garden experiment. Seventeen morphometric and taxonomic measurements were taken and genetic variation was investigated by performing principal components analysis (PCA). Taxonomic measurements confirmed that all plants used in the study were M. parviflora. Greater variation occurred within populations than between populations. Separation between populations was only evident between northern and southern populations along principal components 2 (PC2), which was due mainly to flowering time. Flowering time and consequently photoperiod were highly correlated with latitude and regression analysis revealed a close relationship (r(2) = 0.6). Additionally, the pollination system of M. parviflora was examined. Plants were able to self-pollinate without the need for external vectors and the pollen ovule ratio (31 +/- 1.3) revealed that M. parviflora is most likely to be an obligate inbreeder with a slight potential for outcrossing. The limited variation of M. parviflora enhances the likelihood of suitable control strategies being effective across a broad area.