Visualization of learning in multilayer perceptron networks using principal component analysis

This paper is concerned with the use of scientific visualization methods for the analysis of feedforward neural networks (NNs). Inevitably, the kinds of data associated with the design and implementation of neural networks are of very high dimensionality, presenting a major challenge for visualization. A method is described using the well-known statistical technique of principal component analysis (PCA). This is found to be an effective and useful method of visualizing the learning trajectories of many learning algorithms such as back-propagation and can also be used to provide insight into the learning process and the nature of the error surface.

Using the EM Algorithm to Train Neural Networks: Misconceptions and a New Algorithm for Multiclass Classification

The expectation-maximization (EM) algorithm has been of considerable interest in recent years as the basis for various algorithms in application areas of neural networks such as pattern recognition. However, there exists some misconceptions concerning its application to neural networks. In this paper, we clarify these misconceptions and consider how the EM algorithm can be adopted to train multilayer perceptron (MLP) and mixture of experts (ME) networks in applications to multiclass classification. We identify some situations where the application of the EM algorithm to train MLP networks may be of limited value and discuss some ways of handling the difficulties. For ME networks, it is reported in the literature that networks trained by the EM algorithm using iteratively reweighted least squares (IRLS) algorithm in the inner loop of the M-step, often performed poorly in multiclass classification. However, we found that the convergence of the IRLS algorithm is stable and that the log likelihood is monotonic increasing when a learning rate smaller than one is adopted. Also, we propose the use of an expectation-conditional maximization (ECM) algorithm to train ME networks. Its performance is demonstrated to be superior to the IRLS algorithm on some simulated and real data sets.

A pore-size-dependent equation of state for multilayer adsorption in cylindrical mesopores

The chemical potential of adsorbed film inside cylindrical mesopores is dependent on the attractive interactions between the adsorbed molecules and adsorbent, the curvature of gas/adsorbed phase interface, and surface tension. A state equation of the adsorbed film is proposed to take into account the above factors. Nitrogen adsorption on model adsorbents, MCM-41, which exhibit uniform cylindrical channels, are used to verify the theoretical analysis. The proposed theory is capable of describing the important features of adsorption processes in cylindrical mesopores. According to this theory, at a given relative pressure, the smaller the pore radius is, the thicker the adsorbed film will be. The thickening of adsorbed films in the pores as the vapor pressure increases inevitably causes an increase in the interface curvature, which consequently leads to capillary condensation. Besides, this study confirmed that the interface tension depends substantially on the interface curvature in small mesopores. A quantitative relationship between the condensation pressure and the pore radius can be derived from the state equation and used to predict the pore radius from a condensation pressure, or vice versa.

Surface diffusion of adsorbed molecules in porous media: Monolayer, multilayer, and capillary condensation regimes

This review provides an overview of surface diffusion and capillary condensate flow in porous media. Emphasis has been placed on the distinction between purely surface diffusion, multilayer surface diffusion, and, capillary condensate flow.

Compact clinical MRI magnet design using a multilayer current density approach

In this work, a new method of optimization is successfully applied to the theoretical design of compact, actively shielded, clinical MRI magnets. The problem is formulated as a two-step process in which the desired current densities on multiple, cc-axial surface layers are first calculated by solving Fredholm equations of the first kind. Non-linear optimization methods with inequality constraints are then invoked to fit practical magnet coils to the desired current densities. The current density approach allows rapid prototyping of unusual magnet designs. The emphasis of this work is on the optimal design of short, actively-shielded MRI magnets for whole-body imaging. Details of the hybrid numerical model are presented, and the model is used to investigate compact, symmetric, and asymmetric MRI magnets. Magnet designs are presented for actively-shielded, symmetric magnets of coil length 1.0 m, which is considerably shorter than currently available designs of comparable dsv size. Novel, actively-shielded, asymmetric magnet designs are also presented in which the beginning of a 50-cm dsv is positioned just 11 cm from the end of the coil structure, allowing much improved access to the patient and reduced patient claustrophobia. Magn Reson Med 45:331540, 2001. (C) 2001 Wiley-Liss, Inc.

Empirical evidence for ultrametric structure in multi-layer perceptron error surfaces

Combinatorial optimization problems share an interesting property with spin glass systems in that their state spaces can exhibit ultrametric structure. We use sampling methods to analyse the error surfaces of feedforward multi-layer perceptron neural networks learning encoder problems. The third order statistics of these points of attraction are examined and found to be arranged in a highly ultrametric way. This is a unique result for a finite, continuous parameter space. The implications of this result are discussed.

Rapid colour changes in multilayer reflecting stripes in the paradise whiptail, Pentapodus paradiseus

The Paradise whiptail (Pentapodus paradiseus) has distinct reflective stripes on its head and body. The reflective stripes contain a dense layer of physiologically active iridophores, which act as multilayer reflectors. The wavelengths reflected by these stripes can change from blue to red in 0.25 s. Transmission electron microscopy revealed that the iridophore cells contain plates that are, on average, 51.4 nm thick. This thickness produces a stack, which acts as an ideal quarter-wavelength multilayer reflector (equal optical thickness of plates and spaces) in the blue, but not the red, region of the spectrum. When skin preparations were placed into hyposmotic physiological saline, the peak wavelength of the reflected light shifted towards the longer (red) end of the visible spectrum. Hyperosmotic saline reversed this effect and shifted the peak wavelength towards shorter (blue/UV) wavelengths. Norepinephrine (100 mumol l(-1)) shifted the peak wavelength towards the longer end of the spectrum, while adenosine (100 mumol l(-1)) reversed the effects of norepinephrine. The results from this study show that the wavelength changes are elicited by a change of similar to70 nm in the distance between adjacent plates in the iridophore cells.

Multilayer transverse gradient coil design

In small, cylindrical gradient coils consisting of a single layer of wires, the limiting factor in achieving large magnetic field gradients is the rapid increase in coil resistance with efficiency. This behavior results from the decrease in the maximum usable wire diameter as the number of turns is increased. By adopting a multilayer design in which the coil wires are allowed to spread out into multiple layers wound at increasing radii, a more favorable scaling of resistance with efficiency is achieved, thus allowing the design of more powerful gradient coils with acceptable resistance values. By extending the theory used to design standard cylindrical gradient coils, mathematical expressions have been developed that allow the design of multilayer coils. These expressions have previously been applied to the design of a four-layer z-gradient coil. As a further development, the equations have now been modified to allow the design of multilayer transverse gradient coils. The variation in coil performance with the number of layers employed has been investigated for coils of a size suitable for use in NMR microscopy, and the effect of constructing the coil using wires or cuts in a continuous conducting surface has also been assessed. We find that at fixed resistance a small wire-wound two-layer coil offers an increase in efficiency of a factor of about 1.5 compared with a single-layer coil. In addition, a two-layer coil of 10-mm inner diameter has been designed and built. This coil had an efficiency of 0.41 Tm-1 A(-1), a resistance of 0.96 +/- 0.01 Omega, and an inductance of 22.3 +/- 0.2 muH. The coil produces a gradient that deviates from linearity by less than 5% over a central cylindrical region of interest of height and length 6.2 mm. (C) 2003 Wiley Periodicals, Inc.

Prediction of multilayer adsorption and capillary condensation phenomena in cylindrical mesopores

MCM-41 periodic mesoporous silicates with a high degree of structural ordering are synthesized and used as model adsorbents to study the isotherm prediction of nitrogen adsorption. The nitrogen adsorption isotherm at 77 K for a macroporous silica is measured and used in high-resolution alpha(s)-plot comparative analysis to determine the external surface area, total surface area and primary mesopore volume of the MCM-41 materials. Adsorption equilibrium data of nitrogen on the different pore size MCM-41 samples (pore diameters from 2.40 to 4.92 nm) are also obtained. Based on the Broekhoff and de Boer' thermodynamic analysis, the nitrogen adsorption isotherms for the different pore size MCM-41 samples are interpreted using a novel strategy, in which the parameters of an empirical expression, used to represent the potential of interaction between the adsorbate and adsorbent, are obtained by fitting only the multilayer region prior to capillary condensation for C-16 MCM-41. Subsequently the entire isotherm, including the phase transition, is predicted for all the different pore size MCM-41 samples without any fitting. The results show that the prediction of multilayer adsorption and total adsorbed amount are in good agreement with the experimental isotherms. The predictions of the relative pressure corresponding to capillary equilibrium (coexistence) transition agree remarkably with experimental data on the adsorption branch even for hysteretic isotherms, confirming that this is the branch appropriate for pore size distribution analysis. The impact of pore radius on the adsorption film thickness and capillary coexistence pressure is also investigated, and found to agree with the experimental data. (C) 2003 Elsevier Inc. All rights reserved.

Multigradient field-active contour model for multilayer boundary detection of ultrasound rectal wall image

Extraction and reconstruction of rectal wall structures from an ultrasound image is helpful for surgeons in rectal clinical diagnosis and 3-D reconstruction of rectal structures from ultrasound images. The primary task is to extract the boundary of the muscular layers on the rectal wall. However, due to the low SNR from ultrasound imaging and the thin muscular layer structure of the rectum, this boundary detection task remains a challenge. An active contour model is an effective high-level model, which has been used successfully to aid the tasks of object representation and recognition in many image-processing applications. We present a novel multigradient field active contour algorithm with an extended ability for multiple-object detection, which overcomes some limitations of ordinary active contour models—"snakes." The core part in the algorithm is the proposal of multigradient vector fields, which are used to replace image forces in kinetic function for alternative constraints on the deformation of active contour, thereby partially solving the initialization limitation of active contour for rectal wall boundary detection. An adaptive expanding force is also added to the model to help the active contour go through the homogenous region in the image. The efficacy of the model is explained and tested on the boundary detection of a ring-shaped image, a synthetic image, and an ultrasound image. The experimental results show that the proposed multigradient field-active contour is feasible for multilayer boundary detection of rectal wall

Adsorption in mesopores: A molecular-continuum model with application to MCM-41

The classical model of surface layering followed by capillary condensation during adsorption in mesopores, is modified here by consideration of the adsorbate solid interaction potential. The new theory accurately predicts the capillary coexistence curve as well as pore criticality, matching that predicted by density functional theory. The model also satisfactorily predicts the isotherm for nitrogen adsorption at 77.4 K on MCM-41 material of various pore sizes, synthesized and characterized in our laboratory, including the multilayer region, using only data on the variation of condensation pressures with pore diameter. The results indicate a minimum mesopore diameter for the surface layering model to hold as 14.1 Å, below which size micropore filling must occur, and a minimum pore diameter for mechanical stability of the hemispherical meniscus during desorption as 34.2 Å. For pores in-between these two sizes reversible condensation is predicted to occur, in accord with the experimental data for nitrogen adsorption on MCM-41 at 77.4 K.

Optical properties of metallic films for vertical-cavity optoelectronic devices

We present models for the optical functions of 11 metals used as mirrors and contacts in optoelectronic and optical devices: noble metals (Ag, Au, Cu), aluminum, beryllium, and transition metals (Cr, Ni, Pd, Pt, Ti, W). We used two simple phenomenological models, the Lorentz-Drude (LD) and the Brendel-Bormann (BB), to interpret both the free-electron and the interband parts of the dielectric response of metals in a wide spectral range from 0.1 to 6 eV. Our results show that the BE model was needed to describe appropriately the interband absorption in noble metals, while for Al, Be, and the transition metals both models exhibit good agreement with the experimental data. A comparison with measurements on surface normal structures confirmed that the reflectance and the phase change on reflection from semiconductor-metal interfaces (including the case of metallic multilayers) can be accurately described by use of the proposed models for the optical functions of metallic films and the matrix method for multilayer calculations. (C) 1998 Optical Society of America.

A model for adsorption of condensable vapors on nonporous materials

A new isotherm is proposed here for adsorption of condensable vapors and gases on nonporous materials having type II isotherms according to the Brunauer-Deming-Deming-Teller (BDDH) classification. The isotherm combines the recent molecular-continuum model in the multilayer region, with other widely used models for sub-monolayer coverage, some of which satisfy the requirement of a Henry's law asymptote. The model is successfully tested using isotherm data for nitrogen adsorption on nonporous silica, carbon and alumina, as well as benzene and hexane adsorption on nonporous carbon. Based on the data fits, out of several different alternative choices of model for the monolayer region, the Freundlich and the Unilan models are found to be the most successful when combined with the multilayer model to predict the whole isotherm. The hybrid model is consequently applicable over a wide pressure range. (C) 2000 Elsevier Science B.V. All rights reserved.