Using the EM Algorithm to Train Neural Networks: Misconceptions and a New Algorithm for Multiclass Classification

The expectation-maximization (EM) algorithm has been of considerable interest in recent years as the basis for various algorithms in application areas of neural networks such as pattern recognition. However, there exists some misconceptions concerning its application to neural networks. In this paper, we clarify these misconceptions and consider how the EM algorithm can be adopted to train multilayer perceptron (MLP) and mixture of experts (ME) networks in applications to multiclass classification. We identify some situations where the application of the EM algorithm to train MLP networks may be of limited value and discuss some ways of handling the difficulties. For ME networks, it is reported in the literature that networks trained by the EM algorithm using iteratively reweighted least squares (IRLS) algorithm in the inner loop of the M-step, often performed poorly in multiclass classification. However, we found that the convergence of the IRLS algorithm is stable and that the log likelihood is monotonic increasing when a learning rate smaller than one is adopted. Also, we propose the use of an expectation-conditional maximization (ECM) algorithm to train ME networks. Its performance is demonstrated to be superior to the IRLS algorithm on some simulated and real data sets.

Stability criteria for unsupervised temporal association networks

A biologically realizable, unsupervised learning rule is described for the online extraction of object features, suitable for solving a range of object recognition tasks. Alterations to the basic learning rule are proposed which allow the rule to better suit the parameters of a given input space. One negative consequence of such modifications is the potential for learning instability. The criteria for such instability are modeled using digital filtering techniques and predicted regions of stability and instability tested. The result is a family of learning rules which can be tailored to the specific environment, improving both convergence times and accuracy over the standard learning rule, while simultaneously insuring learning stability.

The parameterless self-organizing map algorithm

The parameterless self-organizing map (PLSOM) is a new neural network algorithm based on the self-organizing map (SOM). It eliminates the need for a learning rate and annealing schemes for learning rate and neighborhood size. We discuss the relative performance of the PLSOM and the SOM and demonstrate some tasks in which the SOM fails but the PLSOM performs satisfactory. Finally we discuss some example applications of the PLSOM and present a proof of ordering under certain limited conditions.

A class of self-stabilizing MCA learning algorithms

In this letter, we propose a class of self-stabilizing learning algorithms for minor component analysis (MCA), which includes a few well-known MCA learning algorithms. Self-stabilizing means that the sign of the weight vector length change is independent of the presented input vector. For these algorithms, rigorous global convergence proof is given and the convergence rate is also discussed. By combining the positive properties of these algorithms, a new learning algorithm is proposed which can improve the performance. Simulations are employed to confirm our theoretical results.

High-field magnetic resonance imaging with reduced field/tissue RF artefacts - A modeling study using hybrid MoM/FEM and FDTD technique

Due to complex field/tissue interactions, high-field magnetic resonance (MR) images suffer significant image distortions that result in compromised diagnostic quality. A new method that attempts to remove these distortions is proposed in this paper and is based on the use of transceiver-phased arrays. The proposed system uses, in the examples presented herein, a shielded four-element transceive-phased array head coil and involves performing two separate scans of the same slice with each scan using different excitations during transmission. By optimizing the amplitudes and phases for each scan, antipodal signal profiles can be obtained, and by combining both the images together, the image distortion can be reduced several fold. A combined hybrid method of moments (MoM)/finite element method (FEM) and finite-difference time-domain (FDTD) technique is proposed and used to elucidate the concept of the new method and to accurately evaluate the electromagnetic field (EMF) in a human head model. In addition, the proposed method is used in conjunction with the generalized auto-calibrating partially parallel acquisitions (GRAPPA) reconstruction technique to enable rapid imaging of the two scans. Simulation results reported herein for 11-T (470-MHz) brain imaging applications show that the new method with GRAPPA reconstruction theoretically results in improved image quality and that the proposed combined hybrid MoM/FEM and FDTD technique is. suitable for high-field magnetic resonance imaging (MRI) numerical analysis.

Engineering of needle-free physical methods to target epidermal cells for DNA vaccination

A challenge in epidermal DNA vaccination is the efficient and targeted delivery of polynucleotides to immunologically sensitive Langerhans cells. This paper investigates this particular challenge for physical delivery approaches. The skin immunology and material properties are examined in the context of the physical cell targeting requirements of the viable epidermis. Selected current physical cell targeting technologies engineered to meet these needs are examined: needle and syringe; diffusion patches; liquid jet injectors; microneedle arrays/patches; and biolistic particle injection. The operating methods and relative performance of these approaches are discussed, with a comment on potential future developments and technologies. (c) 2005 Elsevier Ltd. All rights reserved.

Proceedings of Computational Methods for Coupled Problems in Science and Engineering

Computational Methods for Coupled Problems in Science and Engineering

Electromagnetic fields inside a lossy, multilayered spherical head phantom excited by MRI coils: models and methods

The precise evaluation of electromagnetic field (EMF) distributions inside biological samples is becoming an increasingly important design requirement for high field MRI systems. In evaluating the induced fields caused by magnetic field gradients and RF transmitter coils, a multilayered dielectric spherical head model is proposed to provide a better understanding of electromagnetic interactions when compared to a traditional homogeneous head phantom. This paper presents Debye potential (DP) and Dyadic Green's function (DGF)-based solutions of the EMFs inside a head-sized, stratified sphere with similar radial conductivity and permittivity profiles as a human head. The DP approach is formulated for the symmetric case in which the source is a circular loop carrying a harmonic-formed current over a wide frequency range. The DGF method is developed for generic cases in which the source may be any kind of RF coil whose current distribution can be evaluated using the method of moments. The calculated EMFs can then be used to deduce MRI imaging parameters. The proposed methods, while not representing the full complexity of a head model, offer advantages in rapid prototyping as the computation times are much lower than a full finite difference time domain calculation using a complex head model. Test examples demonstrate the capability of the proposed models/methods. It is anticipated that this model will be of particular value for high field MRI applications, especially the rapid evaluation of RF resonator (surface and volume coils) and high performance gradient set designs.

Implicit stochastic Runge-Kutta methods for stochastic differential equations

In this paper we construct implicit stochastic Runge-Kutta (SRK) methods for solving stochastic differential equations of Stratonovich type. Instead of using the increment of a Wiener process, modified random variables are used. We give convergence conditions of the SRK methods with these modified random variables. In particular, the truncated random variable is used. We present a two-stage stiffly accurate diagonal implicit SRK (SADISRK2) method with strong order 1.0 which has better numerical behaviour than extant methods. We also construct a five-stage diagonal implicit SRK method and a six-stage stiffly accurate diagonal implicit SRK method with strong order 1.5. The mean-square and asymptotic stability properties of the trapezoidal method and the SADISRK2 method are analysed and compared with an explicit method and a semi-implicit method. Numerical results are reported for confirming convergence properties and for comparing the numerical behaviour of these methods.

An industry/university collaboration to upgrade software engineering knowledge and skills in industry

This paper describes an ongoing collaboration between Boeing Australia Limited and the University of Queensland to develop and deliver an introductory course on software engineering. The aims of the course are to provide a common understanding of the nature of software engineering for all Boeing Australia's engineering staff, and to ensure they understand the practices used throughout the company. The course is designed so that it can be presented to people with varying backgrounds, such as recent software engineering graduates, systems engineers, quality assurance personnel, etc. The paper describes the structure and content of the course, and the evaluation techniques used to collect feedback from the participants and the corresponding results. The immediate feedback on the course indicates that it has been well received by the participants, but also indicates a need for more advanced courses in specific areas. The long-term feedback from participants is less positive, and the long-term feedback from the managers of the course participants indicates a need to expand on the coverage of the Boeing-specific processes and methods. (C) 2004 Elsevier Inc. All rights reserved.

Improving learning in undergraduate control engineering courses using context-based learning models

Control Engineering is an essential part of university electrical engineering education. Normally, a control course requires considerable mathematical as well as engineering knowledge and is consequently regarded as a difficult course by many undergraduate students. From the academic point of view, how to help the students to improve their learning of the control engineering knowledge is therefore an important task which requires careful planning and innovative teaching methods. Traditionally, the didactic teaching approach has been used to teach the students the concepts needed to solve control problems. This approach is commonly adopted in many mathematics intensive courses; however it generally lacks reflection from the students to improve their learning. This paper addresses the practice of action learning and context-based learning models in teaching university control courses. This context-based approach has been practised in teaching several control engineering courses in a university with promising results, particularly in view of student learning performances.