Using the EM Algorithm to Train Neural Networks: Misconceptions and a New Algorithm for Multiclass Classification

The expectation-maximization (EM) algorithm has been of considerable interest in recent years as the basis for various algorithms in application areas of neural networks such as pattern recognition. However, there exists some misconceptions concerning its application to neural networks. In this paper, we clarify these misconceptions and consider how the EM algorithm can be adopted to train multilayer perceptron (MLP) and mixture of experts (ME) networks in applications to multiclass classification. We identify some situations where the application of the EM algorithm to train MLP networks may be of limited value and discuss some ways of handling the difficulties. For ME networks, it is reported in the literature that networks trained by the EM algorithm using iteratively reweighted least squares (IRLS) algorithm in the inner loop of the M-step, often performed poorly in multiclass classification. However, we found that the convergence of the IRLS algorithm is stable and that the log likelihood is monotonic increasing when a learning rate smaller than one is adopted. Also, we propose the use of an expectation-conditional maximization (ECM) algorithm to train ME networks. Its performance is demonstrated to be superior to the IRLS algorithm on some simulated and real data sets.

Protein utilisation and turnover in lines of chickens selected for different aspects of body composition

1. Protein utilisation and turnover were measured in male chickens sampled from a line selected for high breast yield and a randombred control line (lines QL and CL, experiment 1) and in male chickens sampled from lines selected for either high or low abdominal fatness (lines FL and LL, experiment 2). In each experiment, 18 birds per line were given iso-energetic (12.9 MJ ME/kg) diets containing either 120 or 220 g CP/kg from 21 to 29 d (experiment 1) and 33 to 43 d (experiment 2). 2. Measurements were made of growth rate, food intake, body composition, excreta production and N-tau-methylhistidine excretion as a measure of myofibrillar protein breakdown, and fractional rates (%/d) of protein deposition, breakdown and synthesis were calculated. 3. In experiment 1, there were no significant differences between the line means for the fractional measures of protein turnover, but there was marked differential response in the two lines in the fractional rates of protein deposition, breakdown and synthesis, to increase in protein intake. The positive slope of the regressions of fractional (%/d) protein deposition and synthesis rates on protein intake (g/d/kg BW) were approximately 1.4- and 2.0-fold higher respectively in the QL than the CL line birds, and the negative slope of the regression of fractional breakdown rate on protein intake was approximately threefold greater in the CL than the QL line birds. 4. In experiment 2, fractional deposition rate was 6.2% lower, but fractional breakdown rate 9.4% higher in the LL than the FL birds, whilst there was essentially no difference in response of the FL and LL birds in the components of protein turnover to increase in protein intake. Line differences in deposition and breakdown rates were thus a reflection of the considerably higher (20%) food and hence protein intake in the FL than the LL birds. 5. The differential line responses in protein turnover in the two experiments suggest that selection for increased breast muscle yield and for reduced body fatness manipulate different physiological pathways in relation to protein turnover, but neither selection strategy results in an improvement in net protein utilisation at typical levels of protein intake by birds on commercial broiler diets, through a reduction in protein breakdown rate.

The mismatch negativity (MMN) response to complex tones and spoken words in individuals with aphasia

Background: The mismatch negativity (MMN) is a fronto-centrally distributed event-related potential (ERP) that is elicited by any discriminable auditory change. It is an ideal neurophysiological tool for measuring the auditory processing skills of individuals with aphasia because it can be elicited even in the absence of attention. Previous MMN studies have shown that acoustic processing of tone or pitch deviance is relatively preserved in aphasia, whereas the basic acoustic processing of speech stimuli can be impaired (e.g., auditory discrimination). However, no MMN study has yet investigated the higher levels of auditory processing, such as language-specific phonological and/or lexical processing, in individuals with aphasia. Aims: The aim of the current study was to investigate the MMN response of normal and language-disordered subjects to tone stimuli and speech stimuli that incorporate the basic auditory processing (acoustic, acoustic-phonetic) levels of non-speech and speech sound processing, and also the language-specific phonological and lexical levels of spoken word processing. Furthermore, this study aimed to correlate the aphasic MMN data with language performance on a variety of tasks specifically targeted at the different levels of spoken word processing. Methods M Procedures: Six adults with aphasia (71.7 years +/- 3.0) and six healthy age-, gender-, and education-matched controls (72.2 years +/- 5.4) participated in the study. All subjects were right-handed and native speakers of English. Each subject was presented with complex harmonic tone stimuli, differing in pitch or duration, and consonant-vowel (CV) speech stimuli (non-word /de:/versus real world/deI/). The probability of the deviant for each tone or speech contrast was 10%. The subjects were also presented with the same stimuli in behavioural discrimination tasks, and were administered a language assessment battery to measure their auditory comprehension skills. Outcomes O Results: The aphasic subjects demonstrated attenuated MMN responses to complex tone duration deviance and to speech stimuli (words and non-words), and their responses to the frequency, duration, and real word deviant stimuli were found to strongly correlate with performance on the auditory comprehension section of the Western Aphasia Battery (WAB). Furthermore, deficits in attentional lexical decision skills demonstrated by the aphasic subjects correlated with a word-related enhancement demonstrated during the automatic MMN paradigm, providing evidence to support the word advantage effect, thought to reflect the activation of language-specific memory traces in the brain for words. Conclusions: These results indicate that the MMN may be used as a technique for investigating general and more specific auditory comprehension skills of individuals with aphasia, using speech and/or non-speech stimuli, independent of the individual's attention. The combined use of the objective MMN technique and current clinical language assessments may result in improved rehabilitative management of aphasic individuals.

A scalarization technique for computing the power and exponential moments of gaussian random matrices

We consider the problems of computing the power and exponential moments EXs and EetX of square Gaussian random matrices X=A+BWC for positive integer s and real t, where W is a standard normal random vector and A, B, C are appropriately dimensioned constant matrices. We solve the problems by a matrix product scalarization technique and interpret the solutions in system-theoretic terms. The results of the paper are applicable to Bayesian prediction in multivariate autoregressive time series and mean-reverting diffusion processes.

A scaleless data model for direct and progressive spatial query processing

A progressive spatial query retrieves spatial data based on previous queries (e.g., to fetch data in a more restricted area with higher resolution). A direct query, on the other side, is defined as an isolated window query. A multi-resolution spatial database system should support both progressive queries and traditional direct queries. It is conceptually challenging to support both types of query at the same time, as direct queries favour location-based data clustering, whereas progressive queries require fragmented data clustered by resolutions. Two new scaleless data structures are proposed in this paper. Experimental results using both synthetic and real world datasets demonstrate that the query processing time based on the new multiresolution approaches is comparable and often better than multi-representation data structures for both types of queries.

Expression of prostate-specific antigen variant MRNA transcripts in prostate cancer and benign prostatic hyperplasia

Prostate-specific antigen (PSA) is important in tumour detection, monitoring disease progression and tumour recurrence. however, PSA is not a cancerspecific marker as levels can also be elevated in benign prostatic disease. A number of different mRNA transcripts of PSA have also been identified in prostatic tissue, but have not been fully characterized (PSA 424, PSA 525, Schulz transcript). Tissue specimens from transurethral resection of the prostate (TURP) or radical prostatectomy were obtained from 17 men with BPH and 15 men with prostate cancer. Total RNA was extracted, and reverse-transcriptionpolymerase chain reaction (RT-PCR) and Southern analysis carried out using transcript-specific primers and probes to determine which mRNA PSA transcripts were expressed. Real-time PCR was performed to determine transcript levels between the two groups using transcript-specific primers and SYBR green fluorescence. Values obtained were normalized to a standard housekeeping gene, B2-microglobulin. Transcripts amplified by RT-PCR and real-time PCR were confirmed by DNA sequencing. Our results show that the transcripts were present in some, but not all, BPH and cancer samples indicating that they are not specific to either BPH or cancer. Analysis of real-time PCR normalized values using a Student’s t -test, shows that there is a significant difference between the two groups for PSA 424, but not wild-type PSA, PSA 525 or the Schulz transcript. Although a larger cohort of samples is needed to further confirm these results, these findings suggest that mRNA levels of PSA 424 may have some utility as a diagnostic or prognostic marker in prostate cancer detection.

A matching pursuit-based signal complexity measure for the analysis of newborn EEG

This paper presents a new relative measure of signal complexity, referred to here as relative structural complexity, which is based on the matching pursuit (MP) decomposition. By relative, we refer to the fact that this new measure is highly dependent on the decomposition dictionary used by MP. The structural part of the definition points to the fact that this new measure is related to the structure, or composition, of the signal under analysis. After a formal definition, the proposed relative structural complexity measure is used in the analysis of newborn EEG. To do this, firstly, a time-frequency (TF) decomposition dictionary is specifically designed to compactly represent the newborn EEG seizure state using MP. We then show, through the analysis of synthetic and real newborn EEG data, that the relative structural complexity measure can indicate changes in EEG structure as it transitions between the two EEG states; namely seizure and background (non-seizure).

Effects of data characteristics on the results of reserve selection algorithms

We tested the effects of four data characteristics on the results of reserve selection algorithms. The data characteristics were nestedness of features (land types in this case), rarity of features, size variation of sites (potential reserves) and size of data sets (numbers of sites and features). We manipulated data sets to produce three levels, with replication, of each of these data characteristics while holding the other three characteristics constant. We then used an optimizing algorithm and three heuristic algorithms to select sites to solve several reservation problems. We measured efficiency as the number or total area of selected sites, indicating the relative cost of a reserve system. Higher nestedness increased the efficiency of all algorithms (reduced the total cost of new reserves). Higher rarity reduced the efficiency of all algorithms (increased the total cost of new reserves). More variation in site size increased the efficiency of all algorithms expressed in terms of total area of selected sites. We measured the suboptimality of heuristic algorithms as the percentage increase of their results over optimal (minimum possible) results. Suboptimality is a measure of the reliability of heuristics as indicative costing analyses. Higher rarity reduced the suboptimality of heuristics (increased their reliability) and there is some evidence that more size variation did the same for the total area of selected sites. We discuss the implications of these results for the use of reserve selection algorithms as indicative and real-world planning tools.

Event-related FMRI analysis with maximum correlation modelling

Functional magnetic resonance imaging (FMRI) analysis methods can be quite generally divided into hypothesis-driven and data-driven approaches. The former are utilised in the majority of FMRI studies, where a specific haemodynamic response is modelled utilising knowledge of event timing during the scan, and is tested against the data using a t test or a correlation analysis. These approaches often lack the flexibility to account for variability in haemodynamic response across subjects and brain regions which is of specific interest in high-temporal resolution event-related studies. Current data-driven approaches attempt to identify components of interest in the data, but currently do not utilise any physiological information for the discrimination of these components. Here we present a hypothesis-driven approach that is an extension of Friman's maximum correlation modelling method (Neurolmage 16, 454-464, 2002) specifically focused on discriminating the temporal characteristics of event-related haemodynamic activity. Test analyses, on both simulated and real event-related FMRI data, will be presented.

Does 'imprinting' with low prenatal vitamin D contribute to the risk of various adult disorders?

Hypovitaminosis D is a candidate risk-modifying factor for a diverse range of disorders apart from rickets and osteoporosis. Based on epidemiology, and on in vitro and animal experiment, vitamin D has been linked to multiple sclerosis, certain cancers (prostate, breast and colorectal), insulin-dependent diabetes mellitus and schizophrenia. I hypothesise that low pre- and perinatal vitamin D levels imprint on the functional characteristics of various tissues throughout the body, leaving the affected individual at increased risk of developing a range of adult-onset disorders. The hypothesis draws from recent advances in our understanding of the early origin of adult disease and proposes a 'critical window' during which vitamin D levels may have a persisting impact on adult health outcomes. Methods to test the hypothesis are outlined. If correct, the hypothesis has important implications for public health. Careful attention to maternal vitamin D status could translate into diverse improvements in health outcomes for the following generation. (C) 2001 Harcourt Publishers Ltd.

Three-dimensional space-borne synthetic aperture radar (SAR) imaging with multiple pass processing

Three-dimensional (3D) synthetic aperture radar (SAR) imaging via multiple-pass processing is an extension of interferometric SAR imaging. It exploits more than two flight passes to achieve a desired resolution in elevation. In this paper, a novel approach is developed to reconstruct a 3D space-borne SAR image with multiple-pass processing. It involves image registration, phase correction and elevational imaging. An image model matching is developed for multiple image registration, an eigenvector method is proposed for the phase correction and the elevational imaging is conducted using a Fourier transform or a super-resolution method for enhancement of elevational resolution. 3D SAR images are obtained by processing simulated data and real data from the first European Remote Sensing satellite (ERS-1) with the proposed approaches.