Using the EM Algorithm to Train Neural Networks: Misconceptions and a New Algorithm for Multiclass Classification

The expectation-maximization (EM) algorithm has been of considerable interest in recent years as the basis for various algorithms in application areas of neural networks such as pattern recognition. However, there exists some misconceptions concerning its application to neural networks. In this paper, we clarify these misconceptions and consider how the EM algorithm can be adopted to train multilayer perceptron (MLP) and mixture of experts (ME) networks in applications to multiclass classification. We identify some situations where the application of the EM algorithm to train MLP networks may be of limited value and discuss some ways of handling the difficulties. For ME networks, it is reported in the literature that networks trained by the EM algorithm using iteratively reweighted least squares (IRLS) algorithm in the inner loop of the M-step, often performed poorly in multiclass classification. However, we found that the convergence of the IRLS algorithm is stable and that the log likelihood is monotonic increasing when a learning rate smaller than one is adopted. Also, we propose the use of an expectation-conditional maximization (ECM) algorithm to train ME networks. Its performance is demonstrated to be superior to the IRLS algorithm on some simulated and real data sets.

Symptoms and diagnosis of borderline, early and advanced epithelial ovarian cancer

Objective. To document symptoms associated with borderline, early and advanced ovarian cancer and identify personal characteristics associated with early versus late diagnosis. Methods. Information concerning symptoms and diagnosis history was available from 811 women with ovarian cancer who took part in an Australian case–control study in the early 1990s. Women were classified into three groups for comparison based on their diagnosis: borderline, early (stage I–II) and advanced (stage III–IV) invasive cancer. Results. Sixteen percent of women with borderline tumors, 7% with early cancer and 4% with advanced cancer experienced no symptoms before diagnosis (P < 0.0001). Among women with symptoms, abdominal pain (44%) or swelling (39%) were most frequently reported; an abdominal mass (12%) and gynecological symptoms (12%) were less common. Compared to advanced stage cancer, women with early stage cancer were more likely to report an abdominal mass or urinary symptoms but less likely to report gastrointestinal problems or general malaise. General malaise and ‘other’ symptoms were least common in borderline disease. Older women, and those with higher parity or a family history of breast or ovarian cancer, were more likely to be diagnosed at an advanced stage of disease. Conclusions. Women who experience persistent or recurrent abdominal symptoms, particularly swelling and/or pain should be encouraged to seek medical attention and physicians should be alert to the possibility of ovarian cancer even in the absence of an abdominal mass. Further information about the prevalence of these symptoms in the general population is essential to assist physicians in patient management.

Sporadic and familial pheochromocytomas are associated with loss of at least two discrete intervals on chromosome 1p

Pheochromocytomas are tumors of the adrenal medulla originating in the chromaffin cells derived from the neural crest. Ten % of these tumors are associated with the familial cancer syndromes multiple endocrine neoplasia type 2, von Hippel-Lindau disease (VHL), and rarely, neurofibromatosis type 1, in which germ-line mutations have been identified in RET, VHL, and NF1, respectively. In both the sporadic and familial forms of pheochromocytoma, allelic loss at 1p, 3p, 17p, and 22q has been reported, yet the molecular pathogenesis of these tumors is largely unknown. Allelic loss at chromosome 1p has also been reported in other endocrine tumors, such as medullary thyroid cancer and tumors of the parathyroid gland, as well as in tumors of neural crest origin including neuroblastoma and malignant melanoma, In this study, we performed fine structure mapping of deletions at chromosome 1p in familial and sporadic pheochromocytomas to identify discrete regions likely housing tumor suppressor genes involved in the development of these tumors. Ten microsatellite markers spanning a region of similar to 70 cM (Ipter to 1p34.3) were used to screen 20 pheochromocytomas from 19 unrelated patients for loss of heterozygosity (LOH). LOH was detected at five or more loci in 8 of 13 (61%)sporadic samples and at five or more loci in four of five (80%) tumor samples from patients with multiple endocrine neoplasia type 2. No LOH at 1p was detected in pheochromocytomas from two VHL patients, Analysis of the combined sporadic and familial tumor data suggested three possible regions of common somatic loss, designated as PCI (D1S243 to D1S244), PC2 (D1S228 to D1S507), and PC3 (D1S507 toward the centromere). We propose that chromosome Ip may be the site of at least three putative tumor suppressor loci involved in the tumorigenesis of pheochromocytomas. At least one of these loci, PC2 spanning an interval of <3.8 cM, is Likely to have a broader role in the development of endocrine malignancies.

A simulated annealing algorithm for finding consensus sequences

Motivation: A consensus sequence for a family of related sequences is, as the name suggests, a sequence that captures the features common to most members of the family. Consensus sequences are important in various DNA sequencing applications and are a convenient way to characterize a family of molecules. Results: This paper describes a new algorithm for finding a consensus sequence, using the popular optimization method known as simulated annealing. Unlike the conventional approach of finding a consensus sequence by first forming a multiple sequence alignment, this algorithm searches for a sequence that minimises the sum of pairwise distances to each of the input sequences. The resulting consensus sequence can then be used to induce a multiple sequence alignment. The time required by the algorithm scales linearly with the number of input sequences and quadratically with the length of the consensus sequence. We present results demonstrating the high quality of the consensus sequences and alignments produced by the new algorithm. For comparison, we also present similar results obtained using ClustalW. The new algorithm outperforms ClustalW in many cases.

A monolithic smoothing-gap algorithm for contact-impact based on the signed distance function

A new algorithm has been developed for smoothing the surfaces in finite element formulations of contact-impact. A key feature of this method is that the smoothing is done implicitly by constructing smooth signed distance functions for the bodies. These functions are then employed for the computation of the gap and other variables needed for implementation of contact-impact. The smoothed signed distance functions are constructed by a moving least-squares approximation with a polynomial basis. Results show that when nodes are placed on a surface, the surface can be reproduced with an error of about one per cent or less with either a quadratic or a linear basis. With a quadratic basis, the method exactly reproduces a circle or a sphere even for coarse meshes. Results are presented for contact problems involving the contact of circular bodies. Copyright (C) 2002 John Wiley Sons, Ltd.

Molecular evidence for the common origin of snap-traps among carnivorous plants

The snap-trap leaves of the aquatic waterwheel plant (Aldrovanda) resemble those of Venus' flytrap (Dionaea), its distribution and habit are reminiscent of bladderworts (Utricularia), but it shares many reproductive characters with sundews (Drosera). Moreover, Aldrovanda has never been included in molecular phylogenetic studies, so it has been unclear whether snap-traps evolved only once or more than once among angiosperms. Using sequences from nuclear 18S and plastid rbcL, atpB, and matK genes, we show that Aldrovanda is sister to Dionaea, and this pair is sister to Drosera. Our results indicate that snap-traps are derived from flypaper-traps and have a common ancestry among flowering plants, despite the fact that this mechanism is used by both a terrestrial species and an aquatic one. Genetic and fossil evidence for the close relationship between these unique and threatened organisms indicate that carnivory evolved from a common ancestor within this caryophyllid clade at least 65 million years ago.

Genetic variability in cultivated common bean beyond the two major gene pools

It is generally accepted that two major gene pools exist in cultivated common bean (Phaseolus vulgaris L.), a Middle American and an Andean one. Some evidence, based on unique phaseolin morphotypes and AFLP analysis, suggests that at least one more gene pool exists in cultivated common bean. To investigate this hypothesis, 1072 accessions from a common bean core collection from the primary centres of origin, held at CIAT, were investigated. Various agronomic and morphological attributes (14 categorical and 11 quantitative) were measured. Multivariate analyses, consisting of homogeneity analysis and clustering for categorical data, clustering and ordination techniques for quantitative data and nonlinear principal component analysis for mixed data, were undertaken. The results of most analyses supported the existence of the two major gene pools. However, the analysis of categorical data of protein types showed an additional minor gene pool. The minor gene pool is designated North Andean and includes phaseolin types CH, S and T; lectin types 312, Pr, B and K; and mostly A5, A6 and A4 types alpha-amylase inhibitor. Analysis of the combined categorical data of protein types and some plant categorical data also suggested that some other germplasm with C type phaseolin are distinguished from the major gene pools.

A subset-oriented algorithm for minimizing the number of steps required for synthesis of cyclic-peptide libraries

Libraries of cyclic peptides are being synthesized using combinatorial chemistry for high throughput screening in the drug discovery process. This paper describes the min_syn_steps.cpp program (available at http://www.imb.uq.edu.au/groups/smythe/tran), which after inputting a list of cyclic peptides to be synthesized, removes cyclic redundant sequences and calculates synthetic strategies which minimize the synthetic steps as well as the reagent requirements. The synthetic steps and reagent requirements could be minimized by finding common subsets within the sequences for block synthesis. Since a brute-force approach to search for optimum synthetic strategies is impractically large, a subset-orientated approach is utilized here to limit the size of the search. (C) 2002 Elsevier Science Ltd. All rights reserved.

Use of the EM algorithm to detect QTL affecting multiple-traits in an across half-sib family analysis

QTL detection experiments in livestock species commonly use the half-sib design. Each male is mated to a number of females, each female producing a limited number of progeny. Analysis consists of attempting to detect associations between phenotype and genotype measured on the progeny. When family sizes are limiting experimenters may wish to incorporate as much information as possible into a single analysis. However, combining information across sires is problematic because of incomplete linkage disequilibrium between the markers and the QTL in the population. This study describes formulae for obtaining MLEs via the expectation maximization (EM) algorithm for use in a multiple-trait, multiple-family analysis. A model specifying a QTL with only two alleles, and a common within sire error variance is assumed. Compared to single-family analyses, power can be improved up to fourfold with multi-family analyses. The accuracy and precision of QTL location estimates are also substantially improved. With small family sizes, the multi-family, multi-trait analyses reduce substantially, but not totally remove, biases in QTL effect estimates. In situations where multiple QTL alleles are segregating the multi-family analysis will average out the effects of the different QTL alleles.

A gradient descent algorithm for minimizing amino acid coupling reactions when synthesizing cyclic-peptide libraries

Combinatorial chemistry has become an invaluable tool in medicinal chemistry for the identification of new drug leads. For example, libraries of predetermined sequences and head-to-tail cyclized peptides are routinely synthesized in our laboratory using the IRORI approach. Such libraries are used as molecular toolkits that enable the development of pharmacophores that define activity and specificity at receptor targets. These libraries can be quite large and difficult to handle, due to physical and chemical constraints imposed by their size. Therefore, smaller sub-libraries are often targeted for synthesis. The number of coupling reactions required can be greatly reduced if the peptides having common amino acids are grouped into the same sub-library (batching). This paper describes a schedule optimizer to minimize the number of coupling reactions by rotating and aligning sequences while simultaneously batching. The gradient descent method thereby reduces the number of coupling reactions required for synthesizing cyclic peptide libraries. We show that the algorithm results in a 75% reduction in the number of coupling reactions for a typical cyclic peptide library.

Enhanced detection-guided NLMS estimation of sparse FIR-modeled signal channels

In various signal-channel-estimation problems, the channel being estimated may be well approximated by a discrete finite impulse response (FIR) model with sparsely separated active or nonzero taps. A common approach to estimating such channels involves a discrete normalized least-mean-square (NLMS) adaptive FIR filter, every tap of which is adapted at each sample interval. Such an approach suffers from slow convergence rates and poor tracking when the required FIR filter is "long." Recently, NLMS-based algorithms have been proposed that employ least-squares-based structural detection techniques to exploit possible sparse channel structure and subsequently provide improved estimation performance. However, these algorithms perform poorly when there is a large dynamic range amongst the active taps. In this paper, we propose two modifications to the previous algorithms, which essentially remove this limitation. The modifications also significantly improve the applicability of the detection technique to structurally time varying channels. Importantly, for sparse channels, the computational cost of the newly proposed detection-guided NLMS estimator is only marginally greater than that of the standard NLMS estimator. Simulations demonstrate the favourable performance of the newly proposed algorithm. © 2006 IEEE.

Efficient spatial clustering algorithm using binary tree

In this paper we present an efficient k-Means clustering algorithm for two dimensional data. The proposed algorithm re-organizes dataset into a form of nested binary tree*. Data items are compared at each node with only two nearest means with respect to each dimension and assigned to the one that has the closer mean. The main intuition of our research is as follows: We build the nested binary tree. Then we scan the data in raster order by in-order traversal of the tree. Lastly we compare data item at each node to the only two nearest means to assign the value to the intendant cluster. In this way we are able to save the computational cost significantly by reducing the number of comparisons with means and also by the least use to Euclidian distance formula. Our results showed that our method can perform clustering operation much faster than the classical ones. © Springer-Verlag Berlin Heidelberg 2005

A randomised control trial of exercise and manipulative therapy for cervicogenic headache

Study Design. A multicenter, randomized controlled trial with unblinded treatment and blinded outcome assessment was conducted. The treatment period was 6 weeks with follow-up assessment after treatment, then at 3, 6, and 12 months. Objectives. To determine the effectiveness of manipulative therapy and a low-load exercise program for cervicogenic headache when used alone and in combination, as compared with a control group. Summary of Background Data. Headaches arising from cervical musculoskeletal disorders are common. Conservative therapies are recommended as the first treatment of choice. Evidence for the effectiveness of manipulative therapy is inconclusive and available only for the short term. There is no evidence for exercise, and no study has investigated the effect of combined therapies for cervicogenic headache. Methods. In this study, 200 participants who met the diagnostic criteria for cervicogenic headache were randomized into four groups: manipulative therapy group, exercise therapy group, combined therapy group, and a control group. The primary outcome was a change in headache frequency. Other outcomes included changes in headache intensity and duration, the Northwick Park Neck Pain Index, medication intake, and patient satisfaction. Physical outcomes included pain on neck movement, upper cervical joint tenderness, a craniocervical flexion muscle test, and a photographic measure of posture. Results. There were no differences in headache-related and demographic characteristics between the groups at baseline. The loss to follow-up evaluation was 3.5%. At the 12-month follow-up assessment, both manipulative therapy and specific exercise had significantly reduced headache frequency and intensity, and the neck pain and effects were maintained (P < 0.05 for all). The combined therapies was not significantly superior to either therapy alone, but 10% more patients gained relief with the combination. Effect sizes were at least moderate and clinically relevant. Conclusion. Manipulative therapy and exercise can reduce the symptoms of cervicogenic headache, and the effects are maintained.

Tenure, mobility and retention of nurses in Queensland, Australia: 2001 and 2004

Aim. Data were collected on tenure, mobility and retention of the nursing workforce in Queensland to aid strategic planning by the Queensland Nurses Union (QNU). Background. Shortages of nurses negatively affect the health outcomes of patients. Population rise is increasing the demand for nurses in Queensland. The supply of nurses is affected by recruitment of new and returning nurses, retention of the existing workforce and mobility within institutions. Methods. A self-reporting, postal survey was undertaken by the QNU members from the major employment sectors of aged care, public acute and community health and private acute and community health. Results. Only 60% of nurses had been with their current employer more than 5 years. In contrast 90% had been in nursing for 5 years or more and most (80%) expected to remain in nursing for at least another 5 years. Breaks from nursing were common and part-time positions in the private and aged care sectors offered flexibility. Conclusion. The study demonstrated a mobile nursing workforce in Queensland although data on tenure and future time in nursing suggested that retention in the industry was high. Concern is expressed for replacement of an ageing nursing population.