Generalization by symbolic abstraction in cascaded recurrent networks

Generalization performance in recurrent neural networks is enhanced by cascading several networks. By discretizing abstractions induced in one network, other networks can operate on a coarse symbolic level with increased performance on sparse and structural prediction tasks. The level of systematicity exhibited by the cascade of recurrent networks is assessed on the basis of three language domains. (C) 2004 Elsevier B.V. All rights reserved.

Prediction of subcellular localization using sequence-biased recurrent networks

Motivation: Targeting peptides direct nascent proteins to their specific subcellular compartment. Knowledge of targeting signals enables informed drug design and reliable annotation of gene products. However, due to the low similarity of such sequences and the dynamical nature of the sorting process, the computational prediction of subcellular localization of proteins is challenging. Results: We contrast the use of feed forward models as employed by the popular TargetP/SignalP predictors with a sequence-biased recurrent network model. The models are evaluated in terms of performance at the residue level and at the sequence level, and demonstrate that recurrent networks improve the overall prediction performance. Compared to the original results reported for TargetP, an ensemble of the tested models increases the accuracy by 6 and 5% on non-plant and plant data, respectively.

Context-free and context-sensitive dynamics in recurrent neural networks

Continuous-valued recurrent neural networks can learn mechanisms for processing context-free languages. The dynamics of such networks is usually based on damped oscillation around fixed points in state space and requires that the dynamical components are arranged in certain ways. It is shown that qualitatively similar dynamics with similar constraints hold for a(n)b(n)c(n), a context-sensitive language. The additional difficulty with a(n)b(n)c(n), compared with the context-free language a(n)b(n), consists of 'counting up' and 'counting down' letters simultaneously. The network solution is to oscillate in two principal dimensions, one for counting up and one for counting down. This study focuses on the dynamics employed by the sequential cascaded network, in contrast to the simple recurrent network, and the use of backpropagation through time. Found solutions generalize well beyond training data, however, learning is not reliable. The contribution of this study lies in demonstrating how the dynamics in recurrent neural networks that process context-free languages can also be employed in processing some context-sensitive languages (traditionally thought of as requiring additional computation resources). This continuity of mechanism between language classes contributes to our understanding of neural networks in modelling language learning and processing.

The applicability of recurrent neural networks for biological sequence analysis

Selection of machine learning techniques requires a certain sensitivity to the requirements of the problem. In particular, the problem can be made more tractable by deliberately using algorithms that are biased toward solutions of the requisite kind. In this paper, we argue that recurrent neural networks have a natural bias toward a problem domain of which biological sequence analysis tasks are a subset. We use experiments with synthetic data to illustrate this bias. We then demonstrate that this bias can be exploitable using a data set of protein sequences containing several classes of subcellular localization targeting peptides. The results show that, compared with feed forward, recurrent neural networks will generally perform better on sequence analysis tasks. Furthermore, as the patterns within the sequence become more ambiguous, the choice of specific recurrent architecture becomes more critical.

Detecting and sorting targeting peptides wtih neural networks and support vector machines

This paper presents a composite multi-layer classifier system for predicting the subcellular localization of proteins based on their amino acid sequence. The work is an extension of our previous predictor PProwler v1.1 which is itself built upon the series of predictors SignalP and TargetP. In this study we outline experiments conducted to improve the classifier design. The major improvement came from using Support Vector machines as a "smart gate" sorting the outputs of several different targeting peptide detection networks. Our final model (PProwler v1.2) gives MCC values of 0.873 for non-plant and 0.849 for plant proteins. The model improves upon the accuracy of our previous subcellular localization predictor (PProwler v1.1) by 2% for plant data (which represents 7.5% improvement upon TargetP).

A fast cross-entropy method for estimating buffer overflows in queuing networks

In this paper, we propose a fast adaptive importance sampling method for the efficient simulation of buffer overflow probabilities in queueing networks. The method comprises three stages. First, we estimate the minimum cross-entropy tilting parameter for a small buffer level; next, we use this as a starting value for the estimation of the optimal tilting parameter for the actual (large) buffer level. Finally, the tilting parameter just found is used to estimate the overflow probability of interest. We study various properties of the method in more detail for the M/M/1 queue and conjecture that similar properties also hold for quite general queueing networks. Numerical results support this conjecture and demonstrate the high efficiency of the proposed algorithm.

A hybrid planning method for transmission networks in a deregulated environment

The reconstruction of power industries has brought fundamental changes to both power system operation and planning. This paper presents a new planning method using multi-objective optimization (MOOP) technique, as well as human knowledge, to expand the transmission network in open access schemes. The method starts with a candidate pool of feasible expansion plans. Consequent selection of the best candidates is carried out through a MOOP approach, of which multiple objectives are tackled simultaneously, aiming at integrating the market operation and planning as one unified process in context of deregulated system. Human knowledge has been applied in both stages to ensure the selection with practical engineering and management concerns. The expansion plan from MOOP is assessed by reliability criteria before it is finalized. The proposed method has been tested with the IEEE 14-bus system and relevant analyses and discussions have been presented.

Long-term risk of first recurrent stroke in the Perth Community Stroke Study

Background and Purpose-Few community-based studies have examined the long-term risk of recurrent stroke after an acute first-ever stroke. This study aimed to determine the absolute and relative risks of a first recurrent stroke over the first 5 years after a first-ever stroke and the predictors of such recurrence in a population-based series of people with first-ever stroke in Perth, Western Australia. Methods-Between February 1989 and August 1990, all people with a suspected acute stroke or transient ischemic attack of the brain who were resident in a geographically defined region of Perth, Western Australia, with a population of 138 708 people, were registered prospectively and assessed according to standardized diagnostic criteria. Patients were followed up prospectively at 4 months, 12 months, and 5 years after the index event. Results-Three hundred seventy patients with a first-ever stroke were registered, of whom 351 survived >2 days. Data were available for 98% of the cohort at 5 years, by which time 199 patients (58%) had died and 52 (15%) had experienced a recurrent stroke, 12 (23%) of which were fatal within 28 days. The 5-year cumulative risk of first recurrent stroke was 22.5% (95% confidence limits [CL], 16.8%, 28.1%). The risk of recurrent stroke was greatest in the first 6 months after stroke, at 8.8% (95% CL, 5.4%, 12.1%). After adjustment for age and sex, the prognostic factors for recurrent stroke were advanced, but not extreme, age (75 to 84 years) (hazard ratio [HR], 2.6; 95% CL, 1.1, 6.2), hemorrhagic index stroke (HR, 2.1; 95% CL, 0.98, 4.4), and diabetes mellitus (HR, 2.1; 95% CL, 0.95, 4.4). Conclusions-Approximately 1 in 6 survivors (15%) of a first-ever stroke experience a recurrent stroke over the next 5 years, of which 25% are fatal within 28 days. The pathological subtype of the recurrent stroke is the same as that of the index stroke in 88% of cases. The predictors of first recurrent stroke in this study were advanced age, hemorrhagic index stroke, and diabetes mellitus, but numbers of recurrent events were modest. Because the risk of recurrent stroke is highest (8.8%) in the first 6 months after stroke, strategies for secondary prevention should be initiated as soon as possible after the index event.

An object-oriented neural network approach to short-term traffic forecasting

This paper discusses an object-oriented neural network model that was developed for predicting short-term traffic conditions on a section of the Pacific Highway between Brisbane and the Gold Coast in Queensland, Australia. The feasibility of this approach is demonstrated through a time-lag recurrent network (TLRN) which was developed for predicting speed data up to 15 minutes into the future. The results obtained indicate that the TLRN is capable of predicting speed up to 5 minutes into the future with a high degree of accuracy (90-94%). Similar models, which were developed for predicting freeway travel times on the same facility, were successful in predicting travel times up to 15 minutes into the future with a similar degree of accuracy (93-95%). These results represent substantial improvements on conventional model performance and clearly demonstrate the feasibility of using the object-oriented approach for short-term traffic prediction. (C) 2001 Elsevier Science B.V. All rights reserved.

Identifying rate-limiting nodes in large-scale cortical networks for visuospatial processing: an illustration using fMRI

With the advent of functional neuroimaging techniques, in particular functional magnetic resonance imaging (fMRI), we have gained greater insight into the neural correlates of visuospatial function. However, it may not always be easy to identify the cerebral regions most specifically associated with performance on a given task. One approach is to examine the quantitative relationships between regional activation and behavioral performance measures. In the present study, we investigated the functional neuroanatomy of two different visuospatial processing tasks, judgement of line orientation and mental rotation. Twenty-four normal participants were scanned with fMRI using blocked periodic designs for experimental task presentation. Accuracy and reaction time (RT) to each trial of both activation and baseline conditions in each experiment was recorded. Both experiments activated dorsal and ventral visual cortical areas as well as dorsolateral prefrontal cortex. More regionally specific associations with task performance were identified by estimating the association between (sinusoidal) power of functional response and mean RT to the activation condition; a permutation test based on spatial statistics was used for inference. There was significant behavioral-physiological association in right ventral extrastriate cortex for the line orientation task and in bilateral (predominantly right) superior parietal lobule for the mental rotation task. Comparable associations were not found between power of response and RT to the baseline conditions of the tasks. These data suggest that one region in a neurocognitive network may be most strongly associated with behavioral performance and this may be regarded as the computationally least efficient or rate-limiting node of the network.

Mortality and recurrent cardiac events after coronary artery bypass graft: long term outcomes in a population study

Objective: To determine 30 day mortality, long term survival, and recurrent cardiac events after coronary artery bypass graft (CABG) in a population. Design: Follow up study of patients prospectively entered on to a cardiothoracic surgical database. Record linkages were used to obtain data on readmissions and deaths. Patients: 8910 patients undergoing isolated first CABG between 1980 and 1993 in Western Australia. Main outcome measures: 30 day and long term survival, readmission for cardiac event (acute myocardial infarction, unstable angina, percutaneous transluminal coronary angioplasty or reoperative CABG). Results: There were 3072 deaths to mid 1999. 30 day and long term survival were significantly better in patients treated in the first five years than during the following decade. The age of the patients, proportion of female patients, and number of grafts increased over time. An urgent procedure (odds ratio 3.3), older age (9% per year) and female sex (odds ratio 1.5) were associated with increased risk for 30 day mortality, while age (7% per year) and a recent myocardial infarction (odds ratio 1.16) influenced long term survival. Internal mammary artery grafts were followed by better short and long term survival, though there was an obvious selection bias in favour of younger male patients. Conclusions: This study shows worsening crude mortality at 30 days after CABG from the mid 1980s, associated with the inclusion of higher risk patients. Older age, an acute myocardial infarction in the year before surgery, and the use of sephenous vein grafts only were associated with poorer long term survival and greater risk of a recurrent cardiac event. Female sex predicted recurrent events but not long term survival.

Visualization of learning in multilayer perceptron networks using principal component analysis

This paper is concerned with the use of scientific visualization methods for the analysis of feedforward neural networks (NNs). Inevitably, the kinds of data associated with the design and implementation of neural networks are of very high dimensionality, presenting a major challenge for visualization. A method is described using the well-known statistical technique of principal component analysis (PCA). This is found to be an effective and useful method of visualizing the learning trajectories of many learning algorithms such as back-propagation and can also be used to provide insight into the learning process and the nature of the error surface.