Visualization of learning in multilayer perceptron networks using principal component analysis

This paper is concerned with the use of scientific visualization methods for the analysis of feedforward neural networks (NNs). Inevitably, the kinds of data associated with the design and implementation of neural networks are of very high dimensionality, presenting a major challenge for visualization. A method is described using the well-known statistical technique of principal component analysis (PCA). This is found to be an effective and useful method of visualizing the learning trajectories of many learning algorithms such as back-propagation and can also be used to provide insight into the learning process and the nature of the error surface.

A spectral clustering algorithm for manufacturing cell formation

A graph clustering algorithm constructs groups of closely related parts and machines separately. After they are matched for the least intercell moves, a refining process runs on the initial cell formation to decrease the number of intercell moves. A simple modification of this main approach can deal with some practical constraints, such as the popular constraint of bounding the maximum number of machines in a cell. Our approach makes a big improvement in the computational time. More importantly, improvement is seen in the number of intercell moves when the computational results were compared with best known solutions from the literature. (C) 2009 Elsevier Ltd. All rights reserved.

Virtual models of the HLA class I antigen processing pathway

Antigen recognition by cytotoxic CD8 T cells is dependent upon a number of critical steps in MHC class I antigen processing including proteosomal cleavage, TAP transport into the endoplasmic reticulum, and MHC class 1 binding. Based on extensive experimental data relating to each of these steps there is now the capacity to model individual antigen processing steps with a high degree of accuracy. This paper demonstrates the potential to bring together models of individual antigen processing steps, for example proteosome cleavage, TAP transport, and MHC binding, to build highly informative models of functional pathways. In particular, we demonstrate how an artificial neural network model of TAP transport was used to mine a HLA-binding database so as to identify H LA-binding peptides transported by TAP. This integrated model of antigen processing provided the unique insight that HLA class I alleles apparently constitute two separate classes: those that are TAP-efficient for peptide loading (HLA-B27, -A3, and -A24) and those that are TAP-inefficient (HLA-A2, -B7, and -B8). Hence, using this integrated model we were able to generate novel hypotheses regarding antigen processing, and these hypotheses are now capable of being tested experimentally. This model confirms the feasibility of constructing a virtual immune system, whereby each additional step in antigen processing is incorporated into a single modular model. Accurate models of antigen processing have implications for the study of basic immunology as well as for the design of peptide-based vaccines and other immunotherapies. (C) 2004 Elsevier Inc. All rights reserved.

Computational methods for prediction of T-cell epitopes - a framework for modelling, testing, and applications

Computational models complement laboratory experimentation for efficient identification of MHC-binding peptides and T-cell epitopes. Methods for prediction of MHC-binding peptides include binding motifs, quantitative matrices, artificial neural networks, hidden Markov models, and molecular modelling. Models derived by these methods have been successfully used for prediction of T-cell epitopes in cancer, autoimmunity, infectious disease, and allergy. For maximum benefit, the use of computer models must be treated as experiments analogous to standard laboratory procedures and performed according to strict standards. This requires careful selection of data for model building, and adequate testing and validation. A range of web-based databases and MHC-binding prediction programs are available. Although some available prediction programs for particular MHC alleles have reasonable accuracy, there is no guarantee that all models produce good quality predictions. In this article, we present and discuss a framework for modelling, testing, and applications of computational methods used in predictions of T-cell epitopes. (C) 2004 Elsevier Inc. All rights reserved.

Occasion setting in Pavlovian ambiguous target discriminations

Rats were trained in a Pavlovian serial ambiguous target discrimination, in which a target cue was reinforced if it was preceded by one stimulus (P -> T+) but was not reinforced if it was preceded by another stimulus (N -> T-). Test performance indicated that stimulus control by these features was weaker than that acquired by features trained within separate serial feature positive (P -> T+, T-) and serial feature negative (N -> W-, W+) discriminations. The form of conditioned responding and the patterns of transfer observed suggested that the serial ambiguous target discrimination was solved by occasion setting. The data are discussed in terms of the use of retrospective coding strategies when solving Pavlovian serial conditional discriminations, and the acquisition of special properties by both feature and target stimuli. (C) 2008 Published by Elsevier B.V.

Expectation Propagation with Factorizing Distributions: A Gaussian Approximation and Performance Results for Simple Models

We discuss the expectation propagation (EP) algorithm for approximate Bayesian inference using a factorizing posterior approximation. For neural network models, we use a central limit theorem argument to make EP tractable when the number of parameters is large. For two types of models, we show that EP can achieve optimal generalization performance when data are drawn from a simple distribution.

Improving Alzheimer`s Disease Diagnosis with Machine Learning Techniques

There is not a specific test to diagnose Alzheimer`s disease (AD). Its diagnosis should be based upon clinical history, neuropsychological and laboratory tests, neuroimaging and electroencephalography (EEG). Therefore, new approaches are necessary to enable earlier and more accurate diagnosis and to follow treatment results. In this study we used a Machine Learning (ML) technique, named Support Vector Machine (SVM), to search patterns in EEG epochs to differentiate AD patients from controls. As a result, we developed a quantitative EEG (qEEG) processing method for automatic differentiation of patients with AD from normal individuals, as a complement to the diagnosis of probable dementia. We studied EEGs from 19 normal subjects (14 females/5 males, mean age 71.6 years) and 16 probable mild to moderate symptoms AD patients (14 females/2 males, mean age 73.4 years. The results obtained from analysis of EEG epochs were accuracy 79.9% and sensitivity 83.2%. The analysis considering the diagnosis of each individual patient reached 87.0% accuracy and 91.7% sensitivity.

Normal and anomalous classification of category exemplars in schizophrenia

Background Schizophrenia has been associated with semantic memory impairment and previous studies report a difficulty in accessing semantic category exemplars (Moelter et al. 2005 Schizophr Res 78:209–217). The anterior temporal cortex (ATC) has been implicated in the representation of semantic knowledge (Rogers et al. 2004 Psychol Rev 111(1):205–235). We conducted a high-field (4T) fMRI study with the Category Judgment and Substitution Task (CJAST), an analogue of the Hayling test. We hypothesised that differential activation of the temporal lobe would be observed in schizophrenia patients versus controls. Methods Eight schizophrenia patients (7M : 1F) and eight matched controls performed the CJAST, involving a randomised series of 55 common nouns (from five semantic categories) across three conditions: semantic categorisation, anomalous categorisation and word reading. High-resolution 3D T1-weighted images and GE EPI with BOLD contrast and sparse temporal sampling were acquired on a 4T Bruker MedSpec system. Image processing and analyses were performed with SPM2. Results Differential activation in the left ATC was found for anomalous categorisation relative to category judgment, in patients versus controls. Conclusions We examined semantic memory deficits in schizophrenia using a novel fMRI task. Since the ATC corresponds to an area involved in accessing abstract semantic representations (Moelter et al. 2005), these results suggest schizophrenia patients utilise the same neural network as healthy controls, however it is compromised in the patients and the different ATC activity might be attributable to weakening of category-to-category associations.

rTMS treatment for depression in Parkinson`s disease increases BOLD responses in the left prefrontal cortex

The mechanisms underlying the effects of antidepressant treatment in patients with Parkinson`s disease (PD) are unclear. The neural changes after successful therapy investigated by neuroimaging methods can give insights into the mechanisms of action related to a specific treatment choice. To study the mechanisms of neural modulation of repetitive transcranial magnetic Stimulation (rTMS) and fluoxetine, 21 PD depressed patients were randomized into only two active treatment groups for 4 wk: active rTMS over left dorsolateral prefrontal cortex (DLPFC) (5 Hz rTMS; 120% motor threshold) with placebo pill and sham rTMS with fluoxetine 20mg/d. Event-related functional magnetic resonance imaging (fMRI) with emotional stimuli was performed before and after treatment - in two sessions (test and re-test) at each time-point. The two groups of treatment had a significant, similar mood improvement. After rTMS treatment, there were brain activity decreases in left fusiform gyrus, cerebellum and right DLPFC and brain activity increases in left DLPFC and anterior cingulate gyrus compared to baseline. In contrast, after fluoxetine treatment, there were brain activity increases in right premotor and right medial prefrontal cortex. There was a significant interaction effect between groups vs. time in the left medial prefrontal cortex, suggesting that the activity in this area changed differently in the two treatment groups. Our findings show that antidepressant effects of rTMS and fluoxetine in PD are associated with changes in different areas of the depression-related neural network.

Analyzing the connectivity between regions of interest: An approach based on cluster Granger causality for NRI data analysis

The identification, modeling, and analysis of interactions between nodes of neural systems in the human brain have become the aim of interest of many studies in neuroscience. The complex neural network structure and its correlations with brain functions have played a role in all areas of neuroscience, including the comprehension of cognitive and emotional processing. Indeed, understanding how information is stored, retrieved, processed, and transmitted is one of the ultimate challenges in brain research. In this context, in functional neuroimaging, connectivity analysis is a major tool for the exploration and characterization of the information flow between specialized brain regions. In most functional magnetic resonance imaging (fMRI) studies, connectivity analysis is carried out by first selecting regions of interest (ROI) and then calculating an average BOLD time series (across the voxels in each cluster). Some studies have shown that the average may not be a good choice and have suggested, as an alternative, the use of principal component analysis (PCA) to extract the principal eigen-time series from the ROI(s). In this paper, we introduce a novel approach called cluster Granger analysis (CGA) to study connectivity between ROIs. The main aim of this method was to employ multiple eigen-time series in each ROI to avoid temporal information loss during identification of Granger causality. Such information loss is inherent in averaging (e.g., to yield a single ""representative"" time series per ROI). This, in turn, may lead to a lack of power in detecting connections. The proposed approach is based on multivariate statistical analysis and integrates PCA and partial canonical correlation in a framework of Granger causality for clusters (sets) of time series. We also describe an algorithm for statistical significance testing based on bootstrapping. By using Monte Carlo simulations, we show that the proposed approach outperforms conventional Granger causality analysis (i.e., using representative time series extracted by signal averaging or first principal components estimation from ROIs). The usefulness of the CGA approach in real fMRI data is illustrated in an experiment using human faces expressing emotions. With this data set, the proposed approach suggested the presence of significantly more connections between the ROIs than were detected using a single representative time series in each ROI. (c) 2010 Elsevier Inc. All rights reserved.

Generation and phenotypic characterization of new human ovarian cancer cell lines with the identification of antigens potentially recognizable by HLA-restricted cytotoxic T cells

This study describes a simple method for long-term establishment of human ovarian tumor lines and prediction of T-cell epitopes that could be potentially useful in the generation of tumor-specific cytotoxic T lymphocytes (CTLs), Nine ovarian tumor lines (INT.Ov) were generated from solid primary or metastatic tumors as well as from ascitic fluid, Notably all lines expressed HLA class I, intercellular adhesion molecule-1 (ICAM-1), polymorphic epithelial mucin (PEM) and cytokeratin (CK), but not HLA class II, B7.1 (CD80) or BAGE, While of the 9 lines tested 4 (INT.Ov1, 2, 5 and 6) expressed the folate receptor (FR-alpha) and 6 (INT.Ov1, 2, 5, 6, 7 and 9) expressed the epidermal growth factor receptor (EGFR); MAGE-1 and p185(HER-2/neu) were only found in 2 lines (INT.Ov1 and 2) and GAGE-1 expression in 1 line (INT.Ov2). The identification of class I MHC ligands and T-cell epitopes within protein antigens was achieved by applying several theoretical methods including: 1) similarity or homology searches to MHCPEP; 2) BIMAS and 3) artificial neural network-based predictions of proteins MACE, GAGE, EGFR, p185(HER-2/neu) and FR-alpha expressed in INT.Ov lines, Because of the high frequency of expression of some of these proteins in ovarian cancer and the ability to determine HLA binding peptides efficiently, it is expected that after appropriate screening, a large cohort of ovarian cancer patients may become candidates to receive peptide based vaccines. (C) 1997 Wiley-Liss, Inc.

A decision support system to facilitate management of patients with acute gastrointestinal bleeding

Objective: To develop a model to predict the bleeding source and identify the cohort amongst patients with acute gastrointestinal bleeding (GIB) who require urgent intervention, including endoscopy. Patients with acute GIB, an unpredictable event, are most commonly evaluated and managed by non-gastroenterologists. Rapid and consistently reliable risk stratification of patients with acute GIB for urgent endoscopy may potentially improve outcomes amongst such patients by targeting scarce health-care resources to those who need it the most. Design and methods: Using ICD-9 codes for acute GIB, 189 patients with acute GIB and all. available data variables required to develop and test models were identified from a hospital medical records database. Data on 122 patients was utilized for development of the model and on 67 patients utilized to perform comparative analysis of the models. Clinical data such as presenting signs and symptoms, demographic data, presence of co-morbidities, laboratory data and corresponding endoscopic diagnosis and outcomes were collected. Clinical data and endoscopic diagnosis collected for each patient was utilized to retrospectively ascertain optimal management for each patient. Clinical presentations and corresponding treatment was utilized as training examples. Eight mathematical models including artificial neural network (ANN), support vector machine (SVM), k-nearest neighbor, linear discriminant analysis (LDA), shrunken centroid (SC), random forest (RF), logistic regression, and boosting were trained and tested. The performance of these models was compared using standard statistical analysis and ROC curves. Results: Overall the random forest model best predicted the source, need for resuscitation, and disposition with accuracies of approximately 80% or higher (accuracy for endoscopy was greater than 75%). The area under ROC curve for RF was greater than 0.85, indicating excellent performance by the random forest model Conclusion: While most mathematical models are effective as a decision support system for evaluation and management of patients with acute GIB, in our testing, the RF model consistently demonstrated the best performance. Amongst patients presenting with acute GIB, mathematical models may facilitate the identification of the source of GIB, need for intervention and allow optimization of care and healthcare resource allocation; these however require further validation. (c) 2007 Elsevier B.V. All rights reserved.

A Dynamical Modeling to Study the Adaptive Immune System and the Influence of Antibodies in the Immune Memory

Immunological systems have been an abundant inspiration to contemporary computer scientists. Problem solving strategies, stemming from known immune system phenomena, have been successfully applied to chall enging problems of modem computing. Simulation systems and mathematical modeling are also beginning use to answer more complex immunological questions as immune memory process and duration of vaccines, where the regulation mechanisms are not still known sufficiently (Lundegaard, Lund, Kesmir, Brunak, Nielsen, 2007). In this article we studied in machina a approach to simulate the process of antigenic mutation and its implications for the process of memory. Our results have suggested that the durability of the immune memory is affected by the process of antigenic mutation.and by populations of soluble antibodies in the blood. The results also strongly suggest that the decrease of the production of antibodies favors the global maintenance of immune memory.

Is dystonic posturing during temporal lobe epileptic seizures the expression of an endogenous anticonvulsant system?

In temporal lobe epilepsy (TLE) seizures, tonic or clonic motor behaviors (TCB) are commonly associated with automatisms, versions, and vocalizations, and frequently occur during secondary generalization. Dystonias are a common finding and appear to be associated with automatisms and head deviation, but have never been directly linked to generalized tonic or clonic behaviors. The objective of the present study was to assess whether dystonias and TCB are coupled in the same seizure or are associated in an antagonistic and exclusive pattern. Ninety-one seizures in 55 patients with TLE due to mesial temporal sclerosis were analyzed. Only patients with postsurgical seizure outcome of Engel class I or II were included. Presence or absence of dystonia and secondary generalization was recorded. Occurrence of dystonia and occurrence of bilateral tonic or clonic behaviors were negatively correlated. Dystonia and TCB may be implicated in exclusive, non-coincidental, or even antagonistic effects or phenomena in TLE seizures. A neural network related to the expression of one behavioral response (e.g., basal ganglia activation and dystonia) might theoretically ""displace"" brain activation or disrupt the synchronism of another network implicated in pathological circuit reverberation and seizure expression. The involvement of basal ganglia in the blockade of convulsive seizures has long been observed in animal models. The question is: Do dystonia and underlying basal ganglia activation represent an attempt of the brain to block imminent secondary generalization? (C) 2007 Elsevier Inc. All rights reserved.

Empirical evidence for ultrametric structure in multi-layer perceptron error surfaces

Combinatorial optimization problems share an interesting property with spin glass systems in that their state spaces can exhibit ultrametric structure. We use sampling methods to analyse the error surfaces of feedforward multi-layer perceptron neural networks learning encoder problems. The third order statistics of these points of attraction are examined and found to be arranged in a highly ultrametric way. This is a unique result for a finite, continuous parameter space. The implications of this result are discussed.