Eliminating Redundant Training Data Using Unsupervised Clustering Techniques

Training data for supervised learning neural networks can be clustered such that the input/output pairs in each cluster are redundant. Redundant training data can adversely affect training time. In this paper we apply two clustering algorithms, ART2 -A and the Generalized Equality Classifier, to identify training data clusters and thus reduce the training data and training time. The approach is demonstrated for a high dimensional nonlinear continuous time mapping. The demonstration shows six-fold decrease in training time at little or no loss of accuracy in the handling of evaluation data.

Neural Pattern Recognition on Multichannel Input Representation

This article presents a new neural pattern recognition architecture on multichannel data representation. The architecture emploies generalized ART modules as building blocks to construct a supervised learning system generating recognition codes on channels dynamically selected in context using serial and parallel match trackings led by inter-ART vigilance signals.

ART-EMAP: A Neural Network Architecture for Object Recognition by Evidence Accumulation

A new neural network architecture is introduced for the recognition of pattern classes after supervised and unsupervised learning. Applications include spatio-temporal image understanding and prediction and 3-D object recognition from a series of ambiguous 2-D views. The architecture, called ART-EMAP, achieves a synthesis of adaptive resonance theory (ART) and spatial and temporal evidence integration for dynamic predictive mapping (EMAP). ART-EMAP extends the capabilities of fuzzy ARTMAP in four incremental stages. Stage 1 introduces distributed pattern representation at a view category field. Stage 2 adds a decision criterion to the mapping between view and object categories, delaying identification of ambiguous objects when faced with a low confidence prediction. Stage 3 augments the system with a field where evidence accumulates in medium-term memory (MTM). Stage 4 adds an unsupervised learning process to fine-tune performance after the limited initial period of supervised network training. Each ART-EMAP stage is illustrated with a benchmark simulation example, using both noisy and noise-free data. A concluding set of simulations demonstrate ART-EMAP performance on a difficult 3-D object recognition problem.

Fast Learning VIEWNET Architectures for Recognizing 3-D Objects from Multiple 2-D Views

The recognition of 3-D objects from sequences of their 2-D views is modeled by a family of self-organizing neural architectures, called VIEWNET, that use View Information Encoded With NETworks. VIEWNET incorporates a preprocessor that generates a compressed but 2-D invariant representation of an image, a supervised incremental learning system that classifies the preprocessed representations into 2-D view categories whose outputs arc combined into 3-D invariant object categories, and a working memory that makes a 3-D object prediction by accumulating evidence from 3-D object category nodes as multiple 2-D views are experienced. The simplest VIEWNET achieves high recognition scores without the need to explicitly code the temporal order of 2-D views in working memory. Working memories are also discussed that save memory resources by implicitly coding temporal order in terms of the relative activity of 2-D view category nodes, rather than as explicit 2-D view transitions. Variants of the VIEWNET architecture may also be used for scene understanding by using a preprocessor and classifier that can determine both What objects are in a scene and Where they are located. The present VIEWNET preprocessor includes the CORT-X 2 filter, which discounts the illuminant, regularizes and completes figural boundaries, and suppresses image noise. This boundary segmentation is rendered invariant under 2-D translation, rotation, and dilation by use of a log-polar transform. The invariant spectra undergo Gaussian coarse coding to further reduce noise and 3-D foreshortening effects, and to increase generalization. These compressed codes are input into the classifier, a supervised learning system based on the fuzzy ARTMAP algorithm. Fuzzy ARTMAP learns 2-D view categories that are invariant under 2-D image translation, rotation, and dilation as well as 3-D image transformations that do not cause a predictive error. Evidence from sequence of 2-D view categories converges at 3-D object nodes that generate a response invariant under changes of 2-D view. These 3-D object nodes input to a working memory that accumulates evidence over time to improve object recognition. ln the simplest working memory, each occurrence (nonoccurrence) of a 2-D view category increases (decreases) the corresponding node's activity in working memory. The maximally active node is used to predict the 3-D object. Recognition is studied with noisy and clean image using slow and fast learning. Slow learning at the fuzzy ARTMAP map field is adapted to learn the conditional probability of the 3-D object given the selected 2-D view category. VIEWNET is demonstrated on an MIT Lincoln Laboratory database of l28x128 2-D views of aircraft with and without additive noise. A recognition rate of up to 90% is achieved with one 2-D view and of up to 98.5% correct with three 2-D views. The properties of 2-D view and 3-D object category nodes are compared with those of cells in monkey inferotemporal cortex.

Fuzzy ART

Adaptive Resonance Theory (ART) models are real-time neural networks for category learning, pattern recognition, and prediction. Unsupervised fuzzy ART and supervised fuzzy ARTMAP synthesize fuzzy logic and ART networks by exploiting the formal similarity between the computations of fuzzy subsethood and the dynamics of ART category choice, search, and learning. Fuzzy ART self-organizes stable recognition categories in response to arbitrary sequences of analog or binary input patterns. It generalizes the binary ART 1 model, replacing the set-theoretic: intersection (∩) with the fuzzy intersection (∧), or component-wise minimum. A normalization procedure called complement coding leads to a symmetric: theory in which the fuzzy inter:>ec:tion and the fuzzy union (∨), or component-wise maximum, play complementary roles. Complement coding preserves individual feature amplitudes while normalizing the input vector, and prevents a potential category proliferation problem. Adaptive weights :otart equal to one and can only decrease in time. A geometric interpretation of fuzzy AHT represents each category as a box that increases in size as weights decrease. A matching criterion controls search, determining how close an input and a learned representation must be for a category to accept the input as a new exemplar. A vigilance parameter (p) sets the matching criterion and determines how finely or coarsely an ART system will partition inputs. High vigilance creates fine categories, represented by small boxes. Learning stops when boxes cover the input space. With fast learning, fixed vigilance, and an arbitrary input set, learning stabilizes after just one presentation of each input. A fast-commit slow-recode option allows rapid learning of rare events yet buffers memories against recoding by noisy inputs. Fuzzy ARTMAP unites two fuzzy ART networks to solve supervised learning and prediction problems. A Minimax Learning Rule controls ARTMAP category structure, conjointly minimizing predictive error and maximizing code compression. Low vigilance maximizes compression but may therefore cause very different inputs to make the same prediction. When this coarse grouping strategy causes a predictive error, an internal match tracking control process increases vigilance just enough to correct the error. ARTMAP automatically constructs a minimal number of recognition categories, or "hidden units," to meet accuracy criteria. An ARTMAP voting strategy improves prediction by training the system several times using different orderings of the input set. Voting assigns confidence estimates to competing predictions given small, noisy, or incomplete training sets. ARPA benchmark simulations illustrate fuzzy ARTMAP dynamics. The chapter also compares fuzzy ARTMAP to Salzberg's Nested Generalized Exemplar (NGE) and to Simpson's Fuzzy Min-Max Classifier (FMMC); and concludes with a summary of ART and ARTMAP applications.

Fuzzy ART Choice Functions

Adaptive Resonance Theory (ART) models are real-time neural networks for category learning, pattern recognition, and prediction. Unsupervised fuzzy ART and supervised fuzzy ARTMAP networks synthesize fuzzy logic and ART by exploiting the formal similarity between tile computations of fuzzy subsethood and the dynamics of ART category choice, search, and learning. Fuzzy ART self-organizes stable recognition categories in response to arbitrary sequences of analog or binary input patterns. It generalizes the binary ART 1 model, replacing the set-theoretic intersection (∩) with the fuzzy intersection(∧), or component-wise minimum. A normalization procedure called complement coding leads to a symmetric theory in which the fuzzy intersection and the fuzzy union (∨), or component-wise maximum, play complementary roles. A geometric interpretation of fuzzy ART represents each category as a box that increases in size as weights decrease. This paper analyzes fuzzy ART models that employ various choice functions for category selection. One such function minimizes total weight change during learning. Benchmark simulations compare peformance of fuzzy ARTMAP systems that use different choice functions.

3-D Object Recognition by the ART-EMAP Evidence Accumulation Network

ART-EMAP synthesizes adaptive resonance theory (AHT) and spatial and temporal evidence integration for dynamic predictive mapping (EMAP). The network extends the capabilities of fuzzy ARTMAP in four incremental stages. Stage I introduces distributed pattern representation at a view category field. Stage 2 adds a decision criterion to the mapping between view and object categories, delaying identification of ambiguous objects when faced with a low confidence prediction. Stage 3 augments the system with a field where evidence accumulates in medium-term memory (MTM). Stage 4 adds an unsupervised learning process to fine-tune performance after the limited initial period of supervised network training. Simulations of the four ART-EMAP stages demonstrate performance on a difficult 3-D object recognition problem.

Vector Associative Maps: Unsupervised Real-time Error-based Learning and Control of Movement Trajectories

This article describes neural network models for adaptive control of arm movement trajectories during visually guided reaching and, more generally, a framework for unsupervised real-time error-based learning. The models clarify how a child, or untrained robot, can learn to reach for objects that it sees. Piaget has provided basic insights with his concept of a circular reaction: As an infant makes internally generated movements of its hand, the eyes automatically follow this motion. A transformation is learned between the visual representation of hand position and the motor representation of hand position. Learning of this transformation eventually enables the child to accurately reach for visually detected targets. Grossberg and Kuperstein have shown how the eye movement system can use visual error signals to correct movement parameters via cerebellar learning. Here it is shown how endogenously generated arm movements lead to adaptive tuning of arm control parameters. These movements also activate the target position representations that are used to learn the visuo-motor transformation that controls visually guided reaching. The AVITE model presented here is an adaptive neural circuit based on the Vector Integration to Endpoint (VITE) model for arm and speech trajectory generation of Bullock and Grossberg. In the VITE model, a Target Position Command (TPC) represents the location of the desired target. The Present Position Command (PPC) encodes the present hand-arm configuration. The Difference Vector (DV) population continuously.computes the difference between the PPC and the TPC. A speed-controlling GO signal multiplies DV output. The PPC integrates the (DV)·(GO) product and generates an outflow command to the arm. Integration at the PPC continues at a rate dependent on GO signal size until the DV reaches zero, at which time the PPC equals the TPC. The AVITE model explains how self-consistent TPC and PPC coordinates are autonomously generated and learned. Learning of AVITE parameters is regulated by activation of a self-regulating Endogenous Random Generator (ERG) of training vectors. Each vector is integrated at the PPC, giving rise to a movement command. The generation of each vector induces a complementary postural phase during which ERG output stops and learning occurs. Then a new vector is generated and the cycle is repeated. This cyclic, biphasic behavior is controlled by a specialized gated dipole circuit. ERG output autonomously stops in such a way that, across trials, a broad sample of workspace target positions is generated. When the ERG shuts off, a modulator gate opens, copying the PPC into the TPC. Learning of a transformation from TPC to PPC occurs using the DV as an error signal that is zeroed due to learning. This learning scheme is called a Vector Associative Map, or VAM. The VAM model is a general-purpose device for autonomous real-time error-based learning and performance of associative maps. The DV stage serves the dual function of reading out new TPCs during performance and reading in new adaptive weights during learning, without a disruption of real-time operation. YAMs thus provide an on-line unsupervised alternative to the off-line properties of supervised error-correction learning algorithms. YAMs and VAM cascades for learning motor-to-motor and spatial-to-motor maps are described. YAM models and Adaptive Resonance Theory (ART) models exhibit complementary matching, learning, and performance properties that together provide a foundation for designing a total sensory-cognitive and cognitive-motor autonomous system.

Multi-sensor classification of tennis strokes

In this work, we investigate tennis stroke recognition using a single inertial measuring unit attached to a player’s forearm during a competitive match. This paper evaluates the best approach for stroke detection using either accelerometers, gyroscopes or magnetometers, which are embedded into the inertial measuring unit. This work concludes what is the optimal training data set for stroke classification and proves that classifiers can perform well when tested on players who were not used to train the classifier. This work provides a significant step forward for our overall goal, which is to develop next generation sports coaching tools using both inertial and visual sensors in an instrumented indoor sporting environment.

Leisure and surveillance in Irish childhoods: A cross-class study of Cork city

This research provides an interpretive cross-class analysis of the leisure experience of children, aged between six and ten years, living in Cork city. This study focuses on the cultural dispositions underpinning parental decisions in relation to children’s leisure activities, with a particular emphasis on their child-surveillance practices. In this research, child-surveillance is defined as the adult monitoring of children by technological means, physical supervision, community supervision, or adult supervised activities (Nelson, 2010; Lareau, 2003; Fotel and Thomsen, 2004). This research adds significantly to understandings of Irish childhood by providing the first in-depth qualitative analysis of the surveillance of children’s leisure-time. Since the 1990s, international research on children has highlighted the increasingly structured nature of children’s leisure-time (Lareau, 2011; Valentine & McKendrick, 1997). Furthermore, research on child-surveillance has found an increase in the intensive supervision of children during their unstructured leisure-time (Nelson, 2010; Furedi, 2008; Fotel and Thomsen, 2004). This research bridges the gap between these two key bodies of literature, providing a more integrated overview of children’s experience of leisure in Ireland. Using Bourdieu’s (1992) model of habitus, field and capital, the dispositions that shape parents’ decisions about their children’s leisure time are interrogated. The holistic view of childhood adopted in this research echoes the ‘Whole Child Approach’ by analysing the child’s experience within a wider set of social relationships including family, school, and community. Underpinned by James and Prout’s (1990) paradigm on childhood, this study considers Irish children’s agency in negotiating with parents’ decisions regarding leisure-time. The data collated in this study enhances our understanding of the micro-interactions between parents and children and, the ability of the child to shape their own experience. Moreover, this is the first Irish sociological research to identify and discuss class distinctions in children’s agentic potential during leisure-time.

Neonatal EEG classification using atomic decomposition

The electroencephalogram (EEG) is an important noninvasive tool used in the neonatal intensive care unit (NICU) for the neurologic evaluation of the sick newborn infant. It provides an excellent assessment of at-risk newborns and formulates a prognosis for long-term neurologic outcome.The automated analysis of neonatal EEG data in the NICU can provide valuable information to the clinician facilitating medical intervention. The aim of this thesis is to develop a system for automatic classification of neonatal EEG which can be mainly divided into two parts: (1) classification of neonatal EEG seizure from nonseizure, and (2) classifying neonatal background EEG into several grades based on the severity of the injury using atomic decomposition. Atomic decomposition techniques use redundant time-frequency dictionaries for sparse signal representations or approximations. The first novel contribution of this thesis is the development of a novel time-frequency dictionary coherent with the neonatal EEG seizure states. This dictionary was able to track the time-varying nature of the EEG signal. It was shown that by using atomic decomposition and the proposed novel dictionary, the neonatal EEG transition from nonseizure to seizure states could be detected efficiently. The second novel contribution of this thesis is the development of a neonatal seizure detection algorithm using several time-frequency features from the proposed novel dictionary. It was shown that the time-frequency features obtained from the atoms in the novel dictionary improved the seizure detection accuracy when compared to that obtained from the raw EEG signal. With the assistance of a supervised multiclass SVM classifier and several timefrequency features, several methods to automatically grade EEG were explored. In summary, the novel techniques proposed in this thesis contribute to the application of advanced signal processing techniques for automatic assessment of neonatal EEG recordings.

Optimized approach to decision fusion of heterogeneous data for breast cancer diagnosis.

As more diagnostic testing options become available to physicians, it becomes more difficult to combine various types of medical information together in order to optimize the overall diagnosis. To improve diagnostic performance, here we introduce an approach to optimize a decision-fusion technique to combine heterogeneous information, such as from different modalities, feature categories, or institutions. For classifier comparison we used two performance metrics: The receiving operator characteristic (ROC) area under the curve [area under the ROC curve (AUC)] and the normalized partial area under the curve (pAUC). This study used four classifiers: Linear discriminant analysis (LDA), artificial neural network (ANN), and two variants of our decision-fusion technique, AUC-optimized (DF-A) and pAUC-optimized (DF-P) decision fusion. We applied each of these classifiers with 100-fold cross-validation to two heterogeneous breast cancer data sets: One of mass lesion features and a much more challenging one of microcalcification lesion features. For the calcification data set, DF-A outperformed the other classifiers in terms of AUC (p < 0.02) and achieved AUC=0.85 +/- 0.01. The DF-P surpassed the other classifiers in terms of pAUC (p < 0.01) and reached pAUC=0.38 +/- 0.02. For the mass data set, DF-A outperformed both the ANN and the LDA (p < 0.04) and achieved AUC=0.94 +/- 0.01. Although for this data set there were no statistically significant differences among the classifiers' pAUC values (pAUC=0.57 +/- 0.07 to 0.67 +/- 0.05, p > 0.10), the DF-P did significantly improve specificity versus the LDA at both 98% and 100% sensitivity (p < 0.04). In conclusion, decision fusion directly optimized clinically significant performance measures, such as AUC and pAUC, and sometimes outperformed two well-known machine-learning techniques when applied to two different breast cancer data sets.

The lung cancer exercise training study: a randomized trial of aerobic training, resistance training, or both in postsurgical lung cancer patients: rationale and design.

BACKGROUND: The Lung Cancer Exercise Training Study (LUNGEVITY) is a randomized trial to investigate the efficacy of different types of exercise training on cardiorespiratory fitness (VO2peak), patient-reported outcomes, and the organ components that govern VO2peak in post-operative non-small cell lung cancer (NSCLC) patients. METHODS/DESIGN: Using a single-center, randomized design, 160 subjects (40 patients/study arm) with histologically confirmed stage I-IIIA NSCLC following curative-intent complete surgical resection at Duke University Medical Center (DUMC) will be potentially eligible for this trial. Following baseline assessments, eligible participants will be randomly assigned to one of four conditions: (1) aerobic training alone, (2) resistance training alone, (3) the combination of aerobic and resistance training, or (4) attention-control (progressive stretching). The ultimate goal for all exercise training groups will be 3 supervised exercise sessions per week an intensity above 70% of the individually determined VO2peak for aerobic training and an intensity between 60 and 80% of one-repetition maximum for resistance training, for 30-45 minutes/session. Progressive stretching will be matched to the exercise groups in terms of program length (i.e., 16 weeks), social interaction (participants will receive one-on-one instruction), and duration (30-45 mins/session). The primary study endpoint is VO2peak. Secondary endpoints include: patient-reported outcomes (PROs) (e.g., quality of life, fatigue, depression, etc.) and organ components of the oxygen cascade (i.e., pulmonary function, cardiac function, skeletal muscle function). All endpoints will be assessed at baseline and postintervention (16 weeks). Substudies will include genetic studies regarding individual responses to an exercise stimulus, theoretical determinants of exercise adherence, examination of the psychological mediators of the exercise - PRO relationship, and exercise-induced changes in gene expression. DISCUSSION: VO2peak is becoming increasingly recognized as an outcome of major importance in NSCLC. LUNGEVITY will identify the optimal form of exercise training for NSCLC survivors as well as provide insight into the physiological mechanisms underlying this effect. Overall, this study will contribute to the establishment of clinical exercise therapy rehabilitation guidelines for patients across the entire NSCLC continuum. TRIAL REGISTRATION: NCT00018255.

Rationale and design of the Exercise Intensity Trial (EXCITE): A randomized trial comparing the effects of moderate versus moderate to high-intensity aerobic training in women with operable breast cancer.

BACKGROUND: The Exercise Intensity Trial (EXcITe) is a randomized trial to compare the efficacy of supervised moderate-intensity aerobic training to moderate to high-intensity aerobic training, relative to attention control, on aerobic capacity, physiologic mechanisms, patient-reported outcomes, and biomarkers in women with operable breast cancer following the completion of definitive adjuvant therapy. METHODS/DESIGN: Using a single-center, randomized design, 174 postmenopausal women (58 patients/study arm) with histologically confirmed, operable breast cancer presenting to Duke University Medical Center (DUMC) will be enrolled in this trial following completion of primary therapy (including surgery, radiation therapy, and chemotherapy). After baseline assessments, eligible participants will be randomized to one of two supervised aerobic training interventions (moderate-intensity or moderate/high-intensity aerobic training) or an attention-control group (progressive stretching). The aerobic training interventions will include 150 mins.wk⁻¹ of supervised treadmill walking per week at an intensity of 60%-70% (moderate-intensity) or 60% to 100% (moderate to high-intensity) of the individually determined peak oxygen consumption (VO₂peak) between 20-45 minutes/session for 16 weeks. The progressive stretching program will be consistent with the exercise interventions in terms of program length (16 weeks), social interaction (participants will receive one-on-one instruction), and duration (20-45 mins/session). The primary study endpoint is VO₂peak, as measured by an incremental cardiopulmonary exercise test. Secondary endpoints include physiologic determinants that govern VO₂peak, patient-reported outcomes, and biomarkers associated with breast cancer recurrence/mortality. All endpoints will be assessed at baseline and after the intervention (16 weeks). DISCUSSION: EXCITE is designed to investigate the intensity of aerobic training required to induce optimal improvements in VO₂peak and other pertinent outcomes in women who have completed definitive adjuvant therapy for operable breast cancer. Overall, this trial will inform and refine exercise guidelines to optimize recovery in breast and other cancer survivors following the completion of primary cytotoxic therapy. TRIAL REGISTRATION: NCT01186367.

An active learning approach for rapid characterization of endothelial cells in human tumors.

Currently, no available pathological or molecular measures of tumor angiogenesis predict response to antiangiogenic therapies used in clinical practice. Recognizing that tumor endothelial cells (EC) and EC activation and survival signaling are the direct targets of these therapies, we sought to develop an automated platform for quantifying activity of critical signaling pathways and other biological events in EC of patient tumors by histopathology. Computer image analysis of EC in highly heterogeneous human tumors by a statistical classifier trained using examples selected by human experts performed poorly due to subjectivity and selection bias. We hypothesized that the analysis can be optimized by a more active process to aid experts in identifying informative training examples. To test this hypothesis, we incorporated a novel active learning (AL) algorithm into FARSIGHT image analysis software that aids the expert by seeking out informative examples for the operator to label. The resulting FARSIGHT-AL system identified EC with specificity and sensitivity consistently greater than 0.9 and outperformed traditional supervised classification algorithms. The system modeled individual operator preferences and generated reproducible results. Using the results of EC classification, we also quantified proliferation (Ki67) and activity in important signal transduction pathways (MAP kinase, STAT3) in immunostained human clear cell renal cell carcinoma and other tumors. FARSIGHT-AL enables characterization of EC in conventionally preserved human tumors in a more automated process suitable for testing and validating in clinical trials. The results of our study support a unique opportunity for quantifying angiogenesis in a manner that can now be tested for its ability to identify novel predictive and response biomarkers.