Fuzzy ART

Adaptive Resonance Theory (ART) models are real-time neural networks for category learning, pattern recognition, and prediction. Unsupervised fuzzy ART and supervised fuzzy ARTMAP synthesize fuzzy logic and ART networks by exploiting the formal similarity between the computations of fuzzy subsethood and the dynamics of ART category choice, search, and learning. Fuzzy ART self-organizes stable recognition categories in response to arbitrary sequences of analog or binary input patterns. It generalizes the binary ART 1 model, replacing the set-theoretic: intersection (∩) with the fuzzy intersection (∧), or component-wise minimum. A normalization procedure called complement coding leads to a symmetric: theory in which the fuzzy inter:>ec:tion and the fuzzy union (∨), or component-wise maximum, play complementary roles. Complement coding preserves individual feature amplitudes while normalizing the input vector, and prevents a potential category proliferation problem. Adaptive weights :otart equal to one and can only decrease in time. A geometric interpretation of fuzzy AHT represents each category as a box that increases in size as weights decrease. A matching criterion controls search, determining how close an input and a learned representation must be for a category to accept the input as a new exemplar. A vigilance parameter (p) sets the matching criterion and determines how finely or coarsely an ART system will partition inputs. High vigilance creates fine categories, represented by small boxes. Learning stops when boxes cover the input space. With fast learning, fixed vigilance, and an arbitrary input set, learning stabilizes after just one presentation of each input. A fast-commit slow-recode option allows rapid learning of rare events yet buffers memories against recoding by noisy inputs. Fuzzy ARTMAP unites two fuzzy ART networks to solve supervised learning and prediction problems. A Minimax Learning Rule controls ARTMAP category structure, conjointly minimizing predictive error and maximizing code compression. Low vigilance maximizes compression but may therefore cause very different inputs to make the same prediction. When this coarse grouping strategy causes a predictive error, an internal match tracking control process increases vigilance just enough to correct the error. ARTMAP automatically constructs a minimal number of recognition categories, or "hidden units," to meet accuracy criteria. An ARTMAP voting strategy improves prediction by training the system several times using different orderings of the input set. Voting assigns confidence estimates to competing predictions given small, noisy, or incomplete training sets. ARPA benchmark simulations illustrate fuzzy ARTMAP dynamics. The chapter also compares fuzzy ARTMAP to Salzberg's Nested Generalized Exemplar (NGE) and to Simpson's Fuzzy Min-Max Classifier (FMMC); and concludes with a summary of ART and ARTMAP applications.

Fuzzy ART Choice Functions

Adaptive Resonance Theory (ART) models are real-time neural networks for category learning, pattern recognition, and prediction. Unsupervised fuzzy ART and supervised fuzzy ARTMAP networks synthesize fuzzy logic and ART by exploiting the formal similarity between tile computations of fuzzy subsethood and the dynamics of ART category choice, search, and learning. Fuzzy ART self-organizes stable recognition categories in response to arbitrary sequences of analog or binary input patterns. It generalizes the binary ART 1 model, replacing the set-theoretic intersection (∩) with the fuzzy intersection(∧), or component-wise minimum. A normalization procedure called complement coding leads to a symmetric theory in which the fuzzy intersection and the fuzzy union (∨), or component-wise maximum, play complementary roles. A geometric interpretation of fuzzy ART represents each category as a box that increases in size as weights decrease. This paper analyzes fuzzy ART models that employ various choice functions for category selection. One such function minimizes total weight change during learning. Benchmark simulations compare peformance of fuzzy ARTMAP systems that use different choice functions.

A Neural Network Architecture for Figure-ground Separation of Connected Scenic Figures

A neural network model, called an FBF network, is proposed for automatic parallel separation of multiple image figures from each other and their backgrounds in noisy grayscale or multi-colored images. The figures can then be processed in parallel by an array of self-organizing Adaptive Resonance Theory (ART) neural networks for automatic target recognition. An FBF network can automatically separate the disconnected but interleaved spirals that Minsky and Papert introduced in their book Perceptrons. The network's design also clarifies why humans cannot rapidly separate interleaved spirals, yet can rapidly detect conjunctions of disparity and color, or of disparity and motion, that distinguish target figures from surrounding distractors. Figure-ground separation is accomplished by iterating operations of a Feature Contour System (FCS) and a Boundary Contour System (BCS) in the order FCS-BCS-FCS, hence the term FBF, that have been derived from an analysis of biological vision. The FCS operations include the use of nonlinear shunting networks to compensate for variable illumination and nonlinear diffusion networks to control filling-in. A key new feature of an FBF network is the use of filling-in for figure-ground separation. The BCS operations include oriented filters joined to competitive and cooperative interactions designed to detect, regularize, and complete boundaries in up to 50 percent noise, while suppressing the noise. A modified CORT-X filter is described which uses both on-cells and off-cells to generate a boundary segmentation from a noisy image.

Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System

A Fuzzy ART model capable of rapid stable learning of recognition categories in response to arbitrary sequences of analog or binary input patterns is described. Fuzzy ART incorporates computations from fuzzy set theory into the ART 1 neural network, which learns to categorize only binary input patterns. The generalization to learning both analog and binary input patterns is achieved by replacing appearances of the intersection operator (n) in AHT 1 by the MIN operator (Λ) of fuzzy set theory. The MIN operator reduces to the intersection operator in the binary case. Category proliferation is prevented by normalizing input vectors at a preprocessing stage. A normalization procedure called complement coding leads to a symmetric theory in which the MIN operator (Λ) and the MAX operator (v) of fuzzy set theory play complementary roles. Complement coding uses on-cells and off-cells to represent the input pattern, and preserves individual feature amplitudes while normalizing the total on-cell/off-cell vector. Learning is stable because all adaptive weights can only decrease in time. Decreasing weights correspond to increasing sizes of category "boxes". Smaller vigilance values lead to larger category boxes. Learning stops when the input space is covered by boxes. With fast learning and a finite input set of arbitrary size and composition, learning stabilizes after just one presentation of each input pattern. A fast-commit slow-recode option combines fast learning with a forgetting rule that buffers system memory against noise. Using this option, rare events can be rapidly learned, yet previously learned memories are not rapidly erased in response to statistically unreliable input fluctuations.

A Neural Network Realization of Fuzzy ART

A neural network realization of the fuzzy Adaptive Resonance Theory (ART) algorithm is described. Fuzzy ART is capable of rapid stable learning of recognition categories in response to arbitrary sequences of analog or binary input patterns. Fuzzy ART incorporates computations from fuzzy set theory into the ART 1 neural network, which learns to categorize only binary input patterns, thus enabling the network to learn both analog and binary input patterns. In the neural network realization of fuzzy ART, signal transduction obeys a path capacity rule. Category choice is determined by a combination of bottom-up signals and learned category biases. Top-down signals impose upper bounds on feature node activations.

A What-and-Where Neural Network for Invariant Image Preprocessing

A feedforward neural network for invariant image preprocessing is proposed that represents the position1 orientation and size of an image figure (where it is) in a multiplexed spatial map. This map is used to generate an invariant representation of the figure that is insensitive to position1 orientation, and size for purposes of pattern recognition (what it is). A multiscale array of oriented filters followed by competition between orientations and scales is used to define the Where filter.

Use of semiconductor optical amplifiers in signal processing applications

We describe a 42.6 Gbit/s all-optical pattern recognition system which uses semiconductor optical amplifiers (SOAs). A circuit with three SOA-based logic gates is used to identify the presence of specific port numbers in an optical packet header.

Identification of novel innate immune mechanisms regulating inflammation in the gastrointestinal tract

The Gastro-Intestinal (GI) tract is a unique region in the body. Our innate immune system retains a fine homeostatic balance between avoiding inappropriate inflammatory responses against the myriad commensal microbes residing in the gut while also remaining active enough to prevent invasive pathogenic attack. The intestinal epithelium represents the frontline of this interface. It has long been known to act as a physical barrier preventing the lumenal bacteria of the gastro-intestinal tract from activating an inflammatory immune response in the immune cells of the underlying mucosa. However, in recent years, an appreciation has grown surrounding the role played by the intestinal epithelium in regulating innate immune responses, both in the prevention of infection and in maintaining a homeostatic environment through modulation of innate immune signalling systems. The aim of this thesis was to identify novel innate immune mechanisms regulating inflammation in the GI tract. To achieve this aim, we chose several aspects of regulatory mechanisms utilised in this region by the innate immune system. We identified several commensal strains of bacteria expressing proteins containing signalling domains used by Pattern Recognition Receptors (PRRs) of the innate immune system. Three such bacterial proteins were studied for their potentially subversive roles in host innate immune signalling as a means of regulating homeostasis in the GI tract. We also examined differential responses to PRR activation depending on their sub-cellular localisation. This was investigated based on reports that apical Toll-Like Receptor (TLR) 9 activation resulted in abrogation of inflammatory responses mediated by other TLRs in Intestinal Epithelial Cells (IECs) such as basolateral TLR4 activation. Using the well-studied invasive intra-cellular pathogen Listeria monocytogenes as a model for infection, we also used a PRR siRNA library screening technique to identify novel PRRs used by IECs in both inhibition and activation of inflammatory responses. Many of the PRRs identified in this screen were previously believed not to be expressed in IECs. Furthermore, the same study has led to the identification of the previously uncharacterised TLR10 as a functional inflammatory receptor of IECs. Further analysis revealed a similar role in macrophages where it was shown to respond to intracellular and motile pathogens such as Gram-positive L.monocytogenes and Gram negative Salmonella typhimurium. TLR10 expression in IECs was predominantly intracellular. This is likely in order to avoid inappropriate inflammatory activation through the recognition of commensal microbial antigens on the apical cell surface of IECs. Moreover, these results have revealed a more complex network of innate immune signalling mechanisms involved in both activating and inhibiting inflammatory responses in IECs than was previously believed. This contribution to our understanding of innate immune regulation in this region has several direct and indirect benefits. The identification of several novel PRRs involved in activating and inhibiting inflammation in the GI tract may be used as novel therapeutic targets in the treatment of disease; both for inducing tolerance and reducing inflammation, or indeed, as targets for adjuvant activation in the development of oral vaccines against pathogenic attack.

Instrument independent diffuse reflectance spectroscopy.

Diffuse reflectance spectroscopy with a fiber optic probe is a powerful tool for quantitative tissue characterization and disease diagnosis. Significant systematic errors can arise in the measured reflectance spectra and thus in the derived tissue physiological and morphological parameters due to real-time instrument fluctuations. We demonstrate a novel fiber optic probe with real-time, self-calibration capability that can be used for UV-visible diffuse reflectance spectroscopy in biological tissue in clinical settings. The probe is tested in a number of synthetic liquid phantoms over a wide range of tissue optical properties for significant variations in source intensity fluctuations caused by instrument warm up and day-to-day drift. While the accuracy for extraction of absorber concentrations is comparable to that achieved with the traditional calibration (with a reflectance standard), the accuracy for extraction of reduced scattering coefficients is significantly improved with the self-calibration probe compared to traditional calibration. This technology could be used to achieve instrument-independent diffuse reflectance spectroscopy in vivo and obviate the need for instrument warm up and post∕premeasurement calibration, thus saving up to an hour of precious clinical time.

Object files can be purely episodic.

Our ability to track an object as the same persisting entity over time and motion may primarily rely on spatiotemporal representations which encode some, but not all, of an object's features. Previous researchers using the 'object reviewing' paradigm have demonstrated that such representations can store featural information of well-learned stimuli such as letters and words at a highly abstract level. However, it is unknown whether these representations can also store purely episodic information (i.e. information obtained from a single, novel encounter) that does not correspond to pre-existing type-representations in long-term memory. Here, in an object-reviewing experiment with novel face images as stimuli, observers still produced reliable object-specific preview benefits in dynamic displays: a preview of a novel face on a specific object speeded the recognition of that particular face at a later point when it appeared again on the same object compared to when it reappeared on a different object (beyond display-wide priming), even when all objects moved to new positions in the intervening delay. This case study demonstrates that the mid-level visual representations which keep track of persisting identity over time--e.g. 'object files', in one popular framework can store not only abstract types from long-term memory, but also specific tokens from online visual experience.

Temporal dynamics of host molecular responses differentiate symptomatic and asymptomatic influenza a infection.

Exposure to influenza viruses is necessary, but not sufficient, for healthy human hosts to develop symptomatic illness. The host response is an important determinant of disease progression. In order to delineate host molecular responses that differentiate symptomatic and asymptomatic Influenza A infection, we inoculated 17 healthy adults with live influenza (H3N2/Wisconsin) and examined changes in host peripheral blood gene expression at 16 timepoints over 132 hours. Here we present distinct transcriptional dynamics of host responses unique to asymptomatic and symptomatic infections. We show that symptomatic hosts invoke, simultaneously, multiple pattern recognition receptors-mediated antiviral and inflammatory responses that may relate to virus-induced oxidative stress. In contrast, asymptomatic subjects tightly regulate these responses and exhibit elevated expression of genes that function in antioxidant responses and cell-mediated responses. We reveal an ab initio molecular signature that strongly correlates to symptomatic clinical disease and biomarkers whose expression patterns best discriminate early from late phases of infection. Our results establish a temporal pattern of host molecular responses that differentiates symptomatic from asymptomatic infections and reveals an asymptomatic host-unique non-passive response signature, suggesting novel putative molecular targets for both prognostic assessment and ameliorative therapeutic intervention in seasonal and pandemic influenza.

Gene selection using iterative feature elimination random forests for survival outcomes.

Although many feature selection methods for classification have been developed, there is a need to identify genes in high-dimensional data with censored survival outcomes. Traditional methods for gene selection in classification problems have several drawbacks. First, the majority of the gene selection approaches for classification are single-gene based. Second, many of the gene selection procedures are not embedded within the algorithm itself. The technique of random forests has been found to perform well in high-dimensional data settings with survival outcomes. It also has an embedded feature to identify variables of importance. Therefore, it is an ideal candidate for gene selection in high-dimensional data with survival outcomes. In this paper, we develop a novel method based on the random forests to identify a set of prognostic genes. We compare our method with several machine learning methods and various node split criteria using several real data sets. Our method performed well in both simulations and real data analysis.Additionally, we have shown the advantages of our approach over single-gene-based approaches. Our method incorporates multivariate correlations in microarray data for survival outcomes. The described method allows us to better utilize the information available from microarray data with survival outcomes.

Bottlenose dolphins exchange signature whistles when meeting at sea.

The bottlenose dolphin, Tursiops truncatus, is one of very few animals that, through vocal learning, can invent novel acoustic signals and copy whistles of conspecifics. Furthermore, receivers can extract identity information from the invented part of whistles. In captivity, dolphins use such signature whistles while separated from the rest of their group. However, little is known about how they use them at sea. If signature whistles are the main vehicle to transmit identity information, then dolphins should exchange these whistles in contexts where groups or individuals join. We used passive acoustic localization during focal boat follows to observe signature whistle use in the wild. We found that stereotypic whistle exchanges occurred primarily when groups of dolphins met and joined at sea. A sequence analysis verified that most of the whistles used during joins were signature whistles. Whistle matching or copying was not observed in any of the joins. The data show that signature whistle exchanges are a significant part of a greeting sequence that allows dolphins to identify conspecifics when encountering them in the wild.

Matching case history patterns in case-based reasoning

This paper introduces a mechanism for representing and recognizing case history patterns with rich internal temporal aspects. A case history is characterized as a collection of elemental cases as in conventional case-based reasoning systems, together with the corresponding temporal constraints that can be relative and/or with absolute values. A graphical representation for case histories is proposed as a directed, partially weighted and labeled simple graph. In terms of such a graphical representation, an eigen-decomposition graph matching algorithm is proposed for recognizing case history patterns.