Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System

A Fuzzy ART model capable of rapid stable learning of recognition categories in response to arbitrary sequences of analog or binary input patterns is described. Fuzzy ART incorporates computations from fuzzy set theory into the ART 1 neural network, which learns to categorize only binary input patterns. The generalization to learning both analog and binary input patterns is achieved by replacing appearances of the intersection operator (n) in AHT 1 by the MIN operator (Λ) of fuzzy set theory. The MIN operator reduces to the intersection operator in the binary case. Category proliferation is prevented by normalizing input vectors at a preprocessing stage. A normalization procedure called complement coding leads to a symmetric theory in which the MIN operator (Λ) and the MAX operator (v) of fuzzy set theory play complementary roles. Complement coding uses on-cells and off-cells to represent the input pattern, and preserves individual feature amplitudes while normalizing the total on-cell/off-cell vector. Learning is stable because all adaptive weights can only decrease in time. Decreasing weights correspond to increasing sizes of category "boxes". Smaller vigilance values lead to larger category boxes. Learning stops when the input space is covered by boxes. With fast learning and a finite input set of arbitrary size and composition, learning stabilizes after just one presentation of each input pattern. A fast-commit slow-recode option combines fast learning with a forgetting rule that buffers system memory against noise. Using this option, rare events can be rapidly learned, yet previously learned memories are not rapidly erased in response to statistically unreliable input fluctuations.

Fuzzy ARTMAP: A Neural Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps

A new neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors. The architecture, called Fuzzy ARTMAP, achieves a synthesis of fuzzy logic and Adaptive Resonance Theory (ART) neural networks by exploiting a close formal similarity between the computations of fuzzy subsethood and ART category choice, resonance, and learning. Fuzzy ARTMAP also realizes a new Minimax Learning Rule that conjointly minimizes predictive error and maximizes code compression, or generalization. This is achieved by a match tracking process that increases the ART vigilance parameter by the minimum amount needed to correct a predictive error. As a result, the system automatically learns a minimal number of recognition categories, or "hidden units", to met accuracy criteria. Category proliferation is prevented by normalizing input vectors at a preprocessing stage. A normalization procedure called complement coding leads to a symmetric theory in which the MIN operator (Λ) and the MAX operator (v) of fuzzy logic play complementary roles. Complement coding uses on-cells and off-cells to represent the input pattern, and preserves individual feature amplitudes while normalizing the total on-cell/off-cell vector. Learning is stable because all adaptive weights can only decrease in time. Decreasing weights correspond to increasing sizes of category "boxes". Smaller vigilance values lead to larger category boxes. Improved prediction is achieved by training the system several times using different orderings of the input set. This voting strategy can also be used to assign probability estimates to competing predictions given small, noisy, or incomplete training sets. Four classes of simulations illustrate Fuzzy ARTMAP performance as compared to benchmark back propagation and genetic algorithm systems. These simulations include (i) finding points inside vs. outside a circle; (ii) learning to tell two spirals apart; (iii) incremental approximation of a piecewise continuous function; and (iv) a letter recognition database. The Fuzzy ARTMAP system is also compared to Salzberg's NGE system and to Simpson's FMMC system.

A Neural Network Realization of Fuzzy ART

A neural network realization of the fuzzy Adaptive Resonance Theory (ART) algorithm is described. Fuzzy ART is capable of rapid stable learning of recognition categories in response to arbitrary sequences of analog or binary input patterns. Fuzzy ART incorporates computations from fuzzy set theory into the ART 1 neural network, which learns to categorize only binary input patterns, thus enabling the network to learn both analog and binary input patterns. In the neural network realization of fuzzy ART, signal transduction obeys a path capacity rule. Category choice is determined by a combination of bottom-up signals and learned category biases. Top-down signals impose upper bounds on feature node activations.

Working Memories for Storage and Recall of Arbitrary Temporal Sequences

A working memory model is described that is capable of storing and recalling arbitrary temporal sequences of events, including repeated items. These memories encode the invariant temporal order of sequential events that may be presented at widely differing speeds, durations, and interstimulus intervals. This temporal order code is designed to enable all possible groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system.

Comparative Performance Measures of Fuzzy ARTMAP, Learned Vector Quantization, and Back Propagation for Handwritten Character Recognition

This article compares the performance of Fuzzy ARTMAP with that of Learned Vector Quantization and Back Propagation on a handwritten character recognition task. Training with Fuzzy ARTMAP to a fixed criterion used many fewer epochs. Voting with Fuzzy ARTMAP yielded the highest recognition rates.

A What-and-Where Neural Network for Invariant Image Preprocessing

A feedforward neural network for invariant image preprocessing is proposed that represents the position1 orientation and size of an image figure (where it is) in a multiplexed spatial map. This map is used to generate an invariant representation of the figure that is insensitive to position1 orientation, and size for purposes of pattern recognition (what it is). A multiscale array of oriented filters followed by competition between orientations and scales is used to define the Where filter.

A Neural Network Model of Spatio-Temporal Pattern Recognition, Recall and Timing

This paper describes the design of a self~organizing, hierarchical neural network model of unsupervised serial learning. The model learns to recognize, store, and recall sequences of unitized patterns, using either short-term memory (STM) or both STM and long-term memory (LTM) mechanisms. Timing information is learned and recall {both from STM and from LTM) is performed with a learned rhythmical structure. The network, bearing similarities with ART (Carpenter & Grossberg 1987a), learns to map temporal sequences to unitized patterns, which makes it suitable for hierarchical operation. It is therefore capable of self-organizing codes for sequences of sequences. The capacity is only limited by the number of nodes provided. Selected simulation results are reported to illustrate system properties.

Normal and Amnesis Learning, Recognition, and Memory by a Neural Model of Cortico-Hippocampal Interactions

The processes by which humans and other primates learn to recognize objects have been the subject of many models. Processes such as learning, categorization, attention, memory search, expectation, and novelty detection work together at different stages to realize object recognition. In this article, Gail Carpenter and Stephen Grossberg describe one such model class (Adaptive Resonance Theory, ART) and discuss how its structure and function might relate to known neurological learning and memory processes, such as how inferotemporal cortex can recognize both specialized and abstract information, and how medial temporal amnesia may be caused by lesions in the hippocampal formation. The model also suggests how hippocampal and inferotemporal processing may be linked during recognition learning.

Store Working Memory Networks for Storage and Recall of Arbitrary Temporal Sequences

Neural network models of working memory, called Sustained Temporal Order REcurrent (STORE) models, are described. They encode the invariant temporal order of sequential events in short term memory (STM) in a way that mimics cognitive data about working memory, including primacy, recency, and bowed order and error gradients. As new items are presented, the pattern of previously stored items is invariant in the sense that, relative activations remain constant through time. This invariant temporal order code enables all possible groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system. Such a competence is needed to design self-organizing temporal recognition and planning systems in which any subsequence of events may need to be categorized in order to to control and predict future behavior or external events. STORE models show how arbitrary event sequences may be invariantly stored, including repeated events. A preprocessor interacts with the working memory to represent event repeats in spatially separate locations. It is shown why at least two processing levels are needed to invariantly store events presented with variable durations and interstimulus intervals. It is also shown how network parameters control the type and shape of primacy, recency, or bowed temporal order gradients that will be stored.

Preschool anxiety disorders predict different patterns of amygdala-prefrontal connectivity at school-age.

OBJECTIVE: In this prospective, longitudinal study of young children, we examined whether a history of preschool generalized anxiety, separation anxiety, and/or social phobia is associated with amygdala-prefrontal dysregulation at school-age. As an exploratory analysis, we investigated whether distinct anxiety disorders differ in the patterns of this amygdala-prefrontal dysregulation. METHODS: Participants were children taking part in a 5-year study of early childhood brain development and anxiety disorders. Preschool symptoms of generalized anxiety, separation anxiety, and social phobia were assessed with the Preschool Age Psychiatric Assessment (PAPA) in the first wave of the study when the children were between 2 and 5 years old. The PAPA was repeated at age 6. We conducted functional MRIs when the children were 5.5 to 9.5 year old to assess neural responses to viewing of angry and fearful faces. RESULTS: A history of preschool social phobia predicted less school-age functional connectivity between the amygdala and the ventral prefrontal cortices to angry faces. Preschool generalized anxiety predicted less functional connectivity between the amygdala and dorsal prefrontal cortices in response to fearful faces. Finally, a history of preschool separation anxiety predicted less school-age functional connectivity between the amygdala and the ventral prefrontal cortices to angry faces and greater school-age functional connectivity between the amygdala and dorsal prefrontal cortices to angry faces. CONCLUSIONS: Our results suggest that there are enduring neurobiological effects associated with a history of preschool anxiety, which occur over-and-above the effect of subsequent emotional symptoms. Our results also provide preliminary evidence for the neurobiological differentiation of specific preschool anxiety disorders.

Biological and physical interactions on a tropical island coral reef: Transport and retention processes on moorea, French Polynesia

The Moorea Coral Reef Long Term Ecological Research project funded by the US National Science Foundation includes multidisciplinary studies of physical processes driving ecological dynamics across the fringing reef, back reef, and fore reef habitats of Moorea, French Polynesia. A network of oceanographic moorings and a variety of other approaches have been used to investigate the biological and biogeochemical aspects of water transport and retention processes in this system. There is evidence to support the hypothesis that a low-frequency counterclockwise flow around the island is superimposed on the relatively strong alongshore currents on each side of the island. Despite the rapid flow and flushing of the back reef, waters over the reef display chemical and biological characteristics distinct from those offshore. The patterns include higher nutrient and lower dissolved organic carbon concentrations, distinct microbial community compositions among habitats, and reef assemblages of zooplankton that exhibit migration behavior, suggesting multigenerational residence on the reef. Zooplankton consumption by planktivorous fish on the reef reflects both retention of reef-associated taxa and capture by the reef community of resources originating offshore. Coral recruitment and population genetics of reef fishes point to retention of larvae within the system and high recruitment levels from local adult populations. The combined results suggest that a broad suite of physical and biological processes contribute to high retention of externally derived and locally produced organic materials within this island coral reef system. © 2013 by The Oceanography Society. All rights reserved.

Observations of spatial flow patterns at the coral colony scale on a shallow reef flat

Although small-scale spatial flow variability can affect both larger-scale circulation patterns and biological processes on coral reefs, there are few direct measurements of spatial flow patterns across horizontal scales <100 m. Here flow patterns on a shallow reef flat were measured at scales from a single colony to several adjacent colonies using an array of acoustic Doppler velocimeters on a diver-operated traverse. We observed recirculation zones immediately behind colonies, reduced currents and elevated dissipation rates in turbulent wakes up to 2 colony diameters downstream and enhanced Reynolds stresses in shear layers around wake peripheries. Flow acceleration zones were observed above and between colonies. Coherent flow structures varied with incident flow speeds; recirculation zones were stronger and wakes were more turbulent in faster flows. Low-frequency (<0.03 Hz) flow variations, for which water excursions were large compared with the colony diameters (Keulegan-Carpenter number, KC >1), had similarspatial patterns to wakes, while higher-frequency variations (0.05-0.1 Hz, KC<1) had no observable spatial structure. On the reef flat, both drag and inertial forces exerted by coral colonies could have significant effects on flow, but within different frequency ranges; drag dominates for low-frequency flow variations and inertial forces dominate for higher frequency variations, including the wave band. Our scaling analyses suggest that spatial flow patterns at colony and patch scales could have important implications or both physical and biological processes at larger reef scales through their effects on forces exerted on the flow, turbulent mixing, and dispersion. © 2013. American Geophysical Union. All Rights Reserved.

SOFT – generating highly flexible object code from XML specifications

Many code generation tools exist to aid developers in carrying out common mappings, such as from Object to XML or from Object to relational database. Such generated code tends to possess a high binding between the Object code and the target mapping, making integration into a broader application tedious or even impossible. In this paper we suggest XML technologies and the multiple inheritance capabilities of interface based languages such as Java, offer a means to unify such executable specifications, thus building complete, consistent and useful object models declaratively, without sacrificing component flexibility.

On the persistence of computer dreams: an application framework for robust adaptive deployment

The anticipated rewards of adaptive approaches will only be fully realised when autonomic algorithms can take configuration and deployment decisions that match and exceed those of human engineers. Such decisions are typically characterised as being based on a foundation of experience and knowledge. In humans, these underpinnings are themselves founded on the ashes of failure, the exuberance of courage and (sometimes) the outrageousness of fortune. In this paper we describe an application framework that will allow the incorporation of similarly risky, error prone and downright dangerous software artefacts into live systems – without undermining the certainty of correctness at application level. We achieve this by introducing the notion of application dreaming.

Towards the design of a portal framework for web services integration

Web services based systems have recently found their way into many applications such as e-commerce, corporate integration and e-learning. Construction of new services or introducing new functions to existing services requires composition of web services. Current approaches to service composition often require major programming effort; this is time consuming and requires considerable developer expertise. In this paper, we explore the real and rich scenarios found in e-learning where education services are offered through the Internet by networked universities to potentially millions in the world. These services are derived from existing/emerging business operation processes and commonly offered through a web interface, combined with other services such as email and ftp services, to support partial/full business processes. We identify the requirements for a generic portal framework for easy integration of existing expertise and services of individual institutions (enterprises). We examine the existing technologies and standards, and point out the gaps to be filled in designing the architecture of the framework