Biased ART: A Neural Architecture that Shifts Attention Toward Previously Disregarded Features Following an Incorrect Prediction

Memories in Adaptive Resonance Theory (ART) networks are based on matched patterns that focus attention on those portions of bottom-up inputs that match active top-down expectations. While this learning strategy has proved successful for both brain models and applications, computational examples show that attention to early critical features may later distort memory representations during online fast learning. For supervised learning, biased ARTMAP (bARTMAP) solves the problem of over-emphasis on early critical features by directing attention away from previously attended features after the system makes a predictive error. Small-scale, hand-computed analog and binary examples illustrate key model dynamics. Twodimensional simulation examples demonstrate the evolution of bARTMAP memories as they are learned online. Benchmark simulations show that featural biasing also improves performance on large-scale examples. One example, which predicts movie genres and is based, in part, on the Netflix Prize database, was developed for this project. Both first principles and consistent performance improvements on all simulation studies suggest that featural biasing should be incorporated by default in all ARTMAP systems. Benchmark datasets and bARTMAP code are available from the CNS Technology Lab Website: http://techlab.bu.edu/bART/.

Spiking Dynamics during Perceptual Grouping in the Laminar Circuits of Visual Cortex

Grouping of collinear boundary contours is a fundamental process during visual perception. Illusory contour completion vividly illustrates how stable perceptual boundaries interpolate between pairs of contour inducers, but do not extrapolate from a single inducer. Neural models have simulated how perceptual grouping occurs in laminar visual cortical circuits. These models predicted the existence of grouping cells that obey a bipole property whereby grouping can occur inwardly between pairs or greater numbers of similarly oriented and co-axial inducers, but not outwardly from individual inducers. These models have not, however, incorporated spiking dynamics. Perceptual grouping is a challenge for spiking cells because its properties of collinear facilitation and analog sensitivity to inducer configurations occur despite irregularities in spike timing across all the interacting cells. Other models have demonstrated spiking dynamics in laminar neocortical circuits, but not how perceptual grouping occurs. The current model begins to unify these two modeling streams by implementing a laminar cortical network of spiking cells whose intracellular temporal dynamics interact with recurrent intercellular spiking interactions to quantitatively simulate data from neurophysiological experiments about perceptual grouping, the structure of non-classical visual receptive fields, and gamma oscillations.

On the Relationship Between the Generalized Equality Classifier and ART 2 Neural Networks

In this paper, we introduce the Generalized Equality Classifier (GEC) for use as an unsupervised clustering algorithm in categorizing analog data. GEC is based on a formal definition of inexact equality originally developed for voting in fault tolerant software applications. GEC is defined using a metric space framework. The only parameter in GEC is a scalar threshold which defines the approximate equality of two patterns. Here, we compare the characteristics of GEC to the ART2-A algorithm (Carpenter, Grossberg, and Rosen, 1991). In particular, we show that GEC with the Hamming distance performs the same optimization as ART2. Moreover, GEC has lower computational requirements than AR12 on serial machines.

Brightness Perception, Illusory Contours and Corticogeniculate Feedback

A neural network model of early visual processing offers an explanation of brightness effects often associated with illusory contours. Top-down feedback from the model's analog of visual cortical complex cells to model lateral geniculate nucleus (LGN) cells are used to enhance contrast at line ends and other areas of boundary discontinuity. The result is an increase in perceived brightness outside a dark line end, akin to what Kennedy (1979) termed "brightness buttons" in his analysis of visual illusions. When several lines form a suitable configuration, as in an Ehrenstein pattern, the perceptual effect of enhanced brightness can be quite strong. Model simulations show the generation of brightness buttons. With the LGN model circuitry embedded in a larger model of preattentive vision, simulations using complex inputs show the interaction of the brightness buttons with real and illusory contours.

A Gestalt Bubble Model of Visuosptial Perception

A neural network model is presented to account for the three dimensional perception of visual space by way of an analog Gestalt-like perceptual mechanism.

A Neural Network Architecture for Autonomous Learning, Recognition, and Prediction in a Nonstationary World

In a constantly changing world, humans are adapted to alternate routinely between attending to familiar objects and testing hypotheses about novel ones. We can rapidly learn to recognize and narne novel objects without unselectively disrupting our memories of familiar ones. We can notice fine details that differentiate nearly identical objects and generalize across broad classes of dissimilar objects. This chapter describes a class of self-organizing neural network architectures--called ARTMAP-- that are capable of fast, yet stable, on-line recognition learning, hypothesis testing, and naming in response to an arbitrary stream of input patterns (Carpenter, Grossberg, Markuzon, Reynolds, and Rosen, 1992; Carpenter, Grossberg, and Reynolds, 1991). The intrinsic stability of ARTMAP allows the system to learn incrementally for an unlimited period of time. System stability properties can be traced to the structure of its learned memories, which encode clusters of attended features into its recognition categories, rather than slow averages of category inputs. The level of detail in the learned attentional focus is determined moment-by-moment, depending on predictive success: an error due to over-generalization automatically focuses attention on additional input details enough of which are learned in a new recognition category so that the predictive error will not be repeated. An ARTMAP system creates an evolving map between a variable number of learned categories that compress one feature space (e.g., visual features) to learned categories of another feature space (e.g., auditory features). Input vectors can be either binary or analog. Computational properties of the networks enable them to perform significantly better in benchmark studies than alternative machine learning, genetic algorithm, or neural network models. Some of the critical problems that challenge and constrain any such autonomous learning system will next be illustrated. Design principles that work together to solve these problems are then outlined. These principles are realized in the ARTMAP architecture, which is specified as an algorithm. Finally, ARTMAP dynamics are illustrated by means of a series of benchmark simulations.

Parallel Auditory Filtering By Sustained and Transient Channels Separates Coarticulated Vowels and Consonants

A neural model of peripheral auditory processing is described and used to separate features of coarticulated vowels and consonants. After preprocessing of speech via a filterbank, the model splits into two parallel channels, a sustained channel and a transient channel. The sustained channel is sensitive to relatively stable parts of the speech waveform, notably synchronous properties of the vocalic portion of the stimulus it extends the dynamic range of eighth nerve filters using coincidence deteectors that combine operations of raising to a power, rectification, delay, multiplication, time averaging, and preemphasis. The transient channel is sensitive to critical features at the onsets and offsets of speech segments. It is built up from fast excitatory neurons that are modulated by slow inhibitory interneurons. These units are combined over high frequency and low frequency ranges using operations of rectification, normalization, multiplicative gating, and opponent processing. Detectors sensitive to frication and to onset or offset of stop consonants and vowels are described. Model properties are characterized by mathematical analysis and computer simulations. Neural analogs of model cells in the cochlear nucleus and inferior colliculus are noted, as are psychophysical data about perception of CV syllables that may be explained by the sustained transient channel hypothesis. The proposed sustained and transient processing seems to be an auditory analog of the sustained and transient processing that is known to occur in vision.

Fuzzy ART

Adaptive Resonance Theory (ART) models are real-time neural networks for category learning, pattern recognition, and prediction. Unsupervised fuzzy ART and supervised fuzzy ARTMAP synthesize fuzzy logic and ART networks by exploiting the formal similarity between the computations of fuzzy subsethood and the dynamics of ART category choice, search, and learning. Fuzzy ART self-organizes stable recognition categories in response to arbitrary sequences of analog or binary input patterns. It generalizes the binary ART 1 model, replacing the set-theoretic: intersection (∩) with the fuzzy intersection (∧), or component-wise minimum. A normalization procedure called complement coding leads to a symmetric: theory in which the fuzzy inter:>ec:tion and the fuzzy union (∨), or component-wise maximum, play complementary roles. Complement coding preserves individual feature amplitudes while normalizing the input vector, and prevents a potential category proliferation problem. Adaptive weights :otart equal to one and can only decrease in time. A geometric interpretation of fuzzy AHT represents each category as a box that increases in size as weights decrease. A matching criterion controls search, determining how close an input and a learned representation must be for a category to accept the input as a new exemplar. A vigilance parameter (p) sets the matching criterion and determines how finely or coarsely an ART system will partition inputs. High vigilance creates fine categories, represented by small boxes. Learning stops when boxes cover the input space. With fast learning, fixed vigilance, and an arbitrary input set, learning stabilizes after just one presentation of each input. A fast-commit slow-recode option allows rapid learning of rare events yet buffers memories against recoding by noisy inputs. Fuzzy ARTMAP unites two fuzzy ART networks to solve supervised learning and prediction problems. A Minimax Learning Rule controls ARTMAP category structure, conjointly minimizing predictive error and maximizing code compression. Low vigilance maximizes compression but may therefore cause very different inputs to make the same prediction. When this coarse grouping strategy causes a predictive error, an internal match tracking control process increases vigilance just enough to correct the error. ARTMAP automatically constructs a minimal number of recognition categories, or "hidden units," to meet accuracy criteria. An ARTMAP voting strategy improves prediction by training the system several times using different orderings of the input set. Voting assigns confidence estimates to competing predictions given small, noisy, or incomplete training sets. ARPA benchmark simulations illustrate fuzzy ARTMAP dynamics. The chapter also compares fuzzy ARTMAP to Salzberg's Nested Generalized Exemplar (NGE) and to Simpson's Fuzzy Min-Max Classifier (FMMC); and concludes with a summary of ART and ARTMAP applications.

Fuzzy ART Choice Functions

Adaptive Resonance Theory (ART) models are real-time neural networks for category learning, pattern recognition, and prediction. Unsupervised fuzzy ART and supervised fuzzy ARTMAP networks synthesize fuzzy logic and ART by exploiting the formal similarity between tile computations of fuzzy subsethood and the dynamics of ART category choice, search, and learning. Fuzzy ART self-organizes stable recognition categories in response to arbitrary sequences of analog or binary input patterns. It generalizes the binary ART 1 model, replacing the set-theoretic intersection (∩) with the fuzzy intersection(∧), or component-wise minimum. A normalization procedure called complement coding leads to a symmetric theory in which the fuzzy intersection and the fuzzy union (∨), or component-wise maximum, play complementary roles. A geometric interpretation of fuzzy ART represents each category as a box that increases in size as weights decrease. This paper analyzes fuzzy ART models that employ various choice functions for category selection. One such function minimizes total weight change during learning. Benchmark simulations compare peformance of fuzzy ARTMAP systems that use different choice functions.

Towards a Unified Theory of Neocortex: Laminar Cortical Circuits for Vision and Cognition

A key goal of computational neuroscience is to link brain mechanisms to behavioral functions. The present article describes recent progress towards explaining how laminar neocortical circuits give rise to biological intelligence. These circuits embody two new and revolutionary computational paradigms: Complementary Computing and Laminar Computing. Circuit properties include a novel synthesis of feedforward and feedback processing, of digital and analog processing, and of pre-attentive and attentive processing. This synthesis clarifies the appeal of Bayesian approaches but has a far greater predictive range that naturally extends to self-organizing processes. Examples from vision and cognition are summarized. A LAMINART architecture unifies properties of visual development, learning, perceptual grouping, attention, and 3D vision. A key modeling theme is that the mechanisms which enable development and learning to occur in a stable way imply properties of adult behavior. It is noted how higher-order attentional constraints can influence multiple cortical regions, and how spatial and object attention work together to learn view-invariant object categories. In particular, a form-fitting spatial attentional shroud can allow an emerging view-invariant object category to remain active while multiple view categories are associated with it during sequences of saccadic eye movements. Finally, the chapter summarizes recent work on the LIST PARSE model of cognitive information processing by the laminar circuits of prefrontal cortex. LIST PARSE models the short-term storage of event sequences in working memory, their unitization through learning into sequence, or list, chunks, and their read-out in planned sequential performance that is under volitional control. LIST PARSE provides a laminar embodiment of Item and Order working memories, also called Competitive Queuing models, that have been supported by both psychophysical and neurobiological data. These examples show how variations of a common laminar cortical design can embody properties of visual and cognitive intelligence that seem, at least on the surface, to be mechanistically unrelated.

A Neural Network Realization of Fuzzy ART

A neural network realization of the fuzzy Adaptive Resonance Theory (ART) algorithm is described. Fuzzy ART is capable of rapid stable learning of recognition categories in response to arbitrary sequences of analog or binary input patterns. Fuzzy ART incorporates computations from fuzzy set theory into the ART 1 neural network, which learns to categorize only binary input patterns, thus enabling the network to learn both analog and binary input patterns. In the neural network realization of fuzzy ART, signal transduction obeys a path capacity rule. Category choice is determined by a combination of bottom-up signals and learned category biases. Top-down signals impose upper bounds on feature node activations.

The role of synchronization in digital communications using chaos - Part II: Chaotic modulation and chaotic synchronization.

For pt. I see ibid., vol. 44, p. 927-36 (1997). In a digital communications system, data are transmitted from one location to another by mapping bit sequences to symbols, and symbols to sample functions of analog waveforms. The analog waveform passes through a bandlimited (possibly time-varying) analog channel, where the signal is distorted and noise is added. In a conventional system the analog sample functions sent through the channel are weighted sums of one or more sinusoids; in a chaotic communications system the sample functions are segments of chaotic waveforms. At the receiver, the symbol may be recovered by means of coherent detection, where all possible sample functions are known, or by noncoherent detection, where one or more characteristics of the sample functions are estimated. In a coherent receiver, synchronization is the most commonly used technique for recovering the sample functions from the received waveform. These sample functions are then used as reference signals for a correlator. Synchronization-based coherent receivers have advantages over noncoherent receivers in terms of noise performance, bandwidth efficiency (in narrow-band systems) and/or data rate (in chaotic systems). These advantages are lost if synchronization cannot be maintained, for example, under poor propagation conditions. In these circumstances, communication without synchronization may be preferable. The theory of conventional telecommunications is extended to chaotic communications, chaotic modulation techniques and receiver configurations are surveyed, and chaotic synchronization schemes are described

Design, synthesis and development of novel indolocarbazole derivatives as potential anti-cancer agents

This thesis describes work carried out on the design of new routes to a range of bisindolylmaleimide and indolo[2,3-a]carbazole analogs, and investigation of their potential as successful anti-cancer agents. Following initial investigation of classical routes to indolo[2,3-a]pyrrolo[3,4-c]carbazole aglycons, a new strategy employing base-mediated condensation of thiourea and guanidine with a bisindolyl β-ketoester intermediate afforded novel 5,6-bisindolylpyrimidin-4(3H)-ones in moderate yields. Chemical diversity within this H-bonding scaffold was then studied by substitution with a panel of biologically relevant electrophiles, and by reductive desulfurisation. Optimisation of difficult heterogeneous literature conditions for oxidative desulfurisation of thiouracils was also accomplished, enabling a mild route to a novel 5,6-bisindolyluracil pharmacophore to be developed within this work. The oxidative cyclisation of selected acyclic bisindolyl systems to form a new planar class of indolo[2,3-a]pyrimido[5,4-c]carbazoles was also investigated. Successful conditions for this transformation, as well as the limitations currently prevailing for this approach are discussed. Synthesis of 3,4-bisindolyl-5-aminopyrazole as a potential isostere of bisindolylmaleimide agents was encountered, along with a comprehensive derivatisation study, in order to probe the chemical space for potential protein backbone H-bonding interactions. Synthesis of a related 3,4-arylindolyl-5-aminopyrazole series was also undertaken, based on identification of potent kinase inhibition within a closely related heterocyclic template. Following synthesis of approximately 50 novel compounds with a diversity of H-bonding enzyme-interacting potential within these classes, biological studies confirmed that significant topo II inhibition was present for 9 lead compounds, in previously unseen pyrazolo[1,5-a]pyrimidine, indolo[2,3-c]carbazole and branched S,N-disubstituted thiouracil derivative series. NCI-60 cancer cell line growth inhibition data for 6 representative compounds also revealed interesting selectivity differences between each compound class, while a new pyrimido[5,4-c]carbazole agent strongly inhibited cancer cell division at 10 µM, with appreciable cytotoxic activity observed across several tumour types.

Synthesis and reactivity of α-diazo-β-keto sulfoxides: scope and synthetic potential

The research described in this thesis focuses on the design and synthesis of stable α-diazosulfoxides and investigation of their reactivity under a variety of conditions (transition-metal catalysis, thermal, photochemical and microwave) with a particular emphasis on the synthesis of novel heterocyclic compounds with potential biological activity. The exclusive reaction pathway for these α-diazosulfoxides was found to be hetero-Wolff rearrangement to give α-oxosulfine intermediates. In the first chapter, a literature review of sulfines is presented, including a discussion of naturally occurring sulfines, and an overview of the synthesis and reactivity of sulfines. The potential of sulfines in organic synthesis and recent developments in particular are highlighted. The second chapter discusses the synthesis and reactivity of α-diazosulfoxides, building on earlier results in this research group. The synthesis of lactone-based α-diazosulfoxides and, for the first time, ketone-based benzofused and monocyclic α-diazosulfoxides is described. The reactivity of these α-diazosulfoxides is then explored under a variety of conditions, such as transition-metal catalysis, photochemical and microwave, generating labile α-oxosulfine intermediates, which are trapped using amines and dienes, in addition to the spontaneous reaction pathways which occur with α-oxosulfines in the absence of a trap. A new reaction pathway was explored with the lactone based α-oxosulfines, involving reaction with amines to generate novel 3-aminofuran-2(5H)-ones via carbophilic attack, in very good yields. The reactivity of ketone-based α-diazosulfoxides was explored for the first time, and once again, pseudo-Wolff rearrangement to the α-oxosulfines was the exclusive reaction pathway observed. The intermediacy of the α-oxosulfines was confirmed by trapping as cycloadducts, with the stereochemical features dependant on the reaction conditions. In the absence of a diene trap, a number of reaction fates from the α-oxosulfines were observed, including complete sulfinyl extrusion to give indanones, sulfur extrusion to give indanediones, and, to a lesser extent, dimerisation. The indanediones were characterised by trapping as quinoxalines, to enable full characterisation. One of the overriding outcomes of this thesis was the provision of new insights into the behaviour of α-oxosulfines with different transition metal catalysts, and under thermal, microwave and photolysis conditions. A series of 3-aminofuran-2(5H)-ones and benzofused dihydro-2H-thiopyran S-oxides were submitted for anticancer screening at the U.S. National Cancer Institute. A number of these derivatives were identified as hit compounds, with excellent cell growth inhibition. One 3-aminofuran-2(5H)-one derivative has been chosen for further screening. The third chapter details the full experimental procedures, including spectroscopic and analytical data for the compounds prepared during this research. The data for the crystal structures are contained in the attached CD.

Synthesis and reactivity of pyridine substituted α-diazocarbonyl compounds and exploration of rhodium carboxylates as asymmetric catalysts

The primary objective of this thesis was the preparation of a series of pyridine-containing α-diazocarbonyl compounds and subsequent investigation of the reactivity of these compounds on exposure to transition metal catalysts. In particular, the reactivity of the pyridyl α-diazocarbonyls was compared to that of the analogous phenyl α-diazocarbonyl compounds to ascertain the impact of replacement of the phenyl ring with pyridine. The first chapter initially provides a brief introduction into α-diazocarbonyl chemistry, comprising a compendium of well-established and recently developed methods in the preparation of these compounds, as well as an outline of the reactivity of these versatile substrates. The substantive element of this introductory chapter comprises a detailed review focused on transition metal-catalysed transformations of heterocyclic α-diazocarbonyl compounds, highlighting the extraordinary diversity of reaction products which can be accessed. This review is undertaken to set the work of this thesis in context. The results of this research are discussed in the second and third chapters together with the associated experimental details, including spectroscopic and analytical data obtained in the synthesis of all compounds during this research. The second chapter describes the preparation of a range of novel pyridine-containing α-diazocarbonyl compounds via a number of synthetic strategies including both acylation and diazo transfer methodologies. In contrast to the phenyl analogues, the generation of the pyridine α-diazocarbonyl substrates was complicated by a number of factors including the inherent basicity of the pyridine ring, tautomerism and existence of rotamers. Rhodium- and copper-mediated transformations of the pyridine-containing α-diazocarbonyl compounds is discussed in detail displaying very different reactivity patterns to those seen with the phenyl analogues; oxidation to 2,3- diketones, 1,2-hydride shift to form enones and oxonium and sulfonium ylide formation/rearrangement are prominent in the pyridyl series, with no evidence of aromatic addition to the pyridine ring. The third chapter focuses on exploration of novel chiral rhodium(II) catalysts, developed in the Maguire team, in both intermolecular cyclopropanations and intramolecular C–H insertion reactions. In this chapter, the studies are focused on standard α-diazocarbonyl compounds without heteroaryl substituents. The most notable outcome was the achievement of high enantiopurities for intramolecular C–H insertions, which were competitive with, and even surpassed, established catalyst systems in some cases. This work has provided insight into solvent and temperature effects on yields as well as enantio- and diastereoselectivity, thereby providing guidance for future development and design of chiral rhodium carboxylate catalysts. While this is a preliminary study, the significance of the results lie in the fact that these are the first reactions to give substantial asymmetric induction with these novel rhodium carboxylates. While the majority of the α-diazocarbonyl compounds explored in this work were α-diazoketones, a number of α-diazoesters are also described. Details of chiral stationary phase HPLC analysis, single crystal analysis and 2D NMR experiments are included in the Appendix (Appendix III-V).