Self-Organizing Map Neural Architectures Based on Limit Cycle Attractors

Recent efforts to develop large-scale neural architectures have paid relatively little attention to the use of self-organizing maps (SOMs). Part of the reason is that most conventional SOMs use a static encoding representation: Each input is typically represented by the fixed activation of a single node in the map layer. This not only carries information in an inefficient and unreliable way that impedes building robust multi-SOM neural architectures, but it is also inconsistent with rhythmic oscillations in biological neural networks. Here I develop and study an alternative encoding scheme that instead uses limit cycle attractors of multi-focal activity patterns to represent input patterns/sequences. Such a fundamental change in representation raises several questions: Can this be done effectively and reliably? If so, will map formation still occur? What properties would limit cycle SOMs exhibit? Could multiple such SOMs interact effectively? Could robust architectures based on such SOMs be built for practical applications? The principal results of examining these questions are as follows. First, conditions are established for limit cycle attractors to emerge in a SOM through self-organization when encoding both static and temporal sequence inputs. It is found that under appropriate conditions a set of learned limit cycles are stable, unique, and preserve input relationships. In spite of the continually changing activity in a limit cycle SOM, map formation continues to occur reliably. Next, associations between limit cycles in different SOMs are learned. It is shown that limit cycles in one SOM can be successfully retrieved by another SOM’s limit cycle activity. Control timings can be set quite arbitrarily during both training and activation. Importantly, the learned associations generalize to new inputs that have never been seen during training. Finally, a complete neural architecture based on multiple limit cycle SOMs is presented for robotic arm control. This architecture combines open-loop and closed-loop methods to achieve high accuracy and fast movements through smooth trajectories. The architecture is robust in that disrupting or damaging the system in a variety of ways does not completely destroy the system. I conclude that limit cycle SOMs have great potentials for use in constructing robust neural architectures.

Modeling habits as self-sustaining patterns of sensorimotor behavior

In the recent history of psychology and cognitive neuroscience, the notion of habit has been reduced to a stimulus-triggered response probability correlation. In this paper we use a computational model to present an alternative theoretical view (with some philosophical implications), where habits are seen as self-maintaining patterns of behavior that share properties in common with self-maintaining biological processes, and that inhabit a complex ecological context, including the presence and influence of other habits. Far from mechanical automatisms, this organismic and self-organizing concept of habit can overcome the dominating atomistic and statistical conceptions, and the high temporal resolution effects of situatedness, embodiment and sensorimotor loops emerge as playing a more central, subtle and complex role in the organization of behavior. The model is based on a novel "iterant deformable sensorimotor medium (IDSM)," designed such that trajectories taken through sensorimotor-space increase the likelihood that in the future, similar trajectories will be taken. We couple the IDSM to sensors and motors of a simulated robot, and show that under certain conditions, the IDSM conditions, the IDSM forms self-maintaining patterns of activity that operate across the IDSM, the robot's body, and the environment. We present various environments and the resulting habits that form in them. The model acts as an abstraction of habits at a much needed sensorimotor "meso-scale" between microscopic neuron-based models and macroscopic descriptions of behavior. Finally, we discuss how this model and extensions of it can help us understand aspects of behavioral self-organization, historicity and autonomy that remain out of the scope of contemporary representationalist frameworks.

An empirical study of neighbourhood decay in Kohonen's self organizing map

In this paper, empirical results are presented which suggest that size and rate of decay of region size plays a much more significant role in the learning, and especially the development, of topographic feature maps. Using these results as a basis, a scheme for decaying region size during SOM training is proposed. The proposed technique provides near optimal training time. This scheme avoids the need for sophisticated learning gain decay schemes, and precludes the need for a priori knowledge of likely training times. This scheme also has some potential uses for continuous learning.

Reduction of invasive tests in chagasic patients with a modified self-organizing map

The applicability of AI methods to the Chagas' disease diagnosis is carried out by the use of Kohonen's self-organizing feature maps. Electrodiagnosis indicators calculated from ECG records are used as features in input vectors to train the network. Cross-validation results are used to modify the maps, providing an outstanding improvement to the interpretation of the resulting output. As a result, the map might be used to reduce the need for invasive explorations in chronic Chagas' disease.

The influence of interhemispheric connections on ongoing and evoked orientation preference maps and spiking activity in the cat primary visual cortex

In the primary visual cortex, neurons with similar physiological features are clustered together in columns extending through all six cortical layers. These columns form modular orientation preference maps. Long-range lateral fibers are associated to the structure of orientation maps since they do not connect columns randomly; they rather cluster in regular intervals and interconnect predominantly columns of neurons responding to similar stimulus features. Single orientation preference maps – the joint activation of domains preferring the same orientation - were observed to emerge spontaneously and it was speculated whether this structured ongoing activation could be caused by the underlying patchy lateral connectivity. Since long-range lateral connections share many features, i.e. clustering, orientation selectivity, with visual inter-hemispheric connections (VIC) through the corpus callosum we used the latter as a model for long-range lateral connectivity. In order to address the question of how the lateral connectivity contributes to spontaneously generated maps of one hemisphere we investigated how these maps react to the deactivation of VICs originating from the contralateral hemisphere. To this end, we performed experiments in eight adult cats. We recorded voltage-sensitive dye (VSD) imaging and electrophysiological spiking activity in one brain hemisphere while reversible deactivating the other hemisphere with a cooling technique. In order to compare ongoing activity with evoked activity patterns we first presented oriented gratings as visual stimuli. Gratings had 8 different orientations distributed equally between 0º and 180º. VSD imaged frames obtained during ongoing activity conditions were then compared to the averaged evoked single orientation maps in three different states: baseline, cooling and recovery. Kohonen self-organizing maps were also used as a means of analysis without prior assumption (like the averaged single condition maps) on ongoing activity. We also evaluated if cooling had a differential effect on evoked and ongoing spiking activity of single units. We found that deactivating VICs caused no spatial disruption on the structure of either evoked or ongoing activity maps. The frequency with which a cardinally preferring (0º or 90º) map would emerge, however, decreased significantly for ongoing but not for evoked activity. The same result was found by training self-organizing maps with recorded data as input. Spiking activity of cardinally preferring units also decreased significantly for ongoing when compared to evoked activity. Based on our results we came to the following conclusions: 1) VICs are not a determinant factor of ongoing map structure. Maps continued to be spontaneously generated with the same quality, probably by a combination of ongoing activity from local recurrent connections, thalamocortical loop and feedback connections. 2) VICs account for a cardinal bias in the temporal sequence of ongoing activity patterns, i.e. deactivating VIC decreases the probability of cardinal maps to emerge spontaneously. 3) Inter- and intrahemispheric long-range connections might serve as a grid preparing primary visual cortex for likely junctions in a larger visual environment encompassing the two hemifields.

The influence of interhemispheric connections on ongoing and evoked orientation preference maps and spiking activity in the cat primary visual cortex

In the primary visual cortex, neurons with similar physiological features are clustered together in columns extending through all six cortical layers. These columns form modular orientation preference maps. Long-range lateral fibers are associated to the structure of orientation maps since they do not connect columns randomly; they rather cluster in regular intervals and interconnect predominantly columns of neurons responding to similar stimulus features. Single orientation preference maps – the joint activation of domains preferring the same orientation - were observed to emerge spontaneously and it was speculated whether this structured ongoing activation could be caused by the underlying patchy lateral connectivity. Since long-range lateral connections share many features, i.e. clustering, orientation selectivity, with visual inter-hemispheric connections (VIC) through the corpus callosum we used the latter as a model for long-range lateral connectivity. In order to address the question of how the lateral connectivity contributes to spontaneously generated maps of one hemisphere we investigated how these maps react to the deactivation of VICs originating from the contralateral hemisphere. To this end, we performed experiments in eight adult cats. We recorded voltage-sensitive dye (VSD) imaging and electrophysiological spiking activity in one brain hemisphere while reversible deactivating the other hemisphere with a cooling technique. In order to compare ongoing activity with evoked activity patterns we first presented oriented gratings as visual stimuli. Gratings had 8 different orientations distributed equally between 0º and 180º. VSD imaged frames obtained during ongoing activity conditions were then compared to the averaged evoked single orientation maps in three different states: baseline, cooling and recovery. Kohonen self-organizing maps were also used as a means of analysis without prior assumption (like the averaged single condition maps) on ongoing activity. We also evaluated if cooling had a differential effect on evoked and ongoing spiking activity of single units. We found that deactivating VICs caused no spatial disruption on the structure of either evoked or ongoing activity maps. The frequency with which a cardinally preferring (0º or 90º) map would emerge, however, decreased significantly for ongoing but not for evoked activity. The same result was found by training self-organizing maps with recorded data as input. Spiking activity of cardinally preferring units also decreased significantly for ongoing when compared to evoked activity. Based on our results we came to the following conclusions: 1) VICs are not a determinant factor of ongoing map structure. Maps continued to be spontaneously generated with the same quality, probably by a combination of ongoing activity from local recurrent connections, thalamocortical loop and feedback connections. 2) VICs account for a cardinal bias in the temporal sequence of ongoing activity patterns, i.e. deactivating VIC decreases the probability of cardinal maps to emerge spontaneously. 3) Inter- and intrahemispheric long-range connections might serve as a grid preparing primary visual cortex for likely junctions in a larger visual environment encompassing the two hemifields.

Impressive Gelation in Organic Solvents by Synthetic, Low Molecular Mass, Self-Organizing Urethane Amides of L-Phenylalanine

Synthetic routes leading to 12 L-phenylalanine based mono- and bipolar derivatives (1-12) and an in-depth study of their structure-property relationship with respect to gelation have been presented. These include monopolar systems such as N-[(benzyloxy)carbonyl]-L-phenylalanine-N-alkylamides and the corresponding bipolar derivatives with flexible and rigid spacers such as with 1,12-diaminododecane and 4,4'-diaminodiphenylmethane, respectively. The two ends of the latter have been functionalized with N-[(benzyloxy)carbonyl]-L-phenylalanine units via amide connection. Another bipolar molecule was synthesized in which the middle portion of the hydrocarbon segment contained polymerizable diacetylene unit. To ascertain the role of the presence of urethane linkages in the gelator molecule protected L-phenylalanine derivatives were also synthesized in which the (benzyloxy)carbonyl group has been replaced with (tert-butyloxy)carbonyl, acetyl, and benzoyl groups, respectively. Upon completion of the synthesis and adequate characterization of the newly described molecules, we examined the aggregation and gelation properties of each of them in a number of solvents and their mixtures. Optical microscopy and electron microscopy further characterized the systems that formed gels. Few representative systems, which showed excellent gelation behavior was, further examined by FT-IR, calorimetric, and powder X-ray diffraction studies. To explain the possible reasons for gelation, the results of molecular modeling and energy-minimization studies were also included. Taken together these results demonstrate the importance of the presence of (benzyloxy)carbonyl unit, urethane and secondary amide linkages, chiral purities of the headgroup and the length of the alkyl chain of the hydrophobic segment as critical determinants toward effective gelation.

A Self-Organizing Multiple-View Representation of 3D Objects

We explore representation of 3D objects in which several distinct 2D views are stored for each object. We demonstrate the ability of a two-layer network of thresholded summation units to support such representations. Using unsupervised Hebbian relaxation, we trained the network to recognise ten objects from different viewpoints. The training process led to the emergence of compact representations of the specific input views. When tested on novel views of the same objects, the network exhibited a substantial generalisation capability. In simulated psychophysical experiments, the network's behavior was qualitatively similar to that of human subjects.

A Self-Organizing Neural Network for Learning a Body-Centered Invariant Representation of 3-D Target Position

This paper describes a self-organizing neural network that rapidly learns a body-centered representation of 3-D target positions. This representation remains invariant under head and eye movements, and is a key component of sensory-motor systems for producing motor equivalent reaches to targets (Bullock, Grossberg, and Guenther, 1993).

Sequential Memory with ART: A Self-Organizing Network Capable of Learning Sequences of Patterns

A model which extends the adaptive resonance theory model to sequential memory is presented. This new model learns sequences of events and recalls a sequence when presented with parts of the sequence. A sequence can have repeated events and different sequences can share events. The ART model is modified by creating interconnected sublayers within ART's F2 layer. Nodes within F2 learn temporal patterns by forming recency gradients within LTM. Versions of the ART model like ART I, ART 2, and fuzzy ART can be used.

ARTMAP: Supervised Real-Time Learning and Classification of Nonstationary Data by a Self-Organizing Neural Network

This article introduces a new neural network architecture, called ARTMAP, that autonomously learns to classify arbitrarily many, arbitrarily ordered vectors into recognition categories based on predictive success. This supervised learning system is built up from a pair of Adaptive Resonance Theory modules (ARTa and ARTb) that are capable of self-organizing stable recognition categories in response to arbitrary sequences of input patterns. During training trials, the ARTa module receives a stream {a^(p)} of input patterns, and ARTb receives a stream {b^(p)} of input patterns, where b^(p) is the correct prediction given a^(p). These ART modules are linked by an associative learning network and an internal controller that ensures autonomous system operation in real time. During test trials, the remaining patterns a^(p) are presented without b^(p), and their predictions at ARTb are compared with b^(p). Tested on a benchmark machine learning database in both on-line and off-line simulations, the ARTMAP system learns orders of magnitude more quickly, efficiently, and accurately than alternative algorithms, and achieves 100% accuracy after training on less than half the input patterns in the database. It achieves these properties by using an internal controller that conjointly maximizes predictive generalization and minimizes predictive error by linking predictive success to category size on a trial-by-trial basis, using only local operations. This computation increases the vigilance parameter ρa of ARTa by the minimal amount needed to correct a predictive error at ARTb· Parameter ρa calibrates the minimum confidence that ARTa must have in a category, or hypothesis, activated by an input a^(p) in order for ARTa to accept that category, rather than search for a better one through an automatically controlled process of hypothesis testing. Parameter ρa is compared with the degree of match between a^(p) and the top-down learned expectation, or prototype, that is read-out subsequent to activation of an ARTa category. Search occurs if the degree of match is less than ρa. ARTMAP is hereby a type of self-organizing expert system that calibrates the selectivity of its hypotheses based upon predictive success. As a result, rare but important events can be quickly and sharply distinguished even if they are similar to frequent events with different consequences. Between input trials ρa relaxes to a baseline vigilance pa When ρa is large, the system runs in a conservative mode, wherein predictions are made only if the system is confident of the outcome. Very few false-alarm errors then occur at any stage of learning, yet the system reaches asymptote with no loss of speed. Because ARTMAP learning is self stabilizing, it can continue learning one or more databases, without degrading its corpus of memories, until its full memory capacity is utilized.

A Self-Organizing Neural Model of Motor Equivalent Reaching and Tool Use by a Multijoint Arm

This paper describes a self-organizing neural model for eye-hand coordination. Called the DIRECT model, it embodies a solution of the classical motor equivalence problem. Motor equivalence computations allow humans and other animals to flexibly employ an arm with more degrees of freedom than the space in which it moves to carry out spatially defined tasks under conditions that may require novel joint configurations. During a motor babbling phase, the model endogenously generates movement commands that activate the correlated visual, spatial, and motor information that are used to learn its internal coordinate transformations. After learning occurs, the model is capable of controlling reaching movements of the arm to prescribed spatial targets using many different combinations of joints. When allowed visual feedback, the model can automatically perform, without additional learning, reaches with tools of variable lengths, with clamped joints, with distortions of visual input by a prism, and with unexpected perturbations. These compensatory computations occur within a single accurate reaching movement. No corrective movements are needed. Blind reaches using internal feedback have also been simulated. The model achieves its competence by transforming visual information about target position and end effector position in 3-D space into a body-centered spatial representation of the direction in 3-D space that the end effector must move to contact the target. The spatial direction vector is adaptively transformed into a motor direction vector, which represents the joint rotations that move the end effector in the desired spatial direction from the present arm configuration. Properties of the model are compared with psychophysical data on human reaching movements, neurophysiological data on the tuning curves of neurons in the monkey motor cortex, and alternative models of movement control.

Understanding the socio-economic distribution of multiple deprivation: An application of self-organising maps

In this paper we seek to contribute to recent efforts to develop and implement multi-dimensional approaches to social exclusion by applying self-organising maps (SOMs) to a set of material deprivation indicators from the Irish component of EU-SILC. The first stage of our analysis involves the identification of sixteen clusters that confirm the multi-dimensional nature of deprivation in contemporary Ireland and the limitations of focusing solely on income. In going beyond this mapping stage, we consider both patterns of socio-economic differentiation in relation to cluster membership and the extent to which such membership contributes to our understanding of economic stress. Our analysis makes clear the continuing importance of traditional forms of stratification relating to factors such as income, social class and housing tenure in accounting for patterns of multiple deprivation. However, it also confirms the role of acute life events and life cycle and location influences. Most importantly, it demonstrates that conclusions relating to the relative impact of different kinds of socio-economic influences are highly dependent on the form of deprivation being considered. Our analysis suggests that debates relating to the extent to which poverty and social exclusion have become individualized should take particular care to distinguish between different kinds of outcomes. Further analysis demonstrates that the SOM approach is considerably more successful than a comparable latent class analysis in identifying those exposed to subjective economic stress. (C) 2010 International Sociological Association Research Committee 28 on Social Stratification and Mobility. Published by Elsevier Ltd. All rights reserved.