Dynamic Programming for Optimal Control of Set-Up Scheduling with Neural Network Modifications

This paper demonstrates an optimal control solution to change of machine set-up scheduling based on dynamic programming average cost per stage value iteration as set forth by Cararnanis et. al. [2] for the 2D case. The difficulty with the optimal approach lies in the explosive computational growth of the resulting solution. A method of reducing the computational complexity is developed using ideas from biology and neural networks. A real time controller is described that uses a linear-log representation of state space with neural networks employed to fit cost surfaces.

Working Memories for Storage and Recall of Arbitrary Temporal Sequences

A working memory model is described that is capable of storing and recalling arbitrary temporal sequences of events, including repeated items. These memories encode the invariant temporal order of sequential events that may be presented at widely differing speeds, durations, and interstimulus intervals. This temporal order code is designed to enable all possible groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system.

A Self-Organizing Neural Network Model for Redundant Sensory-Motor Control, Motor Equivalence, and Tool Use

A neural network is introduced which provides a solution of the classical motor equivalence problem, whereby many different joint configurations of a redundant manipulator can all be used to realize a desired trajectory in 3-D space. To do this, the network self-organizes a mapping from motion directions in 3-D space to velocity commands in joint space. Computer simulations demonstrate that, without any additional learning, the network can generate accurate movement commands that compensate for variable tool lengths, clamping of joints, distortions of visual input by a prism, and unexpected limb perturbations. Blind reaches have also been simulated.

Comparative Performance Measures of Fuzzy ARTMAP, Learned Vector Quantization, and Back Propagation for Handwritten Character Recognition

This article compares the performance of Fuzzy ARTMAP with that of Learned Vector Quantization and Back Propagation on a handwritten character recognition task. Training with Fuzzy ARTMAP to a fixed criterion used many fewer epochs. Voting with Fuzzy ARTMAP yielded the highest recognition rates.

A What-and-Where Neural Network for Invariant Image Preprocessing

A feedforward neural network for invariant image preprocessing is proposed that represents the position1 orientation and size of an image figure (where it is) in a multiplexed spatial map. This map is used to generate an invariant representation of the figure that is insensitive to position1 orientation, and size for purposes of pattern recognition (what it is). A multiscale array of oriented filters followed by competition between orientations and scales is used to define the Where filter.

The Synthesis of Arbitrary Stable Dynamics in Non-linear Neural Networks II: Feedback and Universality

We wish to construct a realization theory of stable neural networks and use this theory to model the variety of stable dynamics apparent in natural data. Such a theory should have numerous applications to constructing specific artificial neural networks with desired dynamical behavior. The networks used in this theory should have well understood dynamics yet be as diverse as possible to capture natural diversity. In this article, I describe a parameterized family of higher order, gradient-like neural networks which have known arbitrary equilibria with unstable manifolds of known specified dimension. Moreover, any system with hyperbolic dynamics is conjugate to one of these systems in a neighborhood of the equilibrium points. Prior work on how to synthesize attractors using dynamical systems theory, optimization, or direct parametric. fits to known stable systems, is either non-constructive, lacks generality, or has unspecified attracting equilibria. More specifically, We construct a parameterized family of gradient-like neural networks with a simple feedback rule which will generate equilibrium points with a set of unstable manifolds of specified dimension. Strict Lyapunov functions and nested periodic orbits are obtained for these systems and used as a method of synthesis to generate a large family of systems with the same local dynamics. This work is applied to show how one can interpolate finite sets of data, on nested periodic orbits.

A Neural Pattern Generator that Exhibits Frequency-Dependent In-Phase and Anti-Phase Oscillations

This article describes a. neural pattern generator based on a cooperative-competitive feedback neural network. The two-channel version of the generator supports both in-phase and anti-phase oscillations. A scalar arousal level controls both the oscillation phase and frequency. As arousal increases, oscillation frequency increases and bifurcations from in-phase to anti-phase, or anti-phase to in-phase oscillations can occur. Coupled versions of the model exhibit oscillatory patterns which correspond to the gaits used in locomotion and other oscillatory movements by various animals.

A Neural Network for Synthesizing the Pitch of an Acoustic Source

This article describes a neural network model capable of generating a spatial representation of the pitch of an acoustic source. Pitch is one of several auditory percepts used by humans to separate multiple sound sources in the environment from each other. The model provides a neural instantiation of a type of "harmonic sieve". It is capable of quantitatively simulating a large body of psychoacoustical data, including new data on octave shift perception.

Construction of Neural Network Classification Expert Systems Using Switching Theory Algorithms

A new family of neural network architectures is presented. This family of architectures solves the problem of constructing and training minimal neural network classification expert systems by using switching theory. The primary insight that leads to the use of switching theory is that the problem of minimizing the number of rules and the number of IF statements (antecedents) per rule in a neural network expert system can be recast into the problem of minimizing the number of digital gates and the number of connections between digital gates in a Very Large Scale Integrated (VLSI) circuit. The rules that the neural network generates to perform a task are readily extractable from the network's weights and topology. Analysis and simulations on the Mushroom database illustrate the system's performance.

A Neural Network Model of Spatio-Temporal Pattern Recognition, Recall and Timing

This paper describes the design of a self~organizing, hierarchical neural network model of unsupervised serial learning. The model learns to recognize, store, and recall sequences of unitized patterns, using either short-term memory (STM) or both STM and long-term memory (LTM) mechanisms. Timing information is learned and recall {both from STM and from LTM) is performed with a learned rhythmical structure. The network, bearing similarities with ART (Carpenter & Grossberg 1987a), learns to map temporal sequences to unitized patterns, which makes it suitable for hierarchical operation. It is therefore capable of self-organizing codes for sequences of sequences. The capacity is only limited by the number of nodes provided. Selected simulation results are reported to illustrate system properties.

Genetically Generated Neural Networks I: Representational Effects

This paper studies several applications of genetic algorithms (GAs) within the neural networks field. After generating a robust GA engine, the system was used to generate neural network circuit architectures. This was accomplished by using the GA to determine the weights in a fully interconnected network. The importance of the internal genetic representation was shown by testing different approaches. The effects in speed of optimization of varying the constraints imposed upon the desired network were also studied. It was observed that relatively loose constraints provided results comparable to a fully constrained system. The type of neural network circuits generated were recurrent competitive fields as described by Grossberg (1982).

Genetically Generated Neural Networks II: Searching for an Optimal Representation

Genetic Algorithms (GAs) make use of an internal representation of a given system in order to perform optimization functions. The actual structural layout of this representation, called a genome, has a crucial impact on the outcome of the optimization process. The purpose of this paper is to study the effects of different internal representations in a GA, which generates neural networks. A second GA was used to optimize the genome structure. This structure produces an optimized system within a shorter time interval.

Adaptive Resonance Associative Map: A Hierarchical ART System for Fast Stable Associative Learning

This paper introduces a new class of predictive ART architectures, called Adaptive Resonance Associative Map (ARAM) which performs rapid, yet stable heteroassociative learning in real time environment. ARAM can be visualized as two ART modules sharing a single recognition code layer. The unit for recruiting a recognition code is a pattern pair. Code stabilization is ensured by restricting coding to states where resonances are reached in both modules. Simulation results have shown that ARAM is capable of self-stabilizing association of arbitrary pattern pairs of arbitrary complexity appearing in arbitrary sequence by fast learning in real time environment. Due to the symmetrical network structure, associative recall can be performed in both directions.

Equilibria and Dynamics of a Neural Network Model for Opponent Muscle Control

One of the advantages of biological skeleto-motor systems is the opponent muscle design, which in principle makes it possible to achieve facile independent control of joint angle and joint stiffness. Prior analysis of equilibrium states of a biologically-based neural network for opponent muscle control, the FLETE model, revealed that such independent control requires specialized interneuronal circuitry to efficiently coordinate the opponent force generators. In this chapter, we refine the FLETE circuit variables specification and update the equilibrium analysis. We also incorporate additional neuronal circuitry that ensures efficient opponent force generation and velocity regulation during movement.

Cortical Dynamics of 3-D Vision and Figure-Ground Pop-Out

Air Force Office of Scientific Research (90-0175); Defense Advanced Research Projects Agency (90-0083); Office of Naval Research (N00014-91-J-4100)