Neural network modeling of associative memory: Beyond the Hopfield model

A number of neural network models, in which fixed-point and limit-cycle attractors of the underlying dynamics are used to store and associatively recall information, are described. In the first class of models, a hierarchical structure is used to store an exponentially large number of strongly correlated memories. The second class of models uses limit cycles to store and retrieve individual memories. A neurobiologically plausible network that generates low-amplitude periodic variations of activity, similar to the oscillations observed in electroencephalographic recordings, is also described. Results obtained from analytic and numerical studies of the properties of these networks are discussed.

On the problem of spurious patterns in neural associative memory models

The problem of spurious patterns in neural associative memory models is discussed, Some suggestions to solve this problem from the literature are reviewed and their inadequacies are pointed out, A solution based on the notion of neural self-interaction with a suitably chosen magnitude is presented for the Hebb learning rule. For an optimal learning rule based on linear programming, asymmetric dilution of synaptic connections is presented as another solution to the problem of spurious patterns, With varying percentages of asymmetric dilution it is demonstrated numerically that this optimal learning rule leads to near total suppression of spurious patterns. For practical usage of neural associative memory networks a combination of the two solutions with the optimal learning rule is recommended to be the best proposition.

Glassy behavior in neural network models of associative memory

Neural network models of associative memory exhibit a large number of spurious attractors of the network dynamics which are not correlated with any memory state. These spurious attractors, analogous to "glassy" local minima of the energy or free energy of a system of particles, degrade the performance of the network by trapping trajectories starting from states that are not close to one of the memory states. Different methods for reducing the adverse effects of spurious attractors are examined with emphasis on the role of synaptic asymmetry. (C) 2002 Elsevier Science B.V. All rights reserved.

Some experiments on human memory and a new neural model

An associative memory with parallel architecture is presented. The neurons are modelled by perceptrons having only binary, rather than continuous valued input. To store m elements each having n features, m neurons each with n connections are needed. The n features are coded as an n-bit binary vector. The weights of the n connections that store the n features of an element has only two values -1 and 1 corresponding to the absence or presence of a feature. This makes the learning very simple and straightforward. For an input corrupted by binary noise, the associative memory indicates the element that is closest (in terms of Hamming distance) to the noisy input. In the case where the noisy input is equidistant from two or more stored vectors, the associative memory indicates two or more elements simultaneously. From some simple experiments performed on the human memory and also on the associative memory, it can be concluded that the associative memory presented in this paper is in some respect more akin to a human memory than a Hopfield model.

Fixed points in a Hopfield model with random asymmetric interactions

We calculate analytically the average number of fixed points in the Hopfield model of associative memory when a random antisymmetric part is added to the otherwise symmetric synaptic matrix. Addition of the antisymmetric part causes an exponential decrease in the total number of fixed points. If the relative strength of the antisymmetric component is small, then its presence does not cause any substantial degradation of the quality of retrieval when the memory loading level is low. We also present results of numerical simulations which provide qualitative (as well as quantitative for some aspects) confirmation of the predictions of the analytic study. Our numerical results suggest that the analytic calculation of the average number of fixed points yields the correct value for the typical number of fixed points.

Structural basis for the function of anti-idiotypic antibody in immune memory

We had earlier proposed a hypothesis to explain the mechanism of perpetuation of immunological memory based on the operation of idiotypic network in the complete absence of antigen. Experimental evidences were provided for memory maintenance through anti-idiotypic antibody (Ab2) carrying the internal image of the antigen. In the present work, we describe a structural basis for such memory perpetuation by molecular modeling and structural analysis studies. A three-dimensional model of Ab2 was generated and the structure of the antigenic site on the hemagglutinin protein H of Rinderpest virus was modeled using the structural template of hemagglutinin protein of Measles virus. Our results show that a large portion of heavy chain containing the CDR regions of Ab2 resembles the domain of the hemagglutinin housing the epitope regions. The similarity demonstrates that an internal image of the H antigen is formed in Ab2, which provides a structural basis for functional mimicry demonstrated earlier. This work brings out the importance of the structural similarity between a domain of hemagglutinin protein to that of its corresponding Ab2. It provides evidence that Ab2 is indeed capable of functioning as surrogate antigen and provides support to earlier proposed relay hypothesis which has provided a mechanism for the maintenance of immunological memory.

Structural basis for the function of anti-idiotypic antibody in immune memory

We had earlier proposed a hypothesis to explain the mechanism of perpetuation of immunological memory based on the operation of idiotypic network in the complete absence of antigen. Experimental evidences were provided for memory maintenance through anti-idiotypic antibody (Ab(2)) carrying the internal image of the antigen. In the present work, we describe a structural basis for such memory perpetuation by molecular modeling and structural analysis studies. A three-dimensional model of Ab(2) was generated and the structure of the antigenic site on the hemagglutinin protein H of Rinderpest virus was modeled using the structural template of hemagglutinin protein of Measles virus. Our results show that a large portion of heavy chain containing the CDR regions of Ab(2) resembles the domain of the hemagglutinin housing the epitope regions. The similarity demonstrates that an internal image of the H antigen is formed in Ab(2), which provides a structural basis for functional mimicry demonstrated earlier. This work brings out the importance of the structural similarity between a domain of hemagglutinin protein to that of its corresponding Ab(2). It provides evidence that Ab(2) is indeed capable of functioning as surrogate antigen and provides support to earlier proposed relay hypothesis which has provided a mechanism for the maintenance of immunological memory.

SHAMP: An experimental shared memory multimicroprocessor system for performance evaluation of parallel algorithms

In this paper, the design and implementation of a single shared bus, shared memory multiprocessing system using Intel's single board computers is presented. The hardware configuration and the operating system developed to execute the parallel algorithms are discussed. The performance evaluation studies carried out on Image are outlined.

Comment on path integration of a quadratic action with a generalized memory

Using the promeasure technique, we give an alternative evaluation of a path integral corresponding to a quadratic action with a generalized memory.

Constitutive modeling of shape memory alloy wire with non-local rate kinetics

A constitutive modeling approach for shape memory alloy (SMA) wire by taking into account the microstructural phase inhomogeneity and the associated solid-solid phase transformation kinetics is reported in this paper. The approach is applicable to general thermomechanical loading. Characterization of various scales in the non-local rate sensitive kinetics is the main focus of this paper. Design of SMA materials and actuators not only involve an optimal exploitation of the hysteresis loops during loading-unloading, but also accounts for fatigue and training cycle identifications. For a successful design of SMA integrated actuator systems, it is essential to include the microstructural inhomogeneity effects and the loading rate dependence of the martensitic evolution, since these factors play predominant role in fatigue. In the proposed formulation, the evolution of new phase is assumed according to Weibull distribution. Fourier transformation and finite difference methods are applied to arrive at the analytical form of two important scaling parameters. The ratio of these scaling parameters is of the order of 10(6) for stress-free temperature-induced transformation and 10(4) for stress-induced transformation. These scaling parameters are used in order to study the effect of microstructural variation on the thermo-mechanical force and interface driving force. It is observed that the interface driving force is significant during the evolution. Increase in the slopes of the transformation start and end regions in the stress-strain hysteresis loop is observed for mechanical loading with higher rates.

Comment on “path integration of a quadratic action with a generalized memory

Using the promeasure technique, we give an alternative evaluation of a path integral corresponding to a quadratic action with a generalized memory.

Sensing Shape Recovery Using Conductivity Noise in Thin Films of NiTi Shape Memory Alloys

Low frequency fluctuations in the electrical resistivity, or noise, have been used as a sensitive tool to probe into the temperature driven martensite transition in dc magnetron sputtered thin films of nickel titanium shape-memory alloys. Even in the equilibrium or static case, the noise magnitude was more than nine orders of magnitude larger than conventional metallic thin films and had a characteristic dependence on temperature. We observe that the noise while the temperature is being ramped is far larger as compared to the equilibrium noise indicating the sensitivity of electrical resistivity to the nucleation and propagation of domains during the shape recovery. Further, the higher order statistics suggests the existence of long range correlations during the transition. This new characterization is based on the kinetics of disorder in the system and separate from existing techniques and can be integrated to many device applications of shape memory alloys for in-situ shape recovery sensing.

Design and evaluation of a dual-microcomputer shared memory system with a shared I/O bus

The design, implementation and evaluation are described of a dual-microcomputer system based on the concept of shared memory. Shared memory is useful for passing large blocks of data and it also provides a means to hold and work with shared data. In addition to the shared memory, a separate bus between the I/O ports of the microcomputers is provided. This bus is utilized for interprocessor synchronization. Software routines helpful in applying the dual-microcomputer system to realistic problems are presented. Performance evaluation of the system is carried out using benchmarks.

On Bit Steering in the Minimization of the Control Memory of Microprogrammed Processors

The microcommands constituting the microprogram of the control memory of a microprogrammed processor can be partitioned into a number of disjoint sets. Some of these sets are then encoded to minimize the word width of the ROM storing the microprogram. A further reduction in the width of the ROM words can be achieved by a technique known as bit steering where one or more bits are shared by two or more sets of microcommands. These sets are called the steerable sets. This correspondence presents a simple method for the detection and encoding of steerable sets. It has been shown that the concurrency matrix of two steerable sets exhibits definite patterns of clusters which can be easily recognized. A relation "connection" has been defined which helps in the detection of three-set steerability. Once steerable sets are identified, their encoding becomes a straightforward procedure following the location of the identifying clusters on the concurrency matrix or matrices.

Effect of mechanical cycling on the stress-strain response of a martensitic Nitinol shape memory alloy

An experimental investigation into the ambient temperature, load-controlled tension�tension fatigue behavior of a martensitic Nitinol shape memory alloy (SMA) was conducted. Fatigue life for several stress levels spanning the critical stress for detwinning was determined and compared with that obtained on an alloy similar in composition but in the austenitic state at room temperature. Results show that the fatigue life of the pseudo-plastic alloy is superior to superelastic shape memory alloy. The stress�strain hysteretic response, monitored throughout the fatigue loading, reveals progressive strain accumulation with the cyclic loading. In addition, the area of hysteresis and recoverable and frictional energies were found to decrease with increasing number of fatigue cycles. Post-mortem characterization of the fatigued specimens through calorimetry and fractography was conducted in order to get further insight into the fatigue micromechanisms. These results are discussed in terms of reversible and irreversible microstructural changes that take place during cyclic loading. Aspects associated with self-heating of martensitic alloy undergoing high frequency stress cycling are discussed.