Description and training of neural network dynamics

Attractor properties of a popular discrete-time neural network model are illustrated through numerical simulations. The most complex dynamics is found to occur within particular ranges of parameters controlling the symmetry and magnitude of the weight matrix. A small network model is observed to produce fixed points, limit cycles, mode-locking, the Ruelle-Takens route to chaos, and the period-doubling route to chaos. Training algorithms for tuning this dynamical behaviour are discussed. Training can be an easy or difficult task, depending whether the problem requires the use of temporal information distributed over long time intervals. Such problems require training algorithms which can handle hidden nodes. The most prominent of these algorithms, back propagation through time, solves the temporal credit assignment problem in a way which can work only if the relevant information is distributed locally in time. The Moving Targets algorithm works for the more general case, but is computationally intensive, and prone to local minima.

Dynamics of on-line gradient descent learning for multilayer neural networks

We consider the problem of on-line gradient descent learning for general two-layer neural networks. An analytic solution is presented and used to investigate the role of the learning rate in controlling the evolution and convergence of the learning process.

The dynamics of a genetic algorithm for a simple learning problem

A formalism for describing the dynamics of Genetic Algorithms (GAs) using method s from statistical mechanics is applied to the problem of generalization in a perceptron with binary weights. The dynamics are solved for the case where a new batch of training patterns is presented to each population member each generation, which considerably simplifies the calculation. The theory is shown to agree closely to simulations of a real GA averaged over many runs, accurately predicting the mean best solution found. For weak selection and large problem size the difference equations describing the dynamics can be expressed analytically and we find that the effects of noise due to the finite size of each training batch can be removed by increasing the population size appropriately. If this population resizing is used, one can deduce the most computationally efficient size of training batch each generation. For independent patterns this choice also gives the minimum total number of training patterns used. Although using independent patterns is a very inefficient use of training patterns in general, this work may also prove useful for determining the optimum batch size in the case where patterns are recycled.

The dynamics of a genetic algorithm under stabilizing selection

A formalism recently introduced by Prugel-Bennett and Shapiro uses the methods of statistical mechanics to model the dynamics of genetic algorithms. To be of more general interest than the test cases they consider. In this paper, the technique is applied to the subset sum problem, which is a combinatorial optimization problem with a strongly non-linear energy (fitness) function and many local minima under single spin flip dynamics. It is a problem which exhibits an interesting dynamics, reminiscent of stabilizing selection in population biology. The dynamics are solved under certain simplifying assumptions and are reduced to a set of difference equations for a small number of relevant quantities. The quantities used are the population's cumulants, which describe its shape, and the mean correlation within the population, which measures the microscopic similarity of population members. Including the mean correlation allows a better description of the population than the cumulants alone would provide and represents a new and important extension of the technique. The formalism includes finite population effects and describes problems of realistic size. The theory is shown to agree closely to simulations of a real genetic algorithm and the mean best energy is accurately predicted.

Modelling the dynamics of genetic algorithms using statistical mechanics

A formalism for modelling the dynamics of Genetic Algorithms (GAs) using methods from statistical mechanics, originally due to Prugel-Bennett and Shapiro, is reviewed, generalized and improved upon. This formalism can be used to predict the averaged trajectory of macroscopic statistics describing the GA's population. These macroscopics are chosen to average well between runs, so that fluctuations from mean behaviour can often be neglected. Where necessary, non-trivial terms are determined by assuming maximum entropy with constraints on known macroscopics. Problems of realistic size are described in compact form and finite population effects are included, often proving to be of fundamental importance. The macroscopics used here are cumulants of an appropriate quantity within the population and the mean correlation (Hamming distance) within the population. Including the correlation as an explicit macroscopic provides a significant improvement over the original formulation. The formalism is applied to a number of simple optimization problems in order to determine its predictive power and to gain insight into GA dynamics. Problems which are most amenable to analysis come from the class where alleles within the genotype contribute additively to the phenotype. This class can be treated with some generality, including problems with inhomogeneous contributions from each site, non-linear or noisy fitness measures, simple diploid representations and temporally varying fitness. The results can also be applied to a simple learning problem, generalization in a binary perceptron, and a limit is identified for which the optimal training batch size can be determined for this problem. The theory is compared to averaged results from a real GA in each case, showing excellent agreement if the maximum entropy principle holds. Some situations where this approximation brakes down are identified. In order to fully test the formalism, an attempt is made on the strong sc np-hard problem of storing random patterns in a binary perceptron. Here, the relationship between the genotype and phenotype (training error) is strongly non-linear. Mutation is modelled under the assumption that perceptron configurations are typical of perceptrons with a given training error. Unfortunately, this assumption does not provide a good approximation in general. It is conjectured that perceptron configurations would have to be constrained by other statistics in order to accurately model mutation for this problem. Issues arising from this study are discussed in conclusion and some possible areas of further research are outlined.

Structure, dynamics, and energetics of siRNA-cationic vector complexation:a molecular dynamics study

The design and synthesis of safe and efficient nonviral vectors for gene delivery has attracted significant attention in recent years. Previous experiments have revealed that the charge density of a polycation (the carrier) plays a crucial role in complexation and the release of the gene from the complex in the cytosol. In this work, we adopt an atomistic molecular dynamics simulation approach to study the complexation of short strand duplex RNA with six cationic carrier systems of varying charge and surface topology. The simulations reveal detailed molecular-level pictures of the structures and dynamics of the RNA-polycation complexes. Estimates for the binding free energy indicate that electrostatic contributions are dominant followed by van der Waals interactions. The binding free energy between the 8(+)polymers and the RNA is found to be larger than that of the 4(+)polymers, in general agreement with previously published data. Because reliable binding free energies provide an effective index of the ability of the polycationic carrier to bind the nucleic acid and also carry implications for the process of gene release within the cytosol, these novel simulations have the potential to provide us with a much better understanding of key mechanistic aspects of gene-polycation complexation and thereby advance the rational design of nonviral gene delivery systems.

Lattice approach to the dynamics of phase-coded soliton trains

We address the collective dynamics of a soliton train propagating in a medium described by the nonlinear Schrödinger equation. Our approach uses the reduction of train dynamics to the discrete complex Toda chain (CTC) model for the evolution of parameters for each train constituent: such a simplification allows one to carry out an approximate analysis of the dynamics of positions and phases of individual interacting pulses. Here, we employ the CTC model to the problem which has relevance to the field of fibre optics communications where each binary digit of transmitted information is encoded via the phase difference between the two adjacent solitons. Our goal is to elucidate different scenarios of the train distortions and the subsequent information garbling caused solely by the intersoliton interactions. First, we examine how the structure of a given phase pattern affects the initial stage of the train dynamics and explain the general mechanisms for the appearance of unstable collective soliton modes. Then we further discuss the nonlinear regime concentrating on the dependence of the Lax scattering matrix on the input phase distribution; this allows one to classify typical features of the train evolution and determine the distance where the soliton escapes from its slot. In both cases, we demonstrate deep mathematical analogies with the classical theory of crystal lattice dynamics.

Dynamics and topographic organization of recursive self-organizing maps

Recently there has been an outburst of interest in extending topographic maps of vectorial data to more general data structures, such as sequences or trees. However, there is no general consensus as to how best to process sequences using topographicmaps, and this topic remains an active focus of neurocomputational research. The representational capabilities and internal representations of the models are not well understood. Here, we rigorously analyze a generalization of the self-organizingmap (SOM) for processing sequential data, recursive SOM (RecSOM) (Voegtlin, 2002), as a nonautonomous dynamical system consisting of a set of fixed input maps. We argue that contractive fixed-input maps are likely to produce Markovian organizations of receptive fields on the RecSOM map. We derive bounds on parameter β (weighting the importance of importing past information when processing sequences) under which contractiveness of the fixed-input maps is guaranteed. Some generalizations of SOM contain a dynamic module responsible for processing temporal contexts as an integral part of the model. We show that Markovian topographic maps of sequential data can be produced using a simple fixed (nonadaptable) dynamic module externally feeding a standard topographic model designed to process static vectorial data of fixed dimensionality (e.g., SOM). However, by allowing trainable feedback connections, one can obtain Markovian maps with superior memory depth and topography preservation. We elaborate on the importance of non-Markovian organizations in topographic maps of sequential data. © 2006 Massachusetts Institute of Technology.

The spatial distribution and temporal dynamics of brain regions activated during the perception of object and non-object patterns

Both animal and human studies suggest that the efficiency with which we are able to grasp objects is attributable to a repertoire of motor signals derived directly from vision. This is in general agreement with the long-held belief that the automatic generation of motor signals by the perception of objects is based on the actions they afford. In this study, we used magnetoencephalography (MEG) to determine the spatial distribution and temporal dynamics of brain regions activated during passive viewing of object and non-object targets that varied in the extent to which they afforded a grasping action. Synthetic Aperture Magnetometry (SAM) was used to localize task-related oscillatory power changes within specific frequency bands, and the time course of activity within given regions-of-interest was determined by calculating time-frequency plots using a Morlet wavelet transform. Both single subject and group-averaged data on the spatial distribution of brain activity are presented. We show that: (i) significant reductions in 10-25 Hz activity within extrastriate cortex, occipito-temporal cortex, sensori-motor cortex and cerebellum were evident with passive viewing of both objects and non-objects; and (ii) reductions in oscillatory activity within the posterior part of the superior parietal cortex (area Ba7) were only evident with the perception of objects. Assuming that focal reductions in low-frequency oscillations (< 30 Hz) reflect areas of heightened neural activity, we conclude that: (i) activity within a network of brain areas, including the sensori-motor cortex, is not critically dependent on stimulus type and may reflect general changes in visual attention; and (ii) the posterior part of the superior parietal cortex, area Ba7, is activated preferentially by objects and may play a role in computations related to grasping. © 2006 Elsevier Inc. All rights reserved.

Phase transitions and memory effects in the dynamics of Boolean networks

The generating functional method is employed to investigate the synchronous dynamics of Boolean networks, providing an exact result for the system dynamics via a set of macroscopic order parameters. The topology of the networks studied and its constituent Boolean functions represent the system's quenched disorder and are sampled from a given distribution. The framework accommodates a variety of topologies and Boolean function distributions and can be used to study both the noisy and noiseless regimes; it enables one to calculate correlation functions at different times that are inaccessible via commonly used approximations. It is also used to determine conditions for the annealed approximation to be valid, explore phases of the system under different levels of noise and obtain results for models with strong memory effects, where existing approximations break down. Links between Boolean networks and general Boolean formulas are identified and results common to both system types are highlighted. © 2012 Copyright Taylor and Francis Group, LLC.

Profit sharing in a manufacturing plant in a developing country : communication and organisational dynamics

The adaptation of profit sharing creates a fundamental change in employee compensation by making a portion of total compensation directly dependent upon the total profits of the firm and the performance of the employee. The major goal of this study is to test for and measure the impact of the independent variable, a profit sharing plan implemented at Shahvand Industrial Company, upon communication behaviour, communication outcomes, and organisational outcomes as dependent variables. A quasi-experimental non-equivalent control group design with pre and posttest was the research design used to test the effects of profit sharing participation on permanent-part-time operative employees implemented by SIC. Several conclusions were reached as a result of the statistical analysis of the data collected in this study. Overall, few of the hypothesised effects of profit sharing participation appeared to have been realised according to the empirical results of this study. The finding that certain communication behaviours were more favourable for profit sharing participants than for non-participants support the general hypothesis of the integrated profit sharing model. The observed changes in communication behaviours indicate that information sharing and idea generation are important components of the profit sharing process. The results of this study did not reveal any changes in either communication or organisational outcomes. A significant finding of this study is that the implementation of profit sharing plans require a relatively long period of time. Patience is required to achieve high levels of success and management must make long-term commitment to profit sharing. Findings of this study should be interpreted with caution, taking into consideration that most of the previo.us researches on profit sharing have been conducted in Western European or American countries, while the current study was based on data collected from an organisation in a developing country. This implies that the findings reported in this thesis may not be comparable in certain respects to results derived from companies in major industrialised economies.

An investigation into the nature, determinants, and dynamics of service quality expectations

On the basis of a review of the substantive quality and service marketing literature current knowledge regarding service quality expectations was found either absent or deficient. The phenomenon is of increasing importance to both marketing researchers and management and was therefore judged worthy of scholarly consideration. Because the service quality literature was insufficiently rich when embarking on the thesis three basic research issues were considered namely the nature, determinants, and dynamics of service quality expectations. These issues were first conceptually and then qualitatively explored. This process generated research hypotheses mainly relating to a model which were subsequently tested through a series of empirical investigations using questionnaire data from field studies in a single context. The results were internally consistent and strongly supported the main research hypotheses. It was found that service quality expectations can be meaningfully described in terms of generic/service-specific, intangible/tangible, and process/outcome categories. Service-specific quality expectations were also shown to be determined by generic service quality expectations, demographic variables, personal values, psychological needs, general service sophistication, service-specific sophistication, purchase motives, and service-specific information when treating service class involvement as an exogenous variable. Subjects who had previously not directly experienced a particular service were additionally found to revise their expectations of quality when exposed to the service with change being driven by a sub-set of identified determinants.

Applications of Reissner's principle to structural dynamics

The analysis and prediction of the dynamic behaviour of s7ructural components plays an important role in modern engineering design. :n this work, the so-called "mixed" finite element models based on Reissnen's variational principle are applied to the solution of free and forced vibration problems, for beam and :late structures. The mixed beam models are obtained by using elements of various shape functions ranging from simple linear to complex cubic and quadratic functions. The elements were in general capable of predicting the natural frequencies and dynamic responses with good accuracy. An isoparametric quadrilateral element with 8-nodes was developed for application to thin plate problems. The element has 32 degrees of freedom (one deflection, two bending and one twisting moment per node) which is suitable for discretization of plates with arbitrary geometry. A linear isoparametric element and two non-conforming displacement elements (4-node and 8-node quadrilateral) were extended to the solution of dynamic problems. An auto-mesh generation program was used to facilitate the preparation of input data required by the 8-node quadrilateral elements of mixed and displacement type. Numerical examples were solved using both the mixed beam and plate elements for predicting a structure's natural frequencies and dynamic response to a variety of forcing functions. The solutions were compared with the available analytical and displacement model solutions. The mixed elements developed have been found to have significant advantages over the conventional displacement elements in the solution of plate type problems. A dramatic saving in computational time is possible without any loss in solution accuracy. With beam type problems, there appears to be no significant advantages in using mixed models.

Modelling the dynamics of cutting during turning

This thesis presents an approach to cutting dynamics during turning based upon the mechanism of deformation of work material around the tool nose known as "ploughing". Starting from the shearing process in the cutting zone and accounting for "ploughing", new mathematical models relating turning force components to cutting conditions, tool geometry and tool vibration are developed. These models are developed separately for steady state and for oscillatory turning with new and worn tools. Experimental results are used to determine mathematical functions expressing the parameters introduced by the steady state model in the case of a new tool. The form of these functions are of general validity though their coefficients are dependent on work and tool materials. Good agreement is achieved between experimental and predicted forces. The model is extended on one hand to include different work material by introducing a hardness factor. The model provides good predictions when predicted forces are compared to present and published experimental results. On the other hand, the extension of the ploughing model to taming with a worn edge showed the ability of the model in predicting machining forces during steady state turning with the worn flank of the tool. In the development of the dynamic models, the dynamic turning force equations define the cutting process as being a system for which vibration of the tool tip in the feed direction is the input and measured forces are the output The model takes into account the shear plane oscillation and the cutting configuration variation in response to tool motion. Theoretical expressions of the turning forces are obtained for new and worn cutting edges. The dynamic analysis revealed the interaction between the cutting mechanism and the machine tool structure. The effect of the machine tool and tool post is accounted for by using experimental data of the transfer function of the tool post system. Steady state coefficients are corrected to include the changes in the cutting configuration with tool vibration and are used in the dynamic model. A series of oscillatory cutting tests at various conditions and various tool flank wear levels are carried out and experimental results are compared with model—predicted forces. Good agreement between predictions and experiments were achieved over a wide range of cutting conditions. This research bridges the gap between the analysis of vibration and turning forces in turning. It offers an explicit expression of the dynamic turning force generated during machining and highlights the relationships between tool wear, tool vibration and turning force. Spectral analysis of tool acceleration and turning force components led to define an "Inertance Power Ratio" as a flank wear monitoring factor. A formulation of an on—line flank wear monitoring methodology is presented and shows how the results of the present model can be applied to practical in—process tool wear monitoring in • turning operations.