The Synthesis of Arbitrary Stable Dynamics in Non-linear Neural Networks II: Feedback and Universality

We wish to construct a realization theory of stable neural networks and use this theory to model the variety of stable dynamics apparent in natural data. Such a theory should have numerous applications to constructing specific artificial neural networks with desired dynamical behavior. The networks used in this theory should have well understood dynamics yet be as diverse as possible to capture natural diversity. In this article, I describe a parameterized family of higher order, gradient-like neural networks which have known arbitrary equilibria with unstable manifolds of known specified dimension. Moreover, any system with hyperbolic dynamics is conjugate to one of these systems in a neighborhood of the equilibrium points. Prior work on how to synthesize attractors using dynamical systems theory, optimization, or direct parametric. fits to known stable systems, is either non-constructive, lacks generality, or has unspecified attracting equilibria. More specifically, We construct a parameterized family of gradient-like neural networks with a simple feedback rule which will generate equilibrium points with a set of unstable manifolds of specified dimension. Strict Lyapunov functions and nested periodic orbits are obtained for these systems and used as a method of synthesis to generate a large family of systems with the same local dynamics. This work is applied to show how one can interpolate finite sets of data, on nested periodic orbits.

Broadband chaos generated by an optoelectronic oscillator.

We study an optoelectronic time-delay oscillator that displays high-speed chaotic behavior with a flat, broad power spectrum. The chaotic state coexists with a linearly stable fixed point, which, when subjected to a finite-amplitude perturbation, loses stability initially via a periodic train of ultrafast pulses. We derive approximate mappings that do an excellent job of capturing the observed instability. The oscillator provides a simple device for fundamental studies of time-delay dynamical systems and can be used as a building block for ultrawide-band sensor networks.

Invariant measure selection by noise. An example

We consider a deterministic system with two conserved quantities and infinity many invariant measures. However the systems possess a unique invariant measure when enough stochastic forcing and balancing dissipation are added. We then show that as the forcing and dissipation are removed a unique limit of the deterministic system is selected. The exact structure of the limiting measure depends on the specifics of the stochastic forcing.

Sensitivity to switching rates in stochastically switched ODEs

We consider a stochastic process driven by a linear ordinary differential equation whose right-hand side switches at exponential times between a collection of different matrices. We construct planar examples that switch between two matrices where the individual matrices and the average of the two matrices are all Hurwitz (all eigenvalues have strictly negative real part), but nonetheless the process goes to infinity at large time for certain values of the switching rate. We further construct examples in higher dimensions where again the two individual matrices and their averages are all Hurwitz, but the process has arbitrarily many transitions between going to zero and going to infinity at large time as the switching rate varies. In order to construct these examples, we first prove in general that if each of the individual matrices is Hurwitz, then the process goes to zero at large time for sufficiently slow switching rate and if the average matrix is Hurwitz, then the process goes to zero at large time for sufficiently fast switching rate. We also give simple conditions that ensure the process goes to zero at large time for all switching rates. © 2014 International Press.

Stochastic switching in infinite dimensions with applications to random parabolic PDE

© 2015 Society for Industrial and Applied Mathematics.We consider parabolic PDEs with randomly switching boundary conditions. In order to analyze these random PDEs, we consider more general stochastic hybrid systems and prove convergence to, and properties of, a stationary distribution. Applying these general results to the heat equation with randomly switching boundary conditions, we find explicit formulae for various statistics of the solution and obtain almost sure results about its regularity and structure. These results are of particular interest for biological applications as well as for their significant departure from behavior seen in PDEs forced by disparate Gaussian noise. Our general results also have applications to other types of stochastic hybrid systems, such as ODEs with randomly switching right-hand sides.

Positive dilations of piecewise monotonic Markov maps

We provide an explicit formula which gives natural extensions of piecewise monotonic Markov maps defined on an interval of the real line. These maps are exact endomorphisms and define chaotic discrete dynamical systems.

Extended spectral decompositions of the Renyi map

We present new general methods to obtain spectral decompositions of dynamical systems in rigged Hilbert spaces and investigate the existence of resonances and the completeness of the associated eigenfunctions. The results are illustrated explicitly for the simplest chaotic endomorphism, namely the Renyi map.

Learning soft computing control strategies in a modular neural network architecture

Modelling and control of nonlinear dynamical systems is a challenging problem since the dynamics of such systems change over their parameter space. Conventional methodologies for designing nonlinear control laws, such as gain scheduling, are effective because the designer partitions the overall complex control into a number of simpler sub-tasks. This paper describes a new genetic algorithm based method for the design of a modular neural network (MNN) control architecture that learns such partitions of an overall complex control task. Here a chromosome represents both the structure and parameters of an individual neural network in the MNN controller and a hierarchical fuzzy approach is used to select the chromosomes required to accomplish a given control task. This new strategy is applied to the end-point tracking of a single-link flexible manipulator modelled from experimental data. Results show that the MNN controller is simple to design and produces superior performance compared to a single neural network (SNN) controller which is theoretically capable of achieving the desired trajectory. (C) 2003 Elsevier Ltd. All rights reserved.

Direct, positive feedbacks produce instability in models of interrelationships among soil structure, plants and arbuscular mycorrhizal fungi

Current conceptual models of reciprocal interactions linking soil structure, plants and arbuscular mycorrhizal fungi emphasise positive feedbacks among the components of the system. However, dynamical systems with high dimensionality and several positive feedbacks (i.e. mutualism) are prone to instability. Further, organisms such as arbuscular mycorrhizal fungi (AMF) are obligate biotrophs of plants and are considered major biological agents in soil aggregate stabilization. With these considerations in mind, we developed dynamical models of soil ecosystems that reflect the main features of current conceptual models and empirical data, especially positive feedbacks and linear interactions among plants, AMF and the component of soil structure dependent on aggregates. We found that systems become increasingly unstable the more positive effects with Type I functional response (i.e., the growth rate of a mutualist is modified by the density of its partner through linear proportionality) are added to the model, to the point that increasing the realism of models by adding linear effects produces the most unstable systems. The present theoretical analysis thus offers a framework for modelling and suggests new directions for experimental studies on the interrelationship between soil structure, plants and AMF. Non-linearity in functional responses, spatial and temporal heterogeneity, and indirect effects can be invoked on a theoretical basis and experimentally tested in laboratory and field experiments in order to account for and buffer the local instability of the simplest of current scenarios. This first model presented here may generate interest in more explicitly representing the role of biota in soil physical structure, a phenomenon that is typically viewed in a more process- and management-focused context. (C) 2011 Elsevier Ltd. All rights reserved.

Overcoming the Scalability Challenges of Epidemic Simulations on Blue Waters

Modeling dynamical systems represents an important application class covering a wide range of disciplines including but not limited to biology, chemistry, finance, national security, and health care. Such applications typically involve large-scale, irregular graph processing, which makes them difficult to scale due to the evolutionary nature of their workload, irregular communication and load imbalance. EpiSimdemics is such an application simulating epidemic diffusion in extremely large and realistic social contact networks. It implements a graph-based system that captures dynamics among co-evolving entities. This paper presents an implementation of EpiSimdemics in Charm++ that enables future research by social, biological and computational scientists at unprecedented data and system scales. We present new methods for application-specific processing of graph data and demonstrate the effectiveness of these methods on a Cray XE6, specifically NCSA's Blue Waters system.

Prediction of limit cycle oscillations under uncertainty using a Harmonic Balance method

The Harmonic Balance method is an attractive solution for computing periodic responses and can be an alternative to time domain methods, at a reduced computational cost. The current paper investigates using a Harmonic Balance method for simulating limit cycle oscillations under uncertainty. The Harmonic Balance method is used in conjunction with a non-intrusive polynomial-chaos approach to propagate variability and is validated against Monte Carlo analysis. Results show the potential of the approach for a range of nonlinear dynamical systems, including a full wing configuration exhibiting supercritical and subcritical bifurcations, at a fraction of the cost of performing time domain simulations.

Just a matter of time: Fungi and roots significantly and rapidly aggregate soil over four decades along the Tagliamento River, NE Italy

Fluvial islands are emergent landforms which form at the interface between the permanently inundated areas of the river channel and the more stable areas of the floodplain as a result of interactions between physical river processes, wood and riparian vegetation. These highly dynamical systems are ideal to study soil structure development in the short to medium term, a process in which soil biota and plants play a substantial role. We investigated soil structure development on islands along a 40 year chronosequence within a 3 km island-braided reach of the Tagliamento River, Northeastern Italy. We used several parameters to capture different aspects of the soil structure, and measured biotic (e.g., fungal and plant root parameters) and abiotic (e.g. organic carbon) factors expected to determine the structure. We estimated models relating soil structure to its determinants, and, in order to confer statistical robustness to our results, we explicitly took into account spatial autocorrelation, which is present due to the space for time substitution inherent in the study of chronosequences and may have confounded results of previous studies. We found that, despite the eroding forces from the hydrological and geomorphological dynamics to which the system is subject, all soil structure variables significantly, and in some case greatly increased with site age. We interpret this as a macroscopic proxy for the major direct and indirect binding effects exerted by root variables and extraradical hyphae of arbuscular mycorrhizal fungi (AMF). Key soil structure parameters such as percentage of water stable aggregates (WSA) can double from the time the island landform is initiated (mean WSA = 30%) to the full 40 years (mean WSA = 64%) covered by our chronosequence. The study demonstrates the fundamental role of soil biota and plant roots in aggregating soils even in a system in which intense short to medium term physical disturbances are common.

A compendium of modelling techniques

This Integration Insight provides a brief overview of the most popular modelling techniques used to analyse complex real-world problems, as well as some less popular but highly relevant techniques. The modelling methods are divided into three categories, with each encompassing a number of methods, as follows: 1) Qualitative Aggregate Models (Soft Systems Methodology, Concept Maps and Mind Mapping, Scenario Planning, Causal (Loop) Diagrams), 2) Quantitative Aggregate Models (Function fitting and Regression, Bayesian Nets, System of differential equations / Dynamical systems, System Dynamics, Evolutionary Algorithms) and 3) Individual Oriented Models (Cellular Automata, Microsimulation, Agent Based Models, Discrete Event Simulation, Social Network
Analysis). Each technique is broadly described with example uses, key attributes and reference material.

Fractional dynamics and MDS visualization of earthquake phenomena

This paper analyses earthquake data in the perspective of dynamical systems and fractional calculus (FC). This new standpoint uses Multidimensional Scaling (MDS) as a powerful clustering and visualization tool. FC extends the concepts of integrals and derivatives to non-integer and complex orders. MDS is a technique that produces spatial or geometric representations of complex objects, such that those objects that are perceived to be similar in some sense are placed on the MDS maps forming clusters. In this study, over three million seismic occurrences, covering the period from January 1, 1904 up to March 14, 2012 are analysed. The events are characterized by their magnitude and spatiotemporal distributions and are divided into fifty groups, according to the Flinn–Engdahl (F–E) seismic regions of Earth. Several correlation indices are proposed to quantify the similarities among regions. MDS maps are proven as an intuitive and useful visual representation of the complex relationships that are present among seismic events, which may not be perceived on traditional geographic maps. Therefore, MDS constitutes a valid alternative to classic visualization tools for understanding the global behaviour of earthquakes.