The 'moving targets' training algorithm

A simple method for training the dynamical behavior of a neural network is derived. It is applicable to any training problem in discrete-time networks with arbitrary feedback. The method resembles back-propagation in that it is a least-squares, gradient-based optimization method, but the optimization is carried out in the hidden part of state space instead of weight space. A straightforward adaptation of this method to feedforward networks offers an alternative to training by conventional back-propagation. Computational results are presented for simple dynamical training problems, with varied success. The failures appear to arise when the method converges to a chaotic attractor. A patch-up for this problem is proposed. The patch-up involves a technique for implementing inequality constraints which may be of interest in its own right.

Vector solitons in mode locked fibre lasers

We overview our recent results on polarisation dynamics of vector solitons in erbium doped fibre laser mode locked with carbon nanotubes. Our experimental and theoretical study revealed new families of vector solitons for fundamental and bound-state soliton operations. The observed scenario of the evolution of the states of polarisation (SOPs) on the Poincare sphere includes fast polarisation switching between two and three SOPs along with slow SOP evolution on a double scroll chaotic attractor. The underlying physics presents an interplay between effects of birefringence of the laser cavity and light induced anisotropy caused by polarisation hole burning. © 2014 IEEE.

Spiral attractor created by vector solitons

Mode-locked lasers emitting a train of femtosecond pulses called dissipative solitons are an enabling technology for metrology, high-resolution spectroscopy, fibre optic communications, nano-optics and many other fields of science and applications. Recently, the vector nature of dissipative solitons has been exploited to demonstrate mode locked lasing with both locked and rapidly evolving states of polarisation. Here, for an erbium-doped fibre laser mode locked with carbon nanotubes, we demonstrate the first experimental and theoretical evidence of a new class of slowly evolving vector solitons characterized by a double-scroll chaotic polarisation attractor substantially different from Lorenz, Rössler and Ikeda strange attractors. The underlying physics comprises a long time scale coherent coupling of two polarisation modes. The observed phenomena, apart from the fundamental interest, provide a base for advances in secure communications, trapping and manipulation of atoms and nanoparticles, control of magnetisation in data storage devices and many other areas. © 2014 CIOMP. All rights reserved.

Wave chaotic behaviour generated by linear systems

It is shown that regimes with dynamical chaos are inherent not only to nonlinear system but they can be generated by initially linear systems and the requirements for chaotic dynamics and characteristics need further elaboration. Three simplest physical models are considered as examples. In the first, dynamic chaos in the interaction of three linear oscillators is investigated. Analogous process is shown in the second model of electromagnetic wave scattering in a double periodical inhomogeneous medium occupying half-space. The third model is a linear parametric problem for the electromagnetic field in homogeneous dielectric medium which permittivity is modulated in time. © 2008 Springer Science+Business Media, LLC.

Investigation of chaotic mixing in laminar fluid systems by the use of computational fluid dynamics

This work presents significant development into chaotic mixing induced through periodic boundaries and twisting flows. Three-dimensional closed and throughput domains are shown to exhibit chaotic motion under both time periodic and time independent boundary motions, A property is developed originating from a signature of chaos, sensitive dependence to initial conditions, which successfully quantifies the degree of disorder withjn the mixing systems presented and enables comparisons of the disorder throughout ranges of operating parameters, This work omits physical experimental results but presents significant computational investigation into chaotic systems using commercial computational fluid dynamics techniques. Physical experiments with chaotic mixing systems are, by their very nature, difficult to extract information beyond the recognition that disorder does, does not of partially occurs. The initial aim of this work is to observe whether it is possible to accurately simulate previously published physical experimental results through using commercial CFD techniques. This is shown to be possible for simple two-dimensional systems with time periodic wall movements. From this, and subsequent macro and microscopic observations of flow regimes, a simple explanation is developed for how boundary operating parameters affect the system disorder. Consider the classic two-dimensional rectangular cavity with time periodic velocity of the upper and lower walls, causing two opposing streamline motions. The degree of disorder within the system is related to the magnitude of displacement of individual particles within these opposing streamlines. The rationale is then employed in this work to develop and investigate more complex three-dimensional mixing systems that exhibit throughputs and time independence and are therefore more realistic and a significant advance towards designing chaotic mixers for process industries. Domains inducing chaotic motion through twisting flows are also briefly considered. This work concludes by offering possible advancements to the property developed to quantify disorder and suggestions of domains and associated boundary conditions that are expected to produce chaotic mixing.

Dynamic complexity of chaotic transitions in high-dimensional classical dynamics:Leu-Enkephalin folding

Leu-Enkephalin in explicit water is simulated using classical molecular dynamics. A ß-turn transition is investigated by calculating the topological complexity (in the "computational mechanics" framework [J. P. Crutchfield and K. Young, Phys. Rev. Lett., 63, 105 (1989)]) of the dynamics of both the peptide and the neighbouring water molecules. The complexity of the atomic trajectories of the (relatively short) simulations used in this study reflect the degree of phase space mixing in the system. It is demonstrated that the dynamic complexity of the hydrogen atoms of the peptide and almost all of the hydrogens of the neighbouring waters exhibit a minimum precisely at the moment of the ß-turn transition. This indicates the appearance of simplified periodic patterns in the atomic motion, which could correspond to high-dimensional tori in the phase space. It is hypothesized that this behaviour is the manifestation of the effect described in the approach to molecular transitions by Komatsuzaki and Berry [T. Komatsuzaki and R.S. Berry, Adv. Chem. Phys., 123, 79 (2002)], where a "quasi-regular" dynamics at the transition is suggested. Therefore, for the first time, the less chaotic character of the folding transition in a realistic molecular system is demonstrated. © Springer-Verlag Berlin Heidelberg 2006.

Long-range and high-resolution correlation optical time-domain reflectometry utilizing an all optical chaotic source

A high resolution optical time domain reflectometry (OTDR) based on an all-fiber chaotic source is demonstrated. We analyze the key factors limiting the operational range of such an OTDR, e.g., integral Rayleigh backscattering and the fiber loss, which degrade the optical signal to noise ratio at the receiver side, and then the guideline for counter-act such signal fading is discussed. The experimentally demonstrated correlation OTDR presents ability of 100km sensing range and 8.2cm spatial resolution (1.2 million resolved points), as a verification of the theoretical analysis. To the best of our knowledge, this is the first time that correlation OTDR measurement is performed over such a long distance with such high precision.

High resolution optical time-domain reflectometry based on correlation utilizing an all-fiber chaotic source

We propose a high-resolution optical time domain reflectometry (OTDR) based on an all-fiber supercontinuum source. The source simply consists of a laser with moderate power and a section of fiber which has a zero dispersion wavelength near the laser's central wavelength. Spectrum and time domain properties of the source are investigated, showing that the source has great capability in nonlinear optics, such as correlation OTDR. We analyze one of the key factors limiting the operational range of such an OTDR, i.e., sampling time. Finally, we experimentally demonstrate a correlation OTDR with 25km sensing range and 5.3cm spatial resolution, as a verification of theoretical analysis.

Long-range and high-precision correlation optical time-domain reflectometry utilizing an all-fiber chaotic source

We propose a long range, high precision optical time domain reflectometry (OTDR) based on an all-fiber supercontinuum source. The source simply consists of a CW pump laser with moderate power and a section of fiber, which has a zero dispersion wavelength near the laser's central wavelength. Spectrum and time domain properties of the source are investigated, showing that the source has great capability in nonlinear optics, such as correlation OTDR due to its ultra-wide-band chaotic behavior, and mm-scale spatial resolution is demonstrated. Then we analyze the key factors limiting the operational range of such an OTDR, e. g., integral Rayleigh backscattering and the fiber loss, which degrades the optical signal to noise ratio at the receiver side, and then the guideline for counter-act such signal fading is discussed. Finally, we experimentally demonstrate a correlation OTDR with 100km sensing range and 8.2cm spatial resolution (1.2 million resolved points), as a verification of theoretical analysis.

Description and training of neural network dynamics

Attractor properties of a popular discrete-time neural network model are illustrated through numerical simulations. The most complex dynamics is found to occur within particular ranges of parameters controlling the symmetry and magnitude of the weight matrix. A small network model is observed to produce fixed points, limit cycles, mode-locking, the Ruelle-Takens route to chaos, and the period-doubling route to chaos. Training algorithms for tuning this dynamical behaviour are discussed. Training can be an easy or difficult task, depending whether the problem requires the use of temporal information distributed over long time intervals. Such problems require training algorithms which can handle hidden nodes. The most prominent of these algorithms, back propagation through time, solves the temporal credit assignment problem in a way which can work only if the relevant information is distributed locally in time. The Moving Targets algorithm works for the more general case, but is computationally intensive, and prone to local minima.

A new entropy measure based on the Renyi entropy rate using Gaussian kernels

The concept of entropy rate is well defined in dynamical systems theory but is impossible to apply it directly to finite real world data sets. With this in mind, Pincus developed Approximate Entropy (ApEn), which uses ideas from Eckmann and Ruelle to create a regularity measure based on entropy rate that can be used to determine the influence of chaotic behaviour in a real world signal. However, this measure was found not to be robust and so an improved formulation known as the Sample Entropy (SampEn) was created by Richman and Moorman to address these issues. We have developed a new, related, regularity measure which is not based on the theory provided by Eckmann and Ruelle and proves a more well-behaved measure of complexity than the previous measures whilst still retaining a low computational cost.

Approximate Bayesian techniques for inference in stochastic dynamical systems

This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.

Computational mechanics of molecular systems:quantifying high-dimensional dynamics by distribution of Poincaré recurrence times

A framework that connects computational mechanics and molecular dynamics has been developed and described. As the key parts of the framework, the problem of symbolising molecular trajectory and the associated interrelation between microscopic phase space variables and macroscopic observables of the molecular system are considered. Following Shalizi and Moore, it is shown that causal states, the constituent parts of the main construct of computational mechanics, the e-machine, define areas of the phase space that are optimal in the sense of transferring information from the micro-variables to the macro-observables. We have demonstrated that, based on the decay of their Poincare´ return times, these areas can be divided into two classes that characterise the separation of the phase space into resonant and chaotic areas. The first class is characterised by predominantly short time returns, typical to quasi-periodic or periodic trajectories. This class includes a countable number of areas corresponding to resonances. The second class includes trajectories with chaotic behaviour characterised by the exponential decay of return times in accordance with the Poincare´ theorem.

Emulation of dynamic computer models with multivariate output

This preliminary report describes work carried out as part of work package 1.2 of the MUCM research project. The report is split in two parts: the ?rst part (Sections 1 and 2) summarises the state of the art in emulation of computer models, while the second presents some initial work on the emulation of dynamic models. In the ?rst part, we describe the basics of emulation, introduce the notation and put together the key results for the emulation of models with single and multiple outputs, with or without the use of mean function. In the second part, we present preliminary results on the chaotic Lorenz 63 model. We look at emulation of a single time step, and repeated application of the emulator for sequential predic- tion. After some design considerations, the emulator is compared with the exact simulator on a number of runs to assess its performance. Several general issues related to emulating dynamic models are raised and discussed. Current work on the larger Lorenz 96 model (40 variables) is presented in the context of dimension reduction, with results to be provided in a follow-up report. The notation used in this report are summarised in appendix.

Nonlinear dynamics of a periodically driven Duffing resonator coupled to a Van der Pol oscillator

We explore the dynamics of a periodically driven Duffing resonator coupled elastically to a van der Pol oscillator in the case of 1?:?1 internal resonance in the cases of weak and strong coupling. Whilst strong coupling leads to dominating synchronization, the weak coupling case leads to a multitude of complex behaviours. A two-time scales method is used to obtain the frequency-amplitude modulation. The internal resonance leads to an antiresonance response of the Duffing resonator and a stagnant response (a small shoulder in the curve) of the van der Pol oscillator. The stability of the dynamic motions is also analyzed. The coupled system shows a hysteretic response pattern and symmetry-breaking facets. Chaotic behaviour of the coupled system is also observed and the dependence of the system dynamics on the parameters are also studied using bifurcation analysis.