Stability theory and numerical analysis of non-autonomous dynamical systems

The development and use of cocycles for analysis of non-autonomous behaviour is a technique that has been known for several years. Initially developed as an extension to semi-group theory for studying rion-autonornous behaviour, it was extensively used in analysing random dynamical systems [2, 9, 10, 12]. Many of the results regarding asymptotic behaviour developed for random dynamical systems, including the concept of cocycle attractors were successfully transferred and reinterpreted for deterministic non-autonomous systems primarily by P. Kloeden and B. Schmalfuss [20, 21, 28, 29]. The theory concerning cocycle attractors was later developed in various contexts specific to particular classes of dynamical systems [6, 7, 13], although a comprehensive understanding of cocycle attractors (redefined as pullback attractors within this thesis) and their role in the stability of non-autonomous dynamical systems was still at this stage incomplete. It was this purpose that motivated Chapters 1-3 to define and formalise the concept of stability within non-autonomous dynamical systems. The approach taken incorporates the elements of classical asymptotic theory, and refines the notion of pullback attraction with further development towards a study of pull-back stability arid pullback asymptotic stability. In a comprehensive manner, it clearly establishes both pullback and forward (classical) stability theory as fundamentally unique and essential components of non-autonomous stability. Many of the introductory theorems and examples highlight the key properties arid differences between pullback and forward stability. The theory also cohesively retains all the properties of classical asymptotic stability theory in an autonomous environment. These chapters are intended as a fundamental framework from which further research in the various fields of non-autonomous dynamical systems may be extended. A preliminary version of a Lyapunov-like theory that characterises pullback attraction is created as a tool for examining non-autonomous behaviour in Chapter 5. The nature of its usefulness however is at this stage restricted to the converse theorem of asymptotic stability. Chapter 7 introduces the theory of Loci Dynamics. A transformation is made to an alternative dynamical system where forward asymptotic (classical asymptotic) behaviour characterises pullback attraction to a particular point in the original dynamical system. This has the advantage in that certain conventional techniques for a forward analysis may be applied. The remainder of the thesis, Chapters 4, 6 and Section 7.3, investigates the effects of perturbations and discretisations on non-autonomous dynamical systems known to possess structures that exhibit some form of stability or attraction. Chapter 4 investigates autonomous systems with semi-group attractors, that have been non-autonomously perturbed, whilst Chapter 6 observes the effects of discretisation on non-autonomous dynamical systems that exhibit properties of forward asymptotic stability. Chapter 7 explores the same problem of discretisation, but for pullback asymptotically stable systems. The theory of Loci Dynamics is used to analyse the nature of the discretisation, but establishment of results directly analogous to those discovered in Chapter 6 is shown to be unachievable. Instead a case by case analysis is provided for specific classes of dynamical systems, for which the results generate a numerical approximation of the pullback attraction in the original continuous dynamical system. The nature of the results regarding discretisation provide a non-autonomous extension to the work initiated by A. Stuart and J. Humphries [34, 35] for the numerical approximation of semi-group attractors within autonomous systems. . Of particular importance is the effect on the system's asymptotic behaviour over non-finite intervals of discretisation.

Parallelization of the discrete gradient method of non-smooth optimization and its applications

We investigate parallelization and performance of the discrete gradient method of nonsmooth optimization. This derivative free method is shown to be an effective optimization tool, able to skip many shallow local minima of nonconvex nondifferentiable objective functions. Although this is a sequential iterative method, we were able to parallelize critical steps of the algorithm, and this lead to a significant improvement in performance on multiprocessor computer clusters. We applied this method to a difficult polyatomic clusters problem in computational chemistry, and found this method to outperform other algorithms.

Dynamical systems for discovering protein complexes and functional modules from biological networks

Recent advances in high throughput experiments and annotations via published literature have provided a wealth of interaction maps of several biomolecular networks, including metabolic, protein-protein, and protein-DNA interaction networks. The architecture of these molecular networks reveals important principles of cellular organization and molecular functions. Analyzing such networks, i.e., discovering dense regions in the network, is an important way to identify protein complexes and functional modules. This task has been formulated as the problem of finding heavy subgraphs, the Heaviest k-Subgraph Problem (k-HSP), which itself is NPhard. However, any method based on the k-HSP requires the parameter k and an exact solution of k-HSP may still end up as a “spurious” heavy subgraph, thus reducing its practicability in analyzing large scale biological networks. We proposed a new formulation, called the rank-HSP, and two dynamical systems to approximate its results. In addition, a novel metric, called the Standard deviation and Mean Ratio (SMR), is proposed for use in “spurious” heavy subgraphs to automate the discovery by setting a fixed threshold. Empirical results on both the simulated graphs and biological networks have demonstrated the efficiency and effectiveness of our proposal.

Non-smooth optimization methods for computation of the conditional value-at-risk and portfolio optimization

We examine numerical performance of various methods of calculation of the Conditional Value-at-risk (CVaR), and portfolio optimization with respect to this risk measure. We concentrate on the method proposed by Rockafellar and Uryasev in (Rockafellar, R.T. and Uryasev, S., 2000, Optimization of conditional value-at-risk. Journal of Risk, 2, 21-41), which converts this problem to that of convex optimization. We compare the use of linear programming techniques against a non-smooth optimization method of the discrete gradient, and establish the supremacy of the latter. We show that non-smooth optimization can be used efficiently for large portfolio optimization, and also examine parallel execution of this method on computer clusters.

Spatial discretization of discrete dynamical systems

Investigates what can go wrong when dynamical systems are modelled with a computer. Number theoretic techniques were used to detail the effects "discretization" errors caused by computer round-off had on characteristics of a system. In particular, a relationship was established between the occurrence of long cycles in a system and the classical result known as Artin's conjecture. Algorithms were then developed which eliminated discretization errors.

Functional observers for dynamical systems

In control theory, a state observer is an auxiliary dynamical system that mirrors the behaviour of a physical system, and it is driven by input and output measurements of the physical system in order to provide an estimate of internal states of the physical system. The primary consideration in the design of an observer is that the estimate of the states should be close to the actual value of the system states. On the other hand, the functional observation problem centers on the construction of an auxiliary dynamical system, known as the functional observer or functional reconstructor, driven by the available system inputs and outputs in order to estimate a linear function or functions of the system states. Obviously, a functional observer is a general form of the state observer because when the linear functions are chosen as the individual states of the system then the problem of functional observation reduces to the problem of state observation.

Global non-smooth optimization in robust multivariate regression

Robust regression in statistics leads to challenging optimization problems. Here, we study one such problem, in which the objective is non-smooth, non-convex and expensive to calculate. We study the numerical performance of several derivative-free optimization algorithms with the aim of computing robust multivariate estimators. Our experiences demonstrate that the existing algorithms often fail to deliver optimal solutions. We introduce three new methods that use Powell's derivative-free algorithm. The proposed methods are reliable and can be used when processing very large data sets containing outliers.

A fault detection technique for dynamical systems

This paper shows how a functional observer can be utilized to detect faults in LTI MIMO systems. The fault detection technique is designed so that the functional observer based fault indicator asymptotically converges to a fault indicator that can be derived based on the nominal system. The asymptotic value of the proposed fault indicator is not dependent on the functional observer parameters; moreover, by choosing appropriate functional observer parameters the convergence rate of the fault indicator can be altered. Observability of the system is not a requirement for the design of the fault detection scheme.

Fault detection for dynamical systems using first-order functional observers

This paper reports a new result on the fault detection of dynamical systems by employing only first-order functional observers. Indeed, we show that fault detection can be achieved by utilizing first-order functional observers. The advantages for having such simple structured observers are obvious from the economical and practical points of view as significant cost saving can be achieved. We derive existence conditions and an algorithm for the generation of residual signals to detect faults using firstorder functional observers. Two numerical examples are given to illustrate the proposed fault detection scheme. In one of the examples, a two-area interconnected power system with reheat thermal turbines is considered where only a first-order functional observer is designed to detect faults in the power system.

Evaluating non-deterministic retrieval systems

The use of sampling, randomized algorithms, or training based on the unpredictable inputs of users in Information Retrieval often leads to non-deterministic outputs. Evaluating the effectiveness of systems incorporating these methods can be challenging since each run may produce different effectiveness scores. Current IR evaluation techniques do not address this problem. Using the context of distributed information retrieval as a case study for our investigation, we propose a solution based on multivariate linear modeling. We show that the approach provides a consistent and reliable method to compare the effectiveness of non-deterministic IR algorithms, and explain how statistics can safely be used to show that two IR algorithms have equivalent effectiveness. Copyright 2014 ACM.

Statistical comparisons of non-deterministic IR systems using two dimensional variance

Retrieval systems with non-deterministic output are widely used in information retrieval. Common examples include sampling, approximation algorithms, or interactive user input. The effectiveness of such systems differs not just for different topics, but also for different instances of the system. The inherent variance presents a dilemma - What is the best way to measure the effectiveness of a non-deterministic IR system? Existing approaches to IR evaluation do not consider this problem, or the potential impact on statistical significance. In this paper, we explore how such variance can affect system comparisons, and propose an evaluation framework and methodologies capable of doing this comparison. Using the context of distributed information retrieval as a case study for our investigation, we show that the approaches provide a consistent and reliable methodology to compare the effectiveness of a non-deterministic system with a deterministic or another non-deterministic system. In addition, we present a statistical best-practice that can be used to safely show how a non-deterministic IR system has equivalent effectiveness to another IR system, and how to avoid the common pitfall of misusing a lack of significance as a proof that two systems have equivalent effectiveness.

A functional observer based fault detection technique for dynamical systems

This paper presents a functional observer based fault detection method. The fault detection is achieved using a functional observer based fault indicator that asymptotically converges to a fault indicator that can be derived based on the nominal system. The asymptotic value of the proposed fault indicator is independent of the functional observer parameters and also the convergence rate of the fault indicator can be altered by choosing appropriate functional observer parameters. The advantage of using this new method is that the observed system is not necessarily needed to be observable; therefore, the proposed fault detection technique is also applicable for systems where state observers cannot be designed; moreover, the functional observer fault detection scheme is always of reduced order in comparison to a state observer based scheme.

Minimal-order functional observer-based residual generators for fault detection and isolation of dynamical systems

This paper examines the design of minimal-order residual generators for the purpose of detecting and isolating actuator and/or component faults in dynamical systems. We first derive existence conditions and design residual generators using only first-order observers to detect and identify the faults. When the first-order functional observers do not exist, then based on a parametric approach to the solution of a generalized Sylvester matrix equation, we develop systematic procedures for designing residual generators utilizing minimal-order functional observers. Our design approach gives lower-order residual generators than existing results in the literature. The advantages for having such lower-order residual generators are obvious from the economical and practical points of view as cost saving and simplicity in implementation can be achieved, particularly when dealing with high-order complex systems. Numerical examples are given to illustrate the proposed fault detection and isolation schemes. In all of the numerical examples, we design minimum-order residual generators to effectively detect and isolate actuator and/or component faults in the system.

Partial state bounding with a pre-specified time of non-linear discrete systems with time-varying delays

In this study, the authors address a new problem of finding, with a pre-specified time, bounds of partial states of non-linear discrete systems with a time-varying delay. A novel computational method for deriving the smallest bounds is presented. The method is based on a new comparison principle, a new algorithm for finding the infimum of a fractal function, and linear programming. The effectiveness of our obtained results is illustrated through a numerical example.

A fast non-smooth nonnegative matrix factorization for learning sparse representation

Nonnegative matrix factorization (NMF) is a hot topic in machine learning and data processing. Recently, a constrained version, non-smooth NMF (NsNMF), shows a great potential in learning meaningful sparse representation of the observed data. However, it suffers from a slow linear convergence rate, discouraging its applications to large-scale data representation. In this paper, a fast NsNMF (FNsNMF) algorithm is proposed to speed up NsNMF. In the proposed method, it first shows that the cost function of the derived sub-problem is convex and the corresponding gradient is Lipschitz continuous. Then, the optimization to this function is replaced by solving a proximal function, which is designed based on the Lipschitz constant and can be solved through utilizing a constructed fast convergent sequence. Due to the usage of the proximal function and its efficient optimization, our method can achieve a nonlinear convergence rate, much faster than NsNMF. Simulations in both computer generated data and the real-world data show the advantages of our algorithm over the compared methods.