Optimal mean-variance control for discrete-time linear systems with Markovian jumps and multiplicative noises

In this paper, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noises under two criteria. The first one is an unconstrained mean-variance trade-off performance criterion along the time, and the second one is a minimum variance criterion along the time with constraints on the expected output. We present explicit conditions for the existence of an optimal control strategy for the problems, generalizing previous results in the literature. We conclude the paper by presenting a numerical example of a multi-period portfolio selection problem with regime switching in which it is desired to minimize the sum of the variances of the portfolio along the time under the restriction of keeping the expected value of the portfolio greater than some minimum values specified by the investor. (C) 2011 Elsevier Ltd. All rights reserved.

Representations and algorithms for finite-state bisimulations of linear discrete-time control systems

While a large amount of research over the past two decades has focused on discrete abstractions of infinite-state dynamical systems, many structural and algorithmic details of these abstractions remain unknown. To clarify the computational resources needed to perform discrete abstractions, this paper examines the algorithmic properties of an existing method for deriving finite-state systems that are bisimilar to linear discrete-time control systems. We explicitly find the structure of the finite-state system, show that it can be enormous compared to the original linear system, and give conditions to guarantee that the finite-state system is reasonably sized and efficiently computable. Though constructing the finite-state system is generally impractical, we see that special cases could be amenable to satisfiability based verification techniques. ©2009 IEEE.

Anisotropy-based robust performance analysis of finite horizon linear discrete time varying systems

We consider a problem of robust performance analysis of linear discrete time varying systems on a bounded time interval. The system is represented in the state-space form. It is driven by a random input disturbance with imprecisely known probability distribution; this distributional uncertainty is described in terms of entropy. The worst-case performance of the system is quantified by its a-anisotropic norm. Computing the anisotropic norm is reduced to solving a set of difference Riccati and Lyapunov equations and a special form equation.

Linear functional observers with guaranteed ε-convergence for discrete time-delay systems with input/output disturbances

The problem of designing linear functional observers for discrete time-delay systems with unknown-but-bounded disturbances in both the plant and the output is considered for the first time in this paper. A novel approach to design a minimum-order observer is proposed to guarantee that the observer error is ϵ-convergent, which means that the estimate converges robustly within an ϵ-bound of the true state. Conditions for the existence of this observer are first derived. Then, by utilising an extended Lyapunov-Krasovskii functional and the free-weighting matrix technique, a sufficient condition for ϵ-convergence of the observer error system is given. This condition is presented in terms of linear matrix inequalities with two parameters needed to be tuned, so that it can be efficiently solved by incorporating a two-dimensional search method into convex optimisation algorithms to obtain the smallest possible value for ϵ. Three numerical examples, including the well-known single-link flexible joint robotic system, are given to illustrate the feasibility and effectiveness of our results.

New finite-sum inequalities with applications to stability of discrete time-delay systems

This paper is concerned with the problem of stability analysis of discrete time-delay systems. New finite-sum inequalities, which encompass the ones based on Abel lemma or Wirtinger type inequality, are first proposed. The potential capability of the newly derived inequalities is then demonstrated by establishing less conservative stability conditions for some classes of linear discrete-time systems with delay. The derived stability criteria are theoretically and numerically proved to be less conservative than existing results.

Componentwise ultimate bounds for positive discrete time-delay systems perturbed by interval disturbances

This paper presents a method to derive componentwise ultimate upper bounds and componentwise ultimate lower bounds for linear positive systems with time-varying delays and bounded disturbances. The disturbance vector is assumed to vary within a known interval whose lower bound may be different from zero. We first derive a sufficient condition for the existence of componentwise ultimate bounds. This condition is given in terms of the spectral radius of the system matrices which is easy to check and allows us to compute directly both the smallest componentwise ultimate upper bound and the largest componentwise ultimate lower bound. Then, by using the comparison method, we extend the obtained result to a class of nonlinear time-delay systems which has linear positive bounds. Two numerical examples are given to illustrate the effectiveness of the obtained results.

On optimal control of constrained linear systems with imperfect state information and stochastic disturbances

We address the problem of finite horizon optimal control of discrete-time linear systems with input constraints and uncertainty. The uncertainty for the problem analysed is related to incomplete state information (output feedback) and stochastic disturbances. We analyse the complexities associated with finding optimal solutions. We also consider two suboptimal strategies that could be employed for larger optimization horizons.

A critique of neural networks for discrete-time linear control

This paper discusses the use of multi-layer perceptron networks for linear or linearizable, adaptive feedback.control schemes in a discrete-time environment. A close look is taken at the model structure selected and the extent of the resulting parametrization. A comparison is made with standard, non-perceptron algorithms, e.g. self-tuning control, and it is shown how gross over-parametrization can occur in the neural network case. Because of the resultant heavy computational burden and poor controller convergence, a strong case is made against the use of neural networks for discrete-time linear control.

Stochastic Stability for Markovian Jump Linear Systems Subject to a Crucial Failure Event

This paper is concerned with the stability of discrete-time linear systems subject to random jumps in the parameters, described by an underlying finite-state Markov chain. In the model studied, a stopping time τ Δ is associated with the occurrence of a crucial failure after which the system is brought to a halt for maintenance. The usual stochastic stability concepts and associated results are not indicated, since they are tailored to pure infinite horizon problems. Using the concept named stochastic τ-stability, equivalent conditions to ensure the stochastic stability of the system until the occurrence of τ Δ is obtained. In addition, an intermediary and mixed case for which τ represents the minimum between the occurrence of a fix number N of failures and the occurrence of a crucial failure τ Δ is also considered. Necessary and sufficient conditions to ensure the stochastic τ-stability are provided in this setting that are auxiliary to the main result.

Optimization of Discrete-Time, Stochastic Systems

* This research was supported by a grant from the Greek Ministry of Industry and Technology.

Stability analysis of a general family of nonlinear positive discrete time-delay systems

In this paper, we propose a new approach to analyse the stability of a general family of nonlinear positive discrete time-delay systems. First, we introduce a new class of nonlinear positive discrete time-delay systems, which generalises some existing discrete time-delay systems. Second, through a new technique that relies on the comparison and mathematical induction method, we establish explicit criteria for stability and instability of the systems. Three numerical examples are given to illustrate the feasibility of the obtained results.

Linear quadratic Gaussian control of discrete-time Markov jump linear systems with horizon defined by stopping times

The linear quadratic Gaussian control of discrete-time Markov jump linear systems is addressed in this paper, first for state feedback, and also for dynamic output feedback using state estimation. in the model studied, the problem horizon is defined by a stopping time τ which represents either, the occurrence of a fix number N of failures or repairs (T N), or the occurrence of a crucial failure event (τ δ), after which the system paralyzed. From the constructive method used here a separation principle holds, and the solutions are given in terms of a Kalman filter and a state feedback sequence of controls. The control gains are obtained by recursions from a set of algebraic Riccati equations for the former case or by a coupled set of algebraic Riccati equation for the latter case. Copyright © 2005 IFAC.

H ∞ state feedback control of discrete-time Markov jump linear systems through linear matrix inequalities

This paper addresses the H ∞ state-feedback control design problem of discretetime Markov jump linear systems. First, under the assumption that the Markov parameter is measured, the main contribution is on the LMI characterization of all linear feedback controllers such that the closed loop output remains bounded by a given norm level. This results allows the robust controller design to deal with convex bounded parameter uncertainty, probability uncertainty and cluster availability of the Markov mode. For partly unknown transition probabilities, the proposed design problem is proved to be less conservative than one available in the current literature. An example is solved for illustration and comparisons. © 2011 IFAC.