Discrete-time mean variance optimal control of linear systems with Markovian jumps and multiplicative noise

In this article, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noise under three kinds of performance criterions related to the final value of the expectation and variance of the output. In the first problem it is desired to minimise the final variance of the output subject to a restriction on its final expectation, in the second one it is desired to maximise the final expectation of the output subject to a restriction on its final variance, and in the third one it is considered a performance criterion composed by a linear combination of the final variance and expectation of the output of the system. We present explicit sufficient conditions for the existence of an optimal control strategy for these problems, generalising previous results in the literature. We conclude this article presenting a numerical example of an asset liabilities management model for pension funds with regime switching.

Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP`s) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space a""e (n) . The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter epsilon > 0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as epsilon goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as epsilon goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.

Linear minimum mean square filter for discrete-time linear systems with Markov jumps and multiplicative noises

In this paper we obtain the linear minimum mean square estimator (LMMSE) for discrete-time linear systems subject to state and measurement multiplicative noises and Markov jumps on the parameters. It is assumed that the Markov chain is not available. By using geometric arguments we obtain a Kalman type filter conveniently implementable in a recurrence form. The stationary case is also studied and a proof for the convergence of the error covariance matrix of the LMMSE to a stationary value under the assumption of mean square stability of the system and ergodicity of the associated Markov chain is obtained. It is shown that there exists a unique positive semi-definite solution for the stationary Riccati-like filter equation and, moreover, this solution is the limit of the error covariance matrix of the LMMSE. The advantage of this scheme is that it is very easy to implement and all calculations can be performed offline. (c) 2011 Elsevier Ltd. All rights reserved.

Further evidence for a deficit in switching attention in schizophrenia

In this study, sustained, selective, divided, and switching attention, and reloading of working memory were investigated in schizophrenia by using a newly developed Visual Attention Battery (VAB). Twenty-four outpatients with schizophrenia and 24 control participants were studied using the VAB. Performance on VAB components was correlated with performance of standard tests. Patients with schizophrenia were significantly impaired on VAB tasks that required switching of attention and reloading of working memory but had normal performance on tasks involving sustained attention or attention to multiple stimulus features. Switching attention and reloading of working memory were highly correlated with Trails (B - A) score for patients. The decline in performance on the switching-attention task in patients with schizophrenia met criteria for a differential deficit in switching attention. Future research should examine the neurophysiological basis of the switching deficit and its sensitivity and specificity to schizophrenia.

Solution structure of methionine-oxidized amyloid beta-peptide (1-40). Does oxidation affect conformational switching?

The solution structure of A beta(1-40)Met(O), the methionine-oxidized form of amyloid beta-peptide A beta(1-40), has been investigated by CD and NMR spectroscopy. Oxidation of Met35 may have implications in the aetiology of Alzheimer's disease. Circular dichroism experiments showed that whereas A beta(1-40) and A beta(1-40)Met(O) both adopt essentially random coil structures in water (pH 4) at micromolar concentrations, the former aggregates within several days while the latter is stable for at least 7 days under these conditions. This remarkable difference led us to determine the solution structure of A beta(1-40)Met(O) using H-1 NMR spectroscopy. In a water-SDS micelle medium needed to solubilize both peptides at the millimolar concentrations required to measure NMR spectra, chemical shift and NOE data for A beta(1-40)Met(O) strongly suggest the presence of a helical region between residues 16 and 24. This is supported by slow H-D exchange of amide protons in this region and by structure calculations using simulated annealing with the program XPLOR. The remainder of the structure is relatively disordered. Our previously reported NMR data for A beta(1-40) in the same solvent shows that helices are present over residues 15-24 (helix 1) and 28-36 (helix 2), Oxidation of Met35 thus causes a local and selective disruption of helix 2. In addition to this helix-coil rearrangement in aqueous micelles, the CD data show that oxidation inhibits a coil-to-beta-sheet transition in water. These significant structural rearrangements in the C-terminal region of A beta may be important clues to the chemistry and biology of A beta(1-40) and A beta(1-42).

A numerical study of large sparse matrix exponentials arising in Markov chains

Krylov subspace techniques have been shown to yield robust methods for the numerical computation of large sparse matrix exponentials and especially the transient solutions of Markov Chains. The attractiveness of these methods results from the fact that they allow us to compute the action of a matrix exponential operator on an operand vector without having to compute, explicitly, the matrix exponential in isolation. In this paper we compare a Krylov-based method with some of the current approaches used for computing transient solutions of Markov chains. After a brief synthesis of the features of the methods used, wide-ranging numerical comparisons are performed on a power challenge array supercomputer on three different models. (C) 1999 Elsevier Science B.V. All rights reserved.AMS Classification: 65F99; 65L05; 65U05.

Asymmetry and duration dependence in Australian GDP and unemployment

The per iodic structure of business cycles suggests that significant asymmetries are present over different phases of the cycle. This paper uses markov regime-switching models with fixed and duration dependent transition probabilities to directly model expansions, contractions and durations in Australian GDP growth and unemployment growth. Evidence is found of significant asymmetry in growth rates across expansions and contractions for both series. GDP contractions exhibit duration dependence implying that as output recessions age the likelihood of switching into an expansion phase increases. Unemployment growth does not exhibit duration dependence in either phase. Evidence is also presented that non-linearities in unemployment growth are well explained by the asymmetries in the GDP growth cycle. The analysis suggests that recessions are periods of rapid and intense job destruction, that Australian unemployment tends to ratchet up in recessionary periods and, in contrast to US and UK studies, that shocks to Australian unemployment growth are more persistent in recessions than expansions. [E37 C5 C41].

Quasistationarity of continuous-time Markov chains with positive drift

We shall study continuous-time Markov chains on the nonnegative integers which are both irreducible and transient, and which exhibit discernible stationarity before drift to infinity sets in. We will show how this 'quasi' stationary behaviour can be modelled using a limiting conditional distribution: specifically, the limiting state probabilities conditional on not having left 0 for the last time. By way of a dual chain, obtained by killing the original process on last exit from 0, we invoke the theory of quasistationarity for absorbing Markov chains. We prove that the conditioned state probabilities of the original chain are equal to the state probabilities of its dual conditioned on non-absorption, thus allowing us to establish the simultaneous existence and then equivalence, of their limiting conditional distributions. Although a limiting conditional distribution for the dual chain is always a quasistationary distribution in the usual sense, a similar statement is not possible for the original chain.

Further results on the relationship between mu-invariant measures and quasi-stationary distributions for absorbing continuous-time Markov chains

This note considers continuous-time Markov chains whose state space consists of an irreducible class, C, and an absorbing state which is accessible from C. The purpose is to provide results on mu-invariant and mu-subinvariant measures where absorption occurs with probability less than one. In particular, the well-known premise that the mu-invariant measure, m, for the transition rates be finite is replaced by the more natural premise that m be finite with respect to the absorption probabilities. The relationship between mu-invariant measures and quasi-stationary distributions is discussed. (C) 2000 Elsevier Science Ltd. All rights reserved.

New methods for determining quasi-stationary distributions for Markov chains

We shall be concerned with the problem of determining quasi-stationary distributions for Markovian models directly from their transition rates Q. We shall present simple conditions for a mu-invariant measure m for Q to be mu-invariant for the transition function, so that if m is finite, it can be normalized to produce a quasi-stationary distribution. (C) 2000 Elsevier Science Ltd. All rights reserved.

Neighborhood and Efficiency in Manufacturing in Brazilian Regions: A Spatial Markov Chain Analysis

This paper analyzes the geography of regional competitiveness in manufacturing in Brazil. The authors estimate stochastic frontiers to calculate regional efficiency of representative firms in 137 regions in the period 2000-2006, in four sectors defined by technological intensity. The efficiency results are analyzed using Markov Spatial Transition Matrices to provide insights into the transition of regions between efficiency levels, considering their local spatial context. The results indicate that geography plays an important role in manufacturing competitiveness. In particular, regions with more competitive neighbors are more likely to improve their relative efficiency (pull effect) over time, and regions with less competitive neighbors are more likely to lose relative efficiency (drag effect). The authors find that the pull effect is stronger than the drag effect.

Characterization of the rat exploratory behavior in the elevated plus-maze with Markov chains

The elevated plus-maze is an animal model of anxiety used to study the effect of different drugs on the behavior of the animal It consists of a plus-shaped maze with two open and two closed arms elevated 50 cm from the floor The standard measures used to characterize exploratory behavior in the elevated plus-maze are the time spent and the number of entries in the open arms In this work we use Markov chains to characterize the exploratory behavior of the rat in the elevated plus-maze under three different conditions normal and under the effects of anxiogenic and anxiolytic drugs The spatial structure of the elevated plus-maze is divided into squares which are associated with states of a Markov chain By counting the frequencies of transitions between states during 5-min sessions in the elevated plus-maze we constructed stochastic matrices for the three conditions studied The stochastic matrices show specific patterns which correspond to the observed behaviors of the rat under the three different conditions For the control group the stochastic matrix shows a clear preference for places in the closed arms This preference is enhanced for the anxiogenic group For the anxiolytic group the stochastic matrix shows a pattern similar to a random walk Our results suggest that Markov chains can be used together with the standard measures to characterize the rat behavior in the elevated plus-maze (C) 2010 Elsevier B V All rights reserved

Similar Markov chains

enin et al. (2000) recently introduced the idea of similarity in the context of birth-death processes. This paper examines the extent to which their results can be extended to arbitrary Markov chains. It is proved that, under a variety of conditions, similar chains are strongly similar in a sense which is described, and it is shown that minimal chains are strongly similar if and only if the corresponding transition-rate matrices are strongly similar. A general framework is given for constructing families of strongly similar chains; it permits the construction of all such chains in the irreducible case.