The Policy Iteration Algorithm for Average Continuous Control of Piecewise Deterministic Markov Processes

The main goal of this paper is to apply the so-called policy iteration algorithm (PIA) for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP`s) taking values in a general Borel space and with compact action space depending on the state variable. In order to do that we first derive some important properties for a pseudo-Poisson equation associated to the problem. In the sequence it is shown that the convergence of the PIA to a solution satisfying the optimality equation holds under some classical hypotheses and that this optimal solution yields to an optimal control strategy for the average control problem for the continuous-time PDMP in a feedback form.

THE VANISHING DISCOUNT APPROACH FOR THE AVERAGE CONTINUOUS CONTROL OF PIECEWISE DETERMINISTIC MARKOV PROCESSES

This work is concerned with the existence of an optimal control strategy for the long-run average continuous control problem of piecewise-deterministic Markov processes (PDMPs). In Costa and Dufour (2008), sufficient conditions were derived to ensure the existence of an optimal control by using the vanishing discount approach. These conditions were mainly expressed in terms of the relative difference of the alpha-discount value functions. The main goal of this paper is to derive tractable conditions directly related to the primitive data of the PDMP to ensure the existence of an optimal control. The present work can be seen as a continuation of the results derived in Costa and Dufour (2008). Our main assumptions are written in terms of some integro-differential inequalities related to the so-called expected growth condition, and geometric convergence of the post-jump location kernel associated to the PDMP. An example based on the capacity expansion problem is presented, illustrating the possible applications of the results developed in the paper.

On the Filtering Problem for Continuous-Time Markov Jump Linear Systems with no Observation of the Markov Chain

We consider in this paper the optimal stationary dynamic linear filtering problem for continuous-time linear systems subject to Markovian jumps in the parameters (LSMJP) and additive noise (Wiener process). It is assumed that only an output of the system is available and therefore the values of the jump parameter are not accessible. It is a well known fact that in this setting the optimal nonlinear filter is infinite dimensional, which makes the linear filtering a natural numerically, treatable choice. The goal is to design a dynamic linear filter such that the closed loop system is mean square stable and minimizes the stationary expected value of the mean square estimation error. It is shown that an explicit analytical solution to this optimal filtering problem is obtained from the stationary solution associated to a certain Riccati equation. It is also shown that the problem can be formulated using a linear matrix inequalities (LMI) approach, which can be extended to consider convex polytopic uncertainties on the parameters of the possible modes of operation of the system and on the transition rate matrix of the Markov process. As far as the authors are aware of this is the first time that this stationary filtering problem (exact and robust versions) for LSMJP with no knowledge of the Markov jump parameters is considered in the literature. Finally, we illustrate the results with an example.

Generalized Coupled Algebraic Riccati Equations for Discrete-time Markov Jump with Multiplicative Noise Systems

In this paper we consider the existence of the maximal and mean square stabilizing solutions for a set of generalized coupled algebraic Riccati equations (GCARE for short) associated to the infinite-horizon stochastic optimal control problem of discrete-time Markov jump with multiplicative noise linear systems. The weighting matrices of the state and control for the quadratic part are allowed to be indefinite. We present a sufficient condition, based only on some positive semi-definite and kernel restrictions on some matrices, under which there exists the maximal solution and a necessary and sufficient condition under which there exists the mean square stabilizing solution fir the GCARE. We also present a solution for the discounted and long run average cost problems when the performance criterion is assumed be composed by a linear combination of an indefinite quadratic part and a linear part in the state and control variables. The paper is concluded with a numerical example for pension fund with regime switching.

Linear minimum mean square filter for discrete-time linear systems with Markov jumps and multiplicative noises

In this paper we obtain the linear minimum mean square estimator (LMMSE) for discrete-time linear systems subject to state and measurement multiplicative noises and Markov jumps on the parameters. It is assumed that the Markov chain is not available. By using geometric arguments we obtain a Kalman type filter conveniently implementable in a recurrence form. The stationary case is also studied and a proof for the convergence of the error covariance matrix of the LMMSE to a stationary value under the assumption of mean square stability of the system and ergodicity of the associated Markov chain is obtained. It is shown that there exists a unique positive semi-definite solution for the stationary Riccati-like filter equation and, moreover, this solution is the limit of the error covariance matrix of the LMMSE. The advantage of this scheme is that it is very easy to implement and all calculations can be performed offline. (c) 2011 Elsevier Ltd. All rights reserved.

A generalized multi-period mean-variance portfolio optimization with Markov switching parameters

In this paper, we deal with a generalized multi-period mean-variance portfolio selection problem with market parameters Subject to Markov random regime switchings. Problems of this kind have been recently considered in the literature for control over bankruptcy, for cases in which there are no jumps in market parameters (see [Zhu, S. S., Li, D., & Wang, S. Y. (2004). Risk control over bankruptcy in dynamic portfolio selection: A generalized mean variance formulation. IEEE Transactions on Automatic Control, 49, 447-457]). We present necessary and Sufficient conditions for obtaining an optimal control policy for this Markovian generalized multi-period meal-variance problem, based on a set of interconnected Riccati difference equations, and oil a set of other recursive equations. Some closed formulas are also derived for two special cases, extending some previous results in the literature. We apply the results to a numerical example with real data for Fisk control over bankruptcy Ill a dynamic portfolio selection problem with Markov jumps selection problem. (C) 2008 Elsevier Ltd. All rights reserved.

Unlocking hidden genomic sequence

Despite the success of conventional Sanger sequencing, significant regions of many genomes still present major obstacles to sequencing. Here we propose a novel approach with the potential to alleviate a wide range of sequencing difficulties. The technique involves extracting target DNA sequence from variants generated by introduction of random mutations. The introduction of mutations does not destroy original sequence information, but distributes it amongst multiple variants. Some of these variants lack problematic features of the target and are more amenable to conventional sequencing. The technique has been successfully demonstrated with mutation levels up to an average 18% base substitution and has been used to read previously intractable poly(A), AT-rich and GC-rich motifs.

Integrable multiparametric quantum spin chains

Using Reshetikhin's construction for multiparametric quantum algebras we obtain the associated multiparametric quantum spin chains. We show that under certain restrictions these models can be mapped to quantum spin chains with twisted boundary conditions, We illustrate how this general formalism applier; to construct multiparametric versions of the supersymmetric t-J and Li models.

Contradiction of quantum mechanics with local hidden variables for quadrature phase amplitude measurements

We demonstrate a contradiction of quantum mechanics with local hidden variable theories for continuous quadrature phase amplitude (position and momentum) measurements. For any quantum state, this contradiction is lost for situations where the quadrature phase amplitude results are always macroscopically distinct. We show that for optical realizations of this experiment, where one uses homodyne detection techniques to perform the quadrature phase amplitude measurement, one has an amplification prior to detection, so that macroscopic fields are incident on photodiode detectors. The high efficiencies of such detectors may open a way for a loophole-free test of local hidden variable theories.

An open-boundary integrable model of three coupled XY spin chains

The integrable open-boundary conditions for the model of three coupled one-dimensional XY spin chains are considered in the framework of the quantum inverse scattering method. The diagonal boundary K-matrices are found and a class of integrable boundary terms is determined. The boundary model Hamiltonian is solved by using the coordinate space Bethe ansatz technique and Bethe ansatz equations are derived. (C) 1998 Elsevier Science B.V.

Effect of disorder on quantum phase transitions in anisotropic XY spin chains in a transverse field

We present some exact results for the effect of disorder on the critical properties of an anisotropic XY spin chain in a transverse held. The continuum limit of the corresponding fermion model is taken and in various cases results in a Dirac equation with a random mass. Exact analytic techniques can then be used to evaluate the density of states and the localization length. In the presence of disorder the ferromagnetic-paramagnetic or Ising transition of the model is in the same universality class as the random transverse field Ising model solved by Fisher using a real-space renormalization-group decimation technique (RSRGDT). If there is only randomness in the anisotropy of the magnetic exchange then the anisotropy transition (from a ferromagnet in the x direction to a ferromagnet in the y direction) is also in this universality class. However, if there is randomness in the isotropic part of the exchange or in the transverse held then in a nonzero transverse field the anisotropy transition is destroyed by the disorder. We show that in the Griffiths' phase near the Ising transition that the ground-state energy has an essential singularity. The results obtained for the dynamical critical exponent, typical correlation length, and for the temperature dependence of the specific heat near the Ising transition agree with the results of the RSRODT and numerical work. [S0163-1829(99)07125-8].

Jordan-Wigner fermionization of the one-dimensional Bariev model of three coupled XY chains

The Jordan-Wigner fermionization for the one-dimensional Bariev model of three coupled XY chains is formulated. The L-matrix in terms of fermion operators and the R-matrix are presented explicitly. Furthermore, the graded reflection equations and their solutions are discussed.

On the construction of integrable closed chains with quantum supersymmetry

We present a general prescription for the construction of integrable one-dimensional systems with closed boundary conditions and quantum supersymmetry.

Glycosaminoglycan chains from alpha(5)beta(1) integrin are involved in fibronectin-dependent cell migration

alpha(5)beta(1) integrin from both wild-type CHO cells (CHO-K1) and deficient in proteoglycan biosynthesis (CHO-745) is post-translationally modified by glycosaminoglycan chains. We demonstrated this using [(35)S]sulfate metabolic labeling of the cells, enzymatic degradation, immunoprecipitation reaction with monoclonal antibody, fluorescence microscopy, and flow cytometry. The alpha(5)beta(1) integrin heterodimer is a hybrid proteoglycan containing both chondroitin and heparan sulfate chains. Xyloside inhibition of sulfate incorporation into alpha(5)beta(1) integrin also supports that integrin is a proteoglycan. Also. cells grown with xyloside adhered on fibronectin with no alteration in alpha(5)beta(1) integrin expression. However, haptotactic motility on fibronectin declined in cells grown with xyloside or chlorate as compared with controls. Thus, alpha(5)beta(1) integrin is a proteoglycan and the glycosaminoglycan chains of the integrin influence cell motility on fibronectin. Similar glycosylation of alpha(5)beta(1) integrin was observed in other normal and malignant cells, suggesting that this modification is conserved and important in the function of this integrin. Therefore, these glycosaminoglycan chains of alpha(5)beta(1) integrin are involved in cellular migration on fibronectin.