Stability and ergodicity of piecewise deterministic Markov processes

The main goal of this paper is to establish some equivalence results on stability, recurrence, and ergodicity between a piecewise deterministic Markov process ( PDMP) {X( t)} and an embedded discrete-time Markov chain {Theta(n)} generated by a Markov kernel G that can be explicitly characterized in terms of the three local characteristics of the PDMP, leading to tractable criterion results. First we establish some important results characterizing {Theta(n)} as a sampling of the PDMP {X( t)} and deriving a connection between the probability of the first return time to a set for the discrete-time Markov chains generated by G and the resolvent kernel R of the PDMP. From these results we obtain equivalence results regarding irreducibility, existence of sigma-finite invariant measures, and ( positive) recurrence and ( positive) Harris recurrence between {X( t)} and {Theta(n)}, generalizing the results of [ F. Dufour and O. L. V. Costa, SIAM J. Control Optim., 37 ( 1999), pp. 1483-1502] in several directions. Sufficient conditions in terms of a modified Foster-Lyapunov criterion are also presented to ensure positive Harris recurrence and ergodicity of the PDMP. We illustrate the use of these conditions by showing the ergodicity of a capacity expansion model.

AVERAGE CONTINUOUS CONTROL OF PIECEWISE DETERMINISTIC MARKOV PROCESSES

This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMPs) taking values in a general Borel space and with compact action space depending on the state variable. The control variable acts on the jump rate and transition measure of the PDMP, and the running and boundary costs are assumed to be positive but not necessarily bounded. Our first main result is to obtain an optimality equation for the long run average cost in terms of a discrete-time optimality equation related to the embedded Markov chain given by the postjump location of the PDMP. Our second main result guarantees the existence of a feedback measurable selector for the discrete-time optimality equation by establishing a connection between this equation and an integro-differential equation. Our final main result is to obtain some sufficient conditions for the existence of a solution for a discrete-time optimality inequality and an ordinary optimal feedback control for the long run average cost using the so-called vanishing discount approach. Two examples are presented illustrating the possible applications of the results developed in the paper.

The Policy Iteration Algorithm for Average Continuous Control of Piecewise Deterministic Markov Processes

The main goal of this paper is to apply the so-called policy iteration algorithm (PIA) for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP`s) taking values in a general Borel space and with compact action space depending on the state variable. In order to do that we first derive some important properties for a pseudo-Poisson equation associated to the problem. In the sequence it is shown that the convergence of the PIA to a solution satisfying the optimality equation holds under some classical hypotheses and that this optimal solution yields to an optimal control strategy for the average control problem for the continuous-time PDMP in a feedback form.

THE VANISHING DISCOUNT APPROACH FOR THE AVERAGE CONTINUOUS CONTROL OF PIECEWISE DETERMINISTIC MARKOV PROCESSES

This work is concerned with the existence of an optimal control strategy for the long-run average continuous control problem of piecewise-deterministic Markov processes (PDMPs). In Costa and Dufour (2008), sufficient conditions were derived to ensure the existence of an optimal control by using the vanishing discount approach. These conditions were mainly expressed in terms of the relative difference of the alpha-discount value functions. The main goal of this paper is to derive tractable conditions directly related to the primitive data of the PDMP to ensure the existence of an optimal control. The present work can be seen as a continuation of the results derived in Costa and Dufour (2008). Our main assumptions are written in terms of some integro-differential inequalities related to the so-called expected growth condition, and geometric convergence of the post-jump location kernel associated to the PDMP. An example based on the capacity expansion problem is presented, illustrating the possible applications of the results developed in the paper.

Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP`s) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space a""e (n) . The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter epsilon > 0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as epsilon goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as epsilon goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.

Description of a new Tetranychus (Acarina, Prostigmata : Tetranychidae) pest of Musa from French Guiana

A new species of tetranychid mite, genus Tetranychus Dufour, 1832, is described and illustrated herein from neotropical area. Tetranychus musae sp. nov. (Acari, Prostigmata: Tetranychidae) differs from other species in the genus by the combination between the arrangement of leg setae on females tarsus I and the shape of the aedeagus. Tarsus I bears one tactile setae proximal to proximal duplex setae and three tactiles in line or almost in line with proximal duplex setae. The aedeagus knob consists of an acute posterior projection bent ventrally and a larger anterior rounded projection directed anterodorsally. T. musae specimens were collected in French Guiana where they appeared to be a pest of Musa sp. A key to adults of neotropical species of the genus Tetranychus feeding on banana is provided.

In vitro macrophage activity: Biphasic effect of prolactin and indirect evidence of dopaminergic modulation

Objective: Prolactin (PRL), a peptide hormone produced by the pituitary gland, is involved in the interaction between the neuroendocrine and immune system. Since dopamine receptor antagonists increase serum levels of PRL, both PRL and dopamine receptors might be involved in the modulation of macrophage activity, providing means of communication between the nervous and immune systems. This study evaluated the effects of PRL and the dopamine antagonist domperidone (DOMP) on macrophage activity of female rats. Methods: Oxidative burst and phagocytosis of peritoneal macrophages were evaluated by flow cytometry. Samples of peritoneal liquid from female rats were first incubated with PRL (10 and 100 nM) for different periods. The same procedure was repeated to evaluate the effects of DOMP (10 and 100 nM). Results: In vitro incubation of macrophages with 10 nM DOMP decreased oxidative burst, after 30 min, whereas the PMA-induced burst was decreased by DOMP 10 nM after 2 and 4 h. Treatment with PRL (10 and 100 nM) for 30 min decreased oxidative burst and rate of phagocytosis (10 nM). After 2 h of incubation, 10 nM PRL decreased oxidative burst and phagocytosis intensity, but increased the rate of phagocytosis. On the other hand, after 4 h, PRL 10 and 100 nM increased oxidative burst and the rate of phagocytosis, but decreased intensity of phagocytosis. Conclusions: These observations suggest that macrophage functions are regulated by an endogenous dopaminergic tone. Our data also suggest that both PRL and dopamine exert their action by acting directly on the peritoneal macrophage. Copyright (C) 2008 S. Karger AG, Basel.