17 resultados para MARKOV DECISION PROCESSES
em Greenwich Academic Literature Archive - UK
Resumo:
Attention has recently focussed on stochastic population processes that can undergo total annihilation followed by immigration into state j at rate αj. The investigation of such models, called Markov branching processes with instantaneous immigration (MBPII), involves the study of existence and recurrence properties. However, results developed to date are generally opaque, and so the primary motivation of this paper is to construct conditions that are far easier to apply in practice. These turn out to be identical to the conditions for positive recurrence, which are very easy to check. We obtain, as a consequence, the surprising result that any MBPII that exists is ergodic, and so must possess an equilibrium distribution. These results are then extended to more general MBPII, and we show how to construct the associated equilibrium distributions.
Resumo:
A generalized Markov Brnching Process (GMBP) is a Markov branching model where the infinitesimal branching rates are modified with an interaction index. It is proved that there always exists only one GMBP. An associated differential-integral equation is derived. The extinction probalility and the mean and conditional mean extinction times are obtained. Ergodicity and stability of GMBP with resurrection are also considered. Easy checking criteria are established for ordinary and strong ergodicty. The equilibrium distribution is given in an elegant closed form. The probability meaning of our results is clear and thus explained.
Resumo:
This paper focuses on the basic problems regarding uniqueness and extinction properties for generalised Markov branching processes. The uniqueness criterion is firstly established and a differential–integral equation satisfied by the transition functions of such processes is derived. The extinction probability is then obtained. A closed form is presented for both the mean extinction time and the conditional mean extinction time. It turns out that these important quantities are closely related to the elementary gamma function.
Resumo:
This paper concentrates on investigating ergodicity and stability for generalised Markov branching processes with resurrection. Easy checking criteria including several clear-cut corollaries are established for ordinary and strong ergodicity of such processes. The equilibrium distribution is given in an elegant closed form for the ergodic case. The probabilistic interpretation of the results is clear and thus explained.
Resumo:
We extend the Harris regularity condition for ordinary Markov branching process to a more general case of non-linear Markov branching process. A regularity criterion which is very easy to check is obtained. In particular, we prove that a super-linear Markov branching process is regular if and only if the per capita offspring mean is less than or equal to I while a sub-linear Markov branching process is regular if the per capita offspring mean is finite. The Harris regularity condition then becomes a special case of our criterion.
Resumo:
This note provides a new probabilistic approach in discussing the weighted Markov branching process (WMBP) which is a natural generalisation of the ordinary Markov branching process. Using this approach, some important characteristics regarding the hitting times of such processes can be easily obtained. In particular, the closed forms for the mean extinction time and conditional mean extinction time are presented. The explosion behaviour of the process is investigated and the mean explosion time is derived. The mean global holding time and the mean total survival time are also obtained. The close link between these newly developed processes and the well-known compound Poisson processes is investigated. It is revealed that any weighted Markov branching process (WMBP) is a random time change of a compound Poisson process.
Resumo:
This paper surveys the recent progresses made in the field of unstable denumerable Markov processes. Emphases are laid upon methodology and applications. The important tools of Feller transition functions and Resolvent Decomposition Theorems are highlighted. Their applications particularly in unstable denumerable Markov processes with a single instantaneous state and Markov branching processes are illustrated.
Resumo:
A Feller–Reuter–Riley function is a Markov transition function whose corresponding semigroup maps the set of the real-valued continuous functions vanishing at infinity into itself. The aim of this paper is to investigate applications of such functions in the dual problem, Markov branching processes, and the Williams-matrix. The remarkable property of a Feller–Reuter–Riley function is that it is a Feller minimal transition function with a stable q-matrix. By using this property we are able to prove that, in the theory of branching processes, the branching property is equivalent to the requirement that the corresponding transition function satisfies the Kolmogorov forward equations associated with a stable q-matrix. It follows that the probabilistic definition and the analytic definition for Markov branching processes are actually equivalent. Also, by using this property, together with the Resolvent Decomposition Theorem, a simple analytical proof of the Williams' existence theorem with respect to the Williams-matrix is obtained. The close link between the dual problem and the Feller–Reuter–Riley transition functions is revealed. It enables us to prove that a dual transition function must satisfy the Kolmogorov forward equations. A necessary and sufficient condition for a dual transition function satisfying the Kolmogorov backward equations is also provided.
Resumo:
The key problems in discussing stochastic monotonicity and duality for continuous time Markov chains are to give the criteria for existence and uniqueness and to construct the associated monotone processes in terms of their infinitesimal q -matrices. In their recent paper, Chen and Zhang [6] discussed these problems under the condition that the given q-matrix Q is conservative. The aim of this paper is to generalize their results to a more general case, i.e., the given q-matrix Q is not necessarily conservative. New problems arise 'in removing the conservative assumption. The existence and uniqueness criteria for this general case are given in this paper. Another important problem, the construction of all stochastically monotone Q-processes, is also considered.
Resumo:
The key problems in discussing duality and monotonicity for continuous-time Markov chains are to find conditions for existence and uniqueness and then to construct corresponding processes in terms of infinitesimal characteristics, i.e., q-matrices. Such problems are solved in this paper under the assumption that the given q-matrix is conservative. Some general properties of stochastically monotone Q-process ( Q is not necessarily conservative) are also discussed.
Resumo:
The problems encountered when using traditional rectangular pulse hierarchical point processmodels for fine temporal resolution and the growing number of available tip-time records suggest that rainfall increments from tipping-bucket gauges be modelled directly. Poisson processes are used with an arrival rate modulated by a Markov chain in Continuous time. The paper shows how, by using two or three states for this chain, much of the structure of the rainfall intensity distribution and the wet/dry sequences can be represented for time-scales as small as 5 minutes.
Resumo:
A new structure with the special property that an instantaneous reflection barrier is imposed on the ordinary birth-death processes is considered. An easy-checking criterion for the existence of such Markov processes is first obtained. The uniqueness criterion is then established. In the nonunique case, all the honest processes are explicitly constructed. Ergodicity properties for these processes are investigated. It is proved that honest processes are always ergodic without necessarily imposing any extra conditions. Equilibrium distributions for all these ergodic processes are established. Several examples are provided to illustrate our results.
Resumo:
A new structure with the special property that instantaneous resurrection and mass disaster are imposed on an ordinary birth-death process is considered. Under the condition that the underlying birth-death process is exit or bilateral, we are able to give easily checked existence criteria for such Markov processes. A very simple uniqueness criterion is also established. All honest processes are explicitly constructed. Ergodicity properties for these processes are investigated. Surprisingly, it can be proved that all the honest processes are not only recurrent but also ergodic without imposing any extra conditions. Equilibrium distributions are then established. Symmetry and reversibility of such processes are also investigated. Several examples are provided to illustrate our results.