A new method for analysing the equilibrium and time-dependent behaviour of Markovian models

Many large-scale stochastic systems, such as telecommunications networks, can be modelled using a continuous-time Markov chain. However, it is frequently the case that a satisfactory analysis of their time-dependent, or even equilibrium, behaviour is impossible. In this paper, we propose a new method of analyzing Markovian models, whereby the existing transition structure is replaced by a more amenable one. Using rates of transition given by the equilibrium expected rates of the corresponding transitions of the original chain, we are able to approximate its behaviour. We present two formulations of the idea of expected rates. The first provides a method for analysing time-dependent behaviour, while the second provides a highly accurate means of analysing equilibrium behaviour. We shall illustrate our approach with reference to a variety of models, giving particular attention to queueing and loss networks. (C) 2003 Elsevier Ltd. All rights reserved.

Conversive Hidden non-Markovian models

Magdeburg, Univ., Fak. für Informatik, Diss., 2012

Using observation ageing to improve Markovian model learning in QoS engineering

Markovian models are widely used to analyse quality-of-service properties of both system designs and deployed systems. Thanks to the emergence of probabilistic model checkers, this analysis can be performed with high accuracy. However, its usefulness is heavily dependent on how well the model captures the actual behaviour of the analysed system. Our work addresses this problem for a class of Markovian models termed discrete-time Markov chains (DTMCs). We propose a new Bayesian technique for learning the state transition probabilities of DTMCs based on observations of the modelled system. Unlike existing approaches, our technique weighs observations based on their age, to account for the fact that older observations are less relevant than more recent ones. A case study from the area of bioinformatics workflows demonstrates the effectiveness of the technique in scenarios where the model parameters change over time.

New methods for determining quasi-stationary distributions for Markov chains

We shall be concerned with the problem of determining quasi-stationary distributions for Markovian models directly from their transition rates Q. We shall present simple conditions for a mu-invariant measure m for Q to be mu-invariant for the transition function, so that if m is finite, it can be normalized to produce a quasi-stationary distribution. (C) 2000 Elsevier Science Ltd. All rights reserved.

Reliability evaluation of distribution systems considering automatic reclosers

This paper presents a new methodology to evaluate in a predictive way the reliability of distribution systems, considering the impact of automatic recloser switches. The developed algorithm is based on state enumeration techniques with Markovian models and on the minimal cut set theory. Some computational aspects related with the implementation of the proposed algorithm in typical distribution networks are also discussed. The description of the proposed approach is carried out using a sample test system. The results obtained with a typical configuration of a Brazilian system (EDP Bandeirante Energia S.A.) are presented and discussed.

O paradoxo da superdifusão de uma caminhada aleatória com memória exponencial

The random walk models with temporal correlation (i.e. memory) are of interest in the study of anomalous diffusion phenomena. The random walk and its generalizations are of prominent place in the characterization of various physical, chemical and biological phenomena. The temporal correlation is an essential feature in anomalous diffusion models. These temporal long-range correlation models can be called non-Markovian models, otherwise, the short-range time correlation counterparts are Markovian ones. Within this context, we reviewed the existing models with temporal correlation, i.e. entire memory, the elephant walk model, or partial memory, alzheimer walk model and walk model with a gaussian memory with profile. It is noticed that these models shows superdiffusion with a Hurst exponent H > 1/2. We study in this work a superdiffusive random walk model with exponentially decaying memory. This seems to be a self-contradictory statement, since it is well known that random walks with exponentially decaying temporal correlations can be approximated arbitrarily well by Markov processes and that central limit theorems prohibit superdiffusion for Markovian walks with finite variance of step sizes. The solution to the apparent paradox is that the model is genuinely non-Markovian, due to a time-dependent decay constant associated with the exponential behavior. In the end, we discuss ideas for future investigations.

Stationary Markovian Equilibrium in Overlapping Generation Models with Stochastic Nonclassical Production

This paper provides new sufficient conditions for the existence, computation via successive approximations, and stability of Markovian equilibrium decision processes for a large class of OLG models with stochastic nonclassical production. Our notion of stability is existence of stationary Markovian equilibrium. With a nonclassical production, our economies encompass a large class of OLG models with public policy, valued fiat money, production externalities, and Markov shocks to production. Our approach combines aspects of both topological and order theoretic fixed point theory, and provides the basis of globally stable numerical iteration procedures for computing extremal Markovian equilibrium objects. In addition to new theoretical results on existence and computation, we provide some monotone comparative statics results on the space of economies.

On the Existence and Characterization of Markovian Equilibrium in Models with Simple Non-Paternalistic Altruism

This paper provides sufficient conditions for existence of Markovian equilibrium in models with non-paternalistic altruism extending to one generation ahead. When utility is non-separable, we show that each equilibrium savings policy correspondence is increasing everywhere and single-valued, except perhaps on a countable number of points. It is also upper hemi-continuous where it is single valued. When utility is separable, we show that the equilibrium is unique, increasing, and continuous, and we provide an algorithm converging uniformly to the equilibrium.

Discrete-time mean variance optimal control of linear systems with Markovian jumps and multiplicative noise

In this article, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noise under three kinds of performance criterions related to the final value of the expectation and variance of the output. In the first problem it is desired to minimise the final variance of the output subject to a restriction on its final expectation, in the second one it is desired to maximise the final expectation of the output subject to a restriction on its final variance, and in the third one it is considered a performance criterion composed by a linear combination of the final variance and expectation of the output of the system. We present explicit sufficient conditions for the existence of an optimal control strategy for these problems, generalising previous results in the literature. We conclude this article presenting a numerical example of an asset liabilities management model for pension funds with regime switching.

Intensity correlation functions of dye lasers: Comparison of colored-gain-noise and colored-loss-noise models

The intensity correlation functions C(t) for the colored-gain-noise model of dye lasers are analyzed and compared with those for the loss-noise model. For correlation times ¿ larger than the deterministic relaxation time td, we show with the use of the adiabatic approximation that C(t) values coincide for both models. For small correlation times we use a method that provides explicit expressions of non-Markovian correlation functions, approximating simultaneously short- and long-time behaviors. Comparison with numerical simulations shows excellent results simultaneously for short- and long-time regimes. It is found that, when the correlation time of the noise increases, differences between the gain- and loss-noise models tend to disappear. The decay of C(t) for both models can be described by a time scale that approaches the deterministic relaxation time. However, in contrast with the loss-noise model, a secondary time scale remains for large times for the gain-noise model, which could allow one to distinguish between both models.

Non-Markovian Dynamics in Continuous Variable Quantum Systems

The present manuscript represents the completion of a research path carried forward during my doctoral studies in the University of Turku. It contains information regarding my scientific contribution to the field of open quantum systems, accomplished in collaboration with other scientists. The main subject investigated in the thesis is the non-Markovian dynamics of open quantum systems with focus on continuous variable quantum channels, e.g. quantum Brownian motion models. Non-Markovianity is here interpreted as a manifestation of the existence of a flow of information exchanged by the system and environment during the dynamical evolution. While in Markovian systems the flow is unidirectional, i.e. from the system to the environment, in non-Markovian systems there are time windows in which the flow is reversed and the quantum state of the system may regain coherence and correlations previously lost. Signatures of a non-Markovian behavior have been studied in connection with the dynamics of quantum correlations like entanglement or quantum discord. Moreover, in the attempt to recognisee non-Markovianity as a resource for quantum technologies, it is proposed, for the first time, to consider its effects in practical quantum key distribution protocols. It has been proven that security of coherent state protocols can be enhanced using non-Markovian properties of the transmission channels. The thesis is divided in two parts: in the first part I introduce the reader to the world of continuous variable open quantum systems and non-Markovian dynamics. The second part instead consists of a collection of five publications inherent to the topic.

Non-Markovian dynamics and quantum information probes

This Thesis discusses the phenomenology of the dynamics of open quantum systems marked by non-Markovian memory effects. Non-Markovian open quantum systems are the focal point of a flurry of recent research aiming to answer, e.g., the following questions: What is the characteristic trait of non-Markovian dynamical processes that discriminates it from forgetful Markovian dynamics? What is the microscopic origin of memory in quantum dynamics, and how can it be controlled? Does the existence of memory effects open new avenues and enable accomplishments that cannot be achieved with Markovian processes? These questions are addressed in the publications forming the core of this Thesis with case studies of both prototypical and more exotic models of open quantum systems. In the first part of the Thesis several ways of characterizing and quantifying non-Markovian phenomena are introduced. Their differences are then explored using a driven, dissipative qubit model. The second part of the Thesis focuses on the dynamics of a purely dephasing qubit model, which is used to unveil the origin of non-Markovianity for a wide class of dynamical models. The emergence of memory is shown to be strongly intertwined with the structure of the spectral density function, as further demonstrated in a physical realization of the dephasing model using ultracold quantum gases. Finally, as an application of memory effects, it is shown that non- Markovian dynamical processes facilitate a novel phenomenon of timeinvariant discord, where the total quantum correlations of a system are frozen to their initial value. Non-Markovianity can also be exploited in the detection of phase transitions using quantum information probes, as shown using the physically interesting models of the Ising chain in a transverse field and a Coulomb chain undergoing a structural phase transition.

Markovian Processes, Two-Sided Autoregressions and Finite-Sample Inference for Stationary and Nonstationary Autoregressive Processes.

In this paper, we develop finite-sample inference procedures for stationary and nonstationary autoregressive (AR) models. The method is based on special properties of Markov processes and a split-sample technique. The results on Markovian processes (intercalary independence and truncation) only require the existence of conditional densities. They are proved for possibly nonstationary and/or non-Gaussian multivariate Markov processes. In the context of a linear regression model with AR(1) errors, we show how these results can be used to simplify the distributional properties of the model by conditioning a subset of the data on the remaining observations. This transformation leads to a new model which has the form of a two-sided autoregression to which standard classical linear regression inference techniques can be applied. We show how to derive tests and confidence sets for the mean and/or autoregressive parameters of the model. We also develop a test on the order of an autoregression. We show that a combination of subsample-based inferences can improve the performance of the procedure. An application to U.S. domestic investment data illustrates the method.

Analysis of Some Stochastic Inventory Models with Pooling Retrial of Customers

The thesis deals with analysis of some Stochastic Inventory Models with Pooling/Retrial of Customers.. In the first model we analyze an (s,S) production Inventory system with retrial of customers. Arrival of customers from outside the system form a Poisson process. The inter production times are exponentially distributed with parameter µ. When inventory level reaches zero further arriving demands are sent to the orbit which has capacity M(<∞). Customers, who find the orbit full and inventory level at zero are lost to the system. Demands arising from the orbital customers are exponentially distributed with parameter γ. In the model-II we extend these results to perishable inventory system assuming that the life-time of each item follows exponential with parameter θ. The study deals with an (s,S) production inventory with service times and retrial of unsatisfied customers. Primary demands occur according to a Markovian Arrival Process(MAP). Consider an (s,S)-retrial inventory with service time in which primary demands occur according to a Batch Markovian Arrival Process (BMAP). The inventory is controlled by the (s,S) policy and (s,S) inventory system with service time. Primary demands occur according to Poissson process with parameter λ. The study concentrates two models. In the first model we analyze an (s,S) Inventory system with postponed demands where arrivals of demands form a Poisson process. In the second model, we extend our results to perishable inventory system assuming that the life-time of each item follows exponential distribution with parameter θ. Also it is assumed that when inventory level is zero the arriving demands choose to enter the pool with probability β and with complementary probability (1- β) it is lost for ever. Finally it analyze an (s,S) production inventory system with switching time. A lot of work is reported under the assumption that the switching time is negligible but this is not the case for several real life situation.