Multilinear and Integer Programming for Markov Decision Processes with Imprecise Probabilities

Markov Decision Processes (MDPs) are extensively used to encode sequences of decisions with probabilistic effects. Markov Decision Processes with Imprecise Probabilities (MDPIPs) encode sequences of decisions whose effects are modeled using sets of probability distributions. In this paper we examine the computation of Γ-maximin policies for MDPIPs using multilinear and integer programming. We discuss the application of our algorithms to “factored” models and to a recent proposal, Markov Decision Processes with Set-valued Transitions (MDPSTs), that unifies the fields of probabilistic and “nondeterministic” planning in artificial intelligence research. 

Option pricing in market models driven by telegraph processes with jumps

Esta tesis está dividida en dos partes: en la primera parte se presentan y estudian los procesos telegráficos, los procesos de Poisson con compensador telegráfico y los procesos telegráficos con saltos. El estudio presentado en esta primera parte incluye el cálculo de las distribuciones de cada proceso, las medias y varianzas, así como las funciones generadoras de momentos entre otras propiedades. Utilizando estas propiedades en la segunda parte se estudian los modelos de valoración de opciones basados en procesos telegráficos con saltos. En esta parte se da una descripción de cómo calcular las medidas neutrales al riesgo, se encuentra la condición de no arbitraje en este tipo de modelos y por último se calcula el precio de las opciones Europeas de compra y venta.

Studying reactive processes with classical dynamics: rebinding dynamics in MbNO

A new surface-crossing algorithm suitable for describing bond-breaking and bond-forming processes in molecular dynamics simulations is presented. The method is formulated for two intersecting potential energy manifolds which dissociate to different adiabatic states. During simulations, crossings are detected by monitoring an energy criterion. If fulfilled, the two manifolds are mixed over a finite number of time steps, after which the system is propagated on the second adiabat and the crossing is carried out with probability one. The algorithm is extensively tested (almost 0.5 mu s of total simulation time) for the rebinding of NO to myoglobin. The unbound surface ((FeNO)-N-...) is represented using a standard force field, whereas the bound surface (Fe-NO) is described by an ab initio potential energy surface. The rebinding is found to be nonexponential in time, in agreement with experimental studies, and can be described using two time constants. Depending on the asymptotic energy separation between the manifolds, the short rebinding timescale is between 1 and 9 ps, whereas the longer timescale is about an order of magnitude larger. NO molecules which do not rebind within 1 ns are typically found in the Xenon-4 pocket, indicating the high affinity of NO to this region in the protein.

Multi-microprocessor control of processes with pure time delay

In 1967 a novel scheme was proposed for controlling processes with large pure time delay (Fellgett et al, 1967) and some of the constituent parts of the scheme were investigated (Swann, 1970; Atkinson et al, 1973). At that time the available computational facilities were inadequate for the scheme to be implemented practically, but with the advent of modern microcomputers the scheme becomes feasible. This paper describes recent work (Mitchell, 1987) in implementing the scheme in a new multi-microprocessor configuration and shows the improved performance it provides compared with conventional three-term controllers.

Decay to equilibrium for jump processes with heavy tails

The case is made for a more careful analysis of the large time asymptotic of infinite particle systems in the thermodynamic limit beyond zero density. The insufficiency of current analysis even in the model case of free particles is demonstrated. Recent advances based on more sophisticated analytical tools like functions of mean variation and Hardy spaces are sketched.

Co-movement between two processes with local persistence

This paper introduces a residual based test where the null hypothesis of c:&InOvement between two processes with local persistenc~ can be tested, even under the presence of an endogenous regressor. It, therefore, fills in an existing lacuna in econometrics, in which longrun relationships can also be tested if the dependent and independent variables do not have a unit root, but do exhibit local persistence.

Monitoring the mean vector and the covariance ­matrix of multivariate processes with sample means and sample ranges

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

A moment symbolic representation of Lévy processes with applications

By using a symbolic method, known in the literature as the classical umbral calculus, a symbolic representation of Lévy processes is given and a new family of time-space harmonic polynomials with respect to such processes, which includes and generalizes the exponential complete Bell polynomials, is introduced. The usefulness of time-space harmonic polynomials with respect to Lévy processes is that it is a martingale the stochastic process obtained by replacing the indeterminate x of the polynomials with a Lévy process, whereas the Lévy process does not necessarily have this property. Therefore to find such polynomials could be particularly meaningful for applications. This new family includes Hermite polynomials, time-space harmonic with respect to Brownian motion, Poisson-Charlier polynomials with respect to Poisson processes, Laguerre and actuarial polynomials with respect to Gamma processes , Meixner polynomials of the first kind with respect to Pascal processes, Euler, Bernoulli, Krawtchuk, and pseudo-Narumi polynomials with respect to suitable random walks. The role played by cumulants is stressed and brought to the light, either in the symbolic representation of Lévy processes and their infinite divisibility property, either in the generalization, via umbral Kailath-Segall formula, of the well-known formulae giving elementary symmetric polynomials in terms of power sum symmetric polynomials. The expression of the family of time-space harmonic polynomials here introduced has some connections with the so-called moment representation of various families of multivariate polynomials. Such moment representation has been studied here for the first time in connection with the time-space harmonic property with respect to suitable symbolic multivariate Lévy processes. In particular, multivariate Hermite polynomials and their properties have been studied in connection with a symbolic version of the multivariate Brownian motion, while multivariate Bernoulli and Euler polynomials are represented as powers of multivariate polynomials which are time-space harmonic with respect to suitable multivariate Lévy processes.