Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

LANDSCAPE DYNAMICS MODELING THROUGH INTEGRATED MARKOV, FUZZY-AHP AND CELLULAR AUTOMATA

Multi temporal land use information were derived using two decades remote sensing data and simulated for 2012 and 2020 with Cellular Automata (CA) considering scenarios, change probabilities (through Markov chain) and Multi Criteria Evaluation (MCE). Agents and constraints were considered for modeling the urbanization process. Agents were nornmlized through fiizzyfication and priority weights were assigned through Analytical Hierarchical Process (AHP) pairwise comparison for each factor (in MCE) to derive behavior-oriented rules of transition for each land use class. Simulation shows a good agreement with the classified data. Fuzzy and AHP helped in analyzing the effects of agents of growth clearly and CA-Markov proved as a powerful tool in modelling and helped in capturing and visualizing the spatiotemporal patterns of urbanization. This provided rapid land evaluation framework with the essential insights of the urban trajectory for effective sustainable city planning.

Markov chains, R-trivial monoids and representation theory

We develop a general theory of Markov chains realizable as random walks on R-trivial monoids. It provides explicit and simple formulas for the eigenvalues of the transition matrix, for multiplicities of the eigenvalues via Mobius inversion along a lattice, a condition for diagonalizability of the transition matrix and some techniques for bounding the mixing time. In addition, we discuss several examples, such as Toom-Tsetlin models, an exchange walk for finite Coxeter groups, as well as examples previously studied by the authors, such as nonabelian sandpile models and the promotion Markov chain on posets. Many of these examples can be viewed as random walks on quotients of free tree monoids, a new class of monoids whose combinatorics we develop.

Markov chain splitting methods in structural reliability integral estimation

Monte Carlo simulation methods involving splitting of Markov chains have been used in evaluation of multi-fold integrals in different application areas. We examine in this paper the performance of these methods in the context of evaluation of reliability integrals from the point of view of characterizing the sampling fluctuations. The methods discussed include the Au-Beck subset simulation, Holmes-Diaconis-Ross method, and generalized splitting algorithm. A few improvisations based on first order reliability method are suggested to select algorithmic parameters of the latter two methods. The bias and sampling variance of the alternative estimators are discussed. Also, an approximation to the sampling distribution of some of these estimators is obtained. Illustrative examples involving component and series system reliability analyses are presented with a view to bring out the relative merits of alternative methods. (C) 2015 Elsevier Ltd. All rights reserved.

Risk-Sensitive Ergodic Control of Continuous Time Markov Processes With Denumerable State Space

In this article, we study risk-sensitive control problem with controlled continuous time Markov chain state dynamics. Using multiplicative dynamic programming principle along with the atomic structure of the state dynamics, we prove the existence and a characterization of optimal risk-sensitive control under geometric ergodicity of the state dynamics along with a smallness condition on the running cost.

Bayesian model selection for time series using Markov chain Monte Carlo

We present a stochastic simulation technique for subset selection in time series models, based on the use of indicator variables with the Gibbs sampler within a hierarchical Bayesian framework. As an example, the method is applied to the selection of subset linear AR models, in which only significant lags are included. Joint sampling of the indicators and parameters is found to speed convergence. We discuss the possibility of model mixing where the model is not well determined by the data, and the extension of the approach to include non-linear model terms.

Hacia la aldea virtual : evolución de las tecnologías comunicativas y de los modelos antropológicos

Estudiar los medios de comunicación significa estudiar al hombre, la sociedad en la que vive, su evolución y las perspectivas de desarrollo futuro. Después de haber puntualizado brevemente los avances tecnológicos salientes en la historia del hombre y de haber identificado las características más relevantes de las diversas redefiniciones antropológicas que siguieron a la introducción de algunos nuevos medios, nos detendremos principalmente en el estudio de las peculiaridades de los instrumentos de comunicación de masa más modernos, de la fotografía al cine, de la televisión a Internet. El objetivo principal de este aporte, lejos de querer proveer sistemas teóricos definitivos, es trazar un conjunto coherente de temáticas capaces de suscitar nuevas y proficuas cuestiones y de promover un debate abierto a los más diversos aportes, con particular atención a los avances de las comunicaciones telemáticas.

La moral y los modelos de optimización económica : una reflexión acerca de la antropología en la teoría económica neoclásica

Resumen: Luego de la crisis financiera global de 2008, el blindaje conceptual del paradigma neoclásico, edificado sobre un atractivo formalismo matemático, ha dejado entrever sus falencias. En ese contexto, este artículo plantea la necesidad de revisar sus fundamentos, en especial la concepción antropológica y la metodología que subyace detrás del modelo. El autor analiza al mainstream como un modelo reduccionista de la naturaleza humana, y postula la necesidad de modificar el rumbo de la ciencia económica hacia una dirección personalista, enfocada en el bienestar social de las personas, que considere la importancia de la moral en la toma de decisiones. Asimismo, el economista debería desarrollar su trabajo desde una perspectiva reflexiva y de diálogo con el resto de las ciencias sociales.

Evaluar la erosión hídrica mediante el uso de diferentes modelos de prediccion en San José de los Remates

El área experimental está ubicada en el departamento de Boaco, municipio de San José de los Remate, finca La Primavera cuya ubicación es latitud norte 12°36'43" y longitud oeste 85° 44'07". El objetivo del presente estudio es determinar los factores de la RUSLE Y USLE durante la estación lluviosa del 2008, bajo diferentes sistemas de cubierta vegetal (Grama natural y Bosque nativo). Se estableció un experimento en bloques, con tres repeticiones y dos tratamientos. Cada parcela tiene una dimensión de 50 metros de largo y 15 metros de ancho para un área útil de 750 m2 con un área total por tratamiento de 2,250 m2. El estudio demuestra que las mayores pérdidas de suelo se dieron en las parcelas de Grama natural con un valor promedio de 0.229 t/ha y en las parcelas de Bosque nativo resultaron con pérdidas menores con 0.033t/ha. Además las pérdidas de suelos en todos los eventos fueron relativamente bajas en comparación a los niveles de tolerancia propuestos por Wischmeier y Smith, 1965-1978. Se utilizo la Ecuación Universal de Pérdida de Suelo (E.U.P.S) el cual está compuesta por un total de 6 parámetro como R = 516.48. MJmm/ha h, K = 0.34-0.63 t.ha.h/ha MJ mm, S = 1.6, L = 4.27 (USLE), L*S (0.34–0.39) (RUSLE), C = Grama natural 0.01 y Bosque nativo 0.001, P = no se asumió por no existir práctica.Para el análisis de la información; se utilizó como método estadístico T student con un grado de significancia del 95 % los efectos de las diferentes variables relacionada a los procesos de erosión del suelo y del escurrimiento superficial resultando no significativos entre modelos.

Parameter Estimation for Hidden Markov Models with Intractable Likelihoods

Approximate Bayesian computation (ABC) is a popular technique for analysing data for complex models where the likelihood function is intractable. It involves using simulation from the model to approximate the likelihood, with this approximate likelihood then being used to construct an approximate posterior. In this paper, we consider methods that estimate the parameters by maximizing the approximate likelihood used in ABC. We give a theoretical analysis of the asymptotic properties of the resulting estimator. In particular, we derive results analogous to those of consistency and asymptotic normality for standard maximum likelihood estimation. We also discuss how sequential Monte Carlo methods provide a natural method for implementing our likelihood-based ABC procedures.

Bayesian learning of noisy Markov decision processes

This work addresses the problem of estimating the optimal value function in a Markov Decision Process from observed state-action pairs. We adopt a Bayesian approach to inference, which allows both the model to be estimated and predictions about actions to be made in a unified framework, providing a principled approach to mimicry of a controller on the basis of observed data. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from theposterior distribution over the optimal value function. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.