Stochastic exponential stability of singular linear systems with Markov jump parameters

This paper deals with exponential stability of discrete-time singular systems with Markov jump parameters. We propose a set of coupled generalized Lyapunov equations (CGLE) that provides sufficient conditions to check this property for this class of systems. A method for solving the obtained CGLE is also presented, based on iterations of standard singular Lyapunov equations. We present also a numerical example to illustrate the effectiveness of the approach we are proposing.

Aplicacion de un proceso en cadena de Markov a la proyeccion de la mortalidad, por sexo y grupos de edades

Incluye Bibliografía

OPF-MRF: Optimum-path forest and Markov random fields for contextual-based image classification

Some machine learning methods do not exploit contextual information in the process of discovering, describing and recognizing patterns. However, spatial/temporal neighboring samples are likely to have same behavior. Here, we propose an approach which unifies a supervised learning algorithm - namely Optimum-Path Forest - together with a Markov Random Field in order to build a prior model holding a spatial smoothness assumption, which takes into account the contextual information for classification purposes. We show its robustness for brain tissue classification over some images of the well-known dataset IBSR. © 2013 Springer-Verlag.

Extração automática de contornos de telhados de edifícios em um modelo digital de elevação, utilizando inferência Bayesiana e campos aleatórios de Markov

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Green-Markov models - new optimization strategies: a case study for user allocation in co-channel macro/femto networks

ABSTRACT: The femtocell concept aims to combine fixed-line broadband access with mobile telephony using the deployment of low-cost, low-power third and fourth generation base stations in the subscribers' homes. While the self-configuration of femtocells is a plus, it can limit the quality of service (QoS) for the users and reduce the efficiency of the network, based on outdated allocation parameters such as signal power level. To this end, this paper presents a proposal for optimized allocation of users on a co-channel macro-femto network, that enable self-configuration and public access, aiming to maximize the quality of service of applications and using more efficiently the available energy, seeking the concept of Green networking. Thus, when the user needs to connect to make a voice or a data call, the mobile phone has to decide which network to connect, using the information of number of connections, the QoS parameters (packet loss and throughput) and the signal power level of each network. For this purpose, the system is modeled as a Markov Decision Process, which is formulated to obtain an optimal policy that can be applied on the mobile phone. The policy created is flexible, allowing different analyzes, and adaptive to the specific characteristics defined by the telephone company. The results show that compared to traditional QoS approaches, the policy proposed here can improve energy efficiency by up to 10%.

Green markov - uma abordagem híbrida de política markoviana e simulação discreta para planejamento de alocação de usuários em redes macro-femto

O uso da comunicação de voz e dados através de dispositivos móveis vem aumentando significativamente nos últimos anos. Tal expansão traz algumas dificuldades inerentes, tais como: ampliação constante de capacidade das redes e eficiência energética. Neste contexto, vem se consolidando o conceito de Green networks, que se concentra no esforço para economia de energia e redução de CO2. Neste sentido, este trabalho propõe validar um modelo de uma política baseado em processo markoviano de decisão, visando a otimizar o consumo de energia, QoS e QoE, na alocação de usuários em redes macrocell e femtocell. Para isso o modelo foi inserido no simulador NS-2, aliando a solução analítica markoviana à flexibilidade característica da simulação discreta. A partir dos resultados apresentados na simulação, a política obteve uma economia significativa no consumo energético, melhorando a eficiência energética em até 4%, além de melhorar a qualidade de serviço em relação às redes macrocell e femtocell, demonstrando-se eficaz, de modo a alterar diretamente as métricas de QoS e de QoE.

Evolução das disparidades da extração vegetal e da silvicultura na Amazônia Legal: uma aplicação da cadeia de Markov

O artigo analisa a convergência municipal da produtividade vegetal (extração vegetal e silvicultura) na região da Amazônia Legal entre os anos de 1996 e 2006. Para analisar a convergência, optou-se pela metodologia da matriz de transição de Markov (Processo Estacionário de Primeira Ordem de Markov). Os resultados mostram a existência de 13 classes de convergência da produtividade vegetal. No longo prazo, a hipótese de convergência absoluta não se mantém, visto que 68,23% dos municípios encontram-se numa classe inferior à média municipal, 33,54% em uma classe intermediária acima da média e 13,41% em uma classe superior acima da média.

Aplicações das Cadeias de Markov para o Ensino Médio

Pós-graduação em Matemática em Rede Nacional - IBILCE

Técnicas de otimização em alinhamentos múltiplos de sequência via Cadeias de Markov

Pós-graduação em Ciência da Computação - IBILCE

Partially Observed Markov Random Fields Are Variable Neighborhood Random Fields

The present paper has two goals. First to present a natural example of a new class of random fields which are the variable neighborhood random fields. The example we consider is a partially observed nearest neighbor binary Markov random field. The second goal is to establish sufficient conditions ensuring that the variable neighborhoods are almost surely finite. We discuss the relationship between the almost sure finiteness of the interaction neighborhoods and the presence/absence of phase transition of the underlying Markov random field. In the case where the underlying random field has no phase transition we show that the finiteness of neighborhoods depends on a specific relation between the noise level and the minimum values of the one-point specification of the Markov random field. The case in which there is phase transition is addressed in the frame of the ferromagnetic Ising model. We prove that the existence of infinite interaction neighborhoods depends on the phase.

Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Reconocimiento de formas manuscritas con modelos ocultos de Markov

Programa de doctorado Tecnologías de las Telecomunicaciones

Modelización econométrica a través de cadenas de Markov de Monte Carlo (MCMC) en valoración contingente

Doctorado en Análisis Económico. Programa en Análisis Económico Aplicado