Ergodicidade em cadeias de Markov não-homogêneas e cadeias de Markov com transições raras

The central objective of a study Non-Homogeneous Markov Chains is the concept of weak and strong ergodicity. A chain is weak ergodic if the dependence on the initial distribution vanishes with time, and it is strong ergodic if it is weak ergodic and converges in distribution. Most theoretical results on strong ergodicity assume some knowledge of the limit behavior of the stationary distributions. In this work, we collect some general results on weak and strong ergodicity for chains with space enumerable states, and also study the asymptotic behavior of the stationary distributions of a particular type of Markov Chains with finite state space, called Markov Chains with Rare Transitions

Evolução das disparidades da extração vegetal e da silvicultura na Amazônia Legal: uma aplicação da cadeia de Markov

O artigo analisa a convergência municipal da produtividade vegetal (extração vegetal e silvicultura) na região da Amazônia Legal entre os anos de 1996 e 2006. Para analisar a convergência, optou-se pela metodologia da matriz de transição de Markov (Processo Estacionário de Primeira Ordem de Markov). Os resultados mostram a existência de 13 classes de convergência da produtividade vegetal. No longo prazo, a hipótese de convergência absoluta não se mantém, visto que 68,23% dos municípios encontram-se numa classe inferior à média municipal, 33,54% em uma classe intermediária acima da média e 13,41% em uma classe superior acima da média.

Aplicações das Cadeias de Markov para o Ensino Médio

Pós-graduação em Matemática em Rede Nacional - IBILCE

Técnicas de otimização em alinhamentos múltiplos de sequência via Cadeias de Markov

Pós-graduação em Ciência da Computação - IBILCE

New markov chain based methods for single and cross-domain sentiment classification

Nowadays communication is switching from a centralized scenario, where communication media like newspapers, radio, TV programs produce information and people are just consumers, to a completely different decentralized scenario, where everyone is potentially an information producer through the use of social networks, blogs, forums that allow a real-time worldwide information exchange. These new instruments, as a result of their widespread diffusion, have started playing an important socio-economic role. They are the most used communication media and, as a consequence, they constitute the main source of information enterprises, political parties and other organizations can rely on. Analyzing data stored in servers all over the world is feasible by means of Text Mining techniques like Sentiment Analysis, which aims to extract opinions from huge amount of unstructured texts. This could lead to determine, for instance, the user satisfaction degree about products, services, politicians and so on. In this context, this dissertation presents new Document Sentiment Classification methods based on the mathematical theory of Markov Chains. All these approaches bank on a Markov Chain based model, which is language independent and whose killing features are simplicity and generality, which make it interesting with respect to previous sophisticated techniques. Every discussed technique has been tested in both Single-Domain and Cross-Domain Sentiment Classification areas, comparing performance with those of other two previous works. The performed analysis shows that some of the examined algorithms produce results comparable with the best methods in literature, with reference to both single-domain and cross-domain tasks, in $2$-classes (i.e. positive and negative) Document Sentiment Classification. However, there is still room for improvement, because this work also shows the way to walk in order to enhance performance, that is, a good novel feature selection process would be enough to outperform the state of the art. Furthermore, since some of the proposed approaches show promising results in $2$-classes Single-Domain Sentiment Classification, another future work will regard validating these results also in tasks with more than $2$ classes.

Searching for convergence in phylogenetic Markov chain Monte Carlo

Markov chain Monte Carlo (MCMC) is a methodology that is gaining widespread use in the phylogenetics community and is central to phylogenetic software packages such as MrBayes. An important issue for users of MCMC methods is how to select appropriate values for adjustable parameters such as the length of the Markov chain or chains, the sampling density, the proposal mechanism, and, if Metropolis-coupled MCMC is being used, the number of heated chains and their temperatures. Although some parameter settings have been examined in detail in the literature, others are frequently chosen with more regard to computational time or personal experience with other data sets. Such choices may lead to inadequate sampling of tree space or an inefficient use of computational resources. We performed a detailed study of convergence and mixing for 70 randomly selected, putatively orthologous protein sets with different sizes and taxonomic compositions. Replicated runs from multiple random starting points permit a more rigorous assessment of convergence, and we developed two novel statistics, delta and epsilon, for this purpose. Although likelihood values invariably stabilized quickly, adequate sampling of the posterior distribution of tree topologies took considerably longer. Our results suggest that multimodality is common for data sets with 30 or more taxa and that this results in slow convergence and mixing. However, we also found that the pragmatic approach of combining data from several short, replicated runs into a metachain to estimate bipartition posterior probabilities provided good approximations, and that such estimates were no worse in approximating a reference posterior distribution than those obtained using a single long run of the same length as the metachain. Precision appears to be best when heated Markov chains have low temperatures, whereas chains with high temperatures appear to sample trees with high posterior probabilities only rarely. [Bayesian phylogenetic inference; heating parameter; Markov chain Monte Carlo; replicated chains.]

A Markov chain location-allocation meta-model for hurricane relief

An emergency is a deviation from a planned course of events that endangers people, properties, or the environment. It can be described as an unexpected event that causes economic damage, destruction, and human suffering. When a disaster happens, Emergency Managers are expected to have a response plan to most likely disaster scenarios. Unlike earthquakes and terrorist attacks, a hurricane response plan can be activated ahead of time, since a hurricane is predicted at least five days before it makes landfall. This research looked into the logistics aspects of the problem, in an attempt to develop a hurricane relief distribution network model. We addressed the problem of how to efficiently and effectively deliver basic relief goods to victims of a hurricane disaster. Specifically, where to preposition State Staging Areas (SSA), which Points of Distributions (PODs) to activate, and the allocation of commodities to each POD. Previous research has addressed several of these issues, but not with the incorporation of the random behavior of the hurricane's intensity and path. This research presents a stochastic meta-model that deals with the location of SSAs and the allocation of commodities. The novelty of the model is that it treats the strength and path of the hurricane as stochastic processes, and models them as Discrete Markov Chains. The demand is also treated as stochastic parameter because it depends on the stochastic behavior of the hurricane. However, for the meta-model, the demand is an input that is determined using Hazards United States (HAZUS), a software developed by the Federal Emergency Management Agency (FEMA) that estimates losses due to hurricanes and floods. A solution heuristic has been developed based on simulated annealing. Since the meta-model is a multi-objective problem, the heuristic is a multi-objective simulated annealing (MOSA), in which the initial solution and the cooling rate were determined via a Design of Experiments. The experiment showed that the initial temperature (T0) is irrelevant, but temperature reduction (δ) must be very gradual. Assessment of the meta-model indicates that the Markov Chains performed as well or better than forecasts made by the National Hurricane Center (NHC). Tests of the MOSA showed that it provides solutions in an efficient manner. Thus, an illustrative example shows that the meta-model is practical.

O uso de modelos ocultos de Markov no estudo do fluxo de rios intermitentes

In this work, we present our understanding about the article of Aksoy [1], which uses Markov chains to model the flow of intermittent rivers. Then, we executed an application of his model in order to generate data for intermittent streamflows, based on a data set of Brazilian streams. After that, we build a hidden Markov model as a proposed new approach to the problem of simulation of such flows. We used the Gamma distribution to simulate the increases and decreases in river flows, along with a two-state Markov chain. The motivation for us to use a hidden Markov model comes from the possibility of obtaining the same information that the Aksoy’s model provides, but using a single tool capable of treating the problem as a whole, and not through multiple independent processes

Likelihood-Based Inference for Partially Observed Multi-Type Markov Branching Processes

Thesis (Ph.D.)--University of Washington, 2016-08

Análise do crescimento econômico no Rio Grande do Sul através da aplicação de cadeias de Markov

O estudo do crescimento econômico é de suma importância para que possamos averiguar a trajetória de uma economia ao longo do tempo, a proposta desse trabalho é analisar o crescimento econômico no estado do Rio Grande do Sul, através do instrumental das cadeias de Markov, a ideia principal do estudo está na hipótese de convergência de renda. Primeiramente será testado a hipótese de convergência de renda do estado por meio das microrregiões, para isso serão utilizados dados de produto per capita dos anos de 1990, 2000 e 2010. Também será testado a hipótese de convergência para os municípios do Conselho Regional de Desenvolvimento Sul, situado no Rio Grande do Sul, utilizando dados de renda per capita dos anos de 1991, 2000 e 2010. Os resultados obtidos para as microrregiões do Rio Grande do Sul mostram que as economias não estão convergindo em sua totalidade para uma classe de renda especifica, porém é percebido que no longo prazo haverá uma maior concentração das microrregiões nos extratos de renda próximos a média, o tempo esperado para que as economias cheguem ao seu estado estacionário é de seis períodos. Por meio dos resultados obtidos para a região do Corede Sul, temos que as economias convergirão em sua maioria para a classe de renda médio pobre, seguido pela classe dos médios ricos. Ambas as classes estão situadas próximas a média regional, sendo que as classes de renda pobre e rico situadas aos extremos serão extintas no longo prazo. O tempo esperado para que as economias cheguem ao estado estacionário é de onze períodos.

Análise do crescimento econômico no Rio Grande do Sul através da aplicação de cadeias de markov

O estudo do crescimento econômico é de suma importância para que possamos averiguar a trajetória de uma economia ao longo do tempo, a proposta desse trabalho é analisar o crescimento econômico no estado do Rio Grande do Sul, através do instrumental das cadeias de Markov, a ideia principal do estudo está na hipótese de convergência de renda. Primeiramente será testado a hipótese de convergência de renda do estado por meio das microrregiões, para isso serão utilizados dados de produto per capita dos anos de 1990, 2000 e 2010. Também será testado a hipótese de convergência para os municípios do Conselho Regional de Desenvolvimento Sul, situado no Rio Grande do Sul, utilizando dados de renda per capita dos anos de 1991, 2000 e 2010. Os resultados obtidos para as microrregiões do Rio Grande do Sul mostram que as economias não estão convergindo em sua totalidade para uma classe de renda especifica, porém é percebido que no longo prazo haverá uma maior concentração das microrregiões nos extratos de renda próximos a média, o tempo esperado para que as economias cheguem ao seu estado estacionário é de seis períodos. Por meio dos resultados obtidos para a região do Corede Sul, temos que as economias convergirão em sua maioria para a classe de renda médio pobre, seguido pela classe dos médio ricos. Ambas as classes estão situadas próximas a média regional, sendo que as classes de renda pobre e rico situadas aos extremos serão extintas no longo prazo. O tempo esperado para que as economias cheguem ao estado estacionário é de onze períodos.

A path-based framework for analyzing large markov models

Reliability and dependability modeling can be employed during many stages of analysis of a computing system to gain insights into its critical behaviors. To provide useful results, realistic models of systems are often necessarily large and complex. Numerical analysis of these models presents a formidable challenge because the sizes of their state-space descriptions grow exponentially in proportion to the sizes of the models. On the other hand, simulation of the models requires analysis of many trajectories in order to compute statistically correct solutions. This dissertation presents a novel framework for performing both numerical analysis and simulation. The new numerical approach computes bounds on the solutions of transient measures in large continuous-time Markov chains (CTMCs). It extends existing path-based and uniformization-based methods by identifying sets of paths that are equivalent with respect to a reward measure and related to one another via a simple structural relationship. This relationship makes it possible for the approach to explore multiple paths at the same time,· thus significantly increasing the number of paths that can be explored in a given amount of time. Furthermore, the use of a structured representation for the state space and the direct computation of the desired reward measure (without ever storing the solution vector) allow it to analyze very large models using a very small amount of storage. Often, path-based techniques must compute many paths to obtain tight bounds. In addition to presenting the basic path-based approach, we also present algorithms for computing more paths and tighter bounds quickly. One resulting approach is based on the concept of path composition whereby precomputed subpaths are composed to compute the whole paths efficiently. Another approach is based on selecting important paths (among a set of many paths) for evaluation. Many path-based techniques suffer from having to evaluate many (unimportant) paths. Evaluating the important ones helps to compute tight bounds efficiently and quickly.

Neighborhood disadvantage and physical activity : baseline results from the HABITAT multilevel longitudinal study

PURPOSE: To examine the association between neighborhood disadvantage and physical activity (PA). ---------- METHODS: We use data from the HABITAT multilevel longitudinal study of PA among mid-aged (40-65 years) men and women (n=11, 037, 68.5% response rate) living in 200 neighborhoods in Brisbane, Australia. PA was measured using three questions from the Active Australia Survey (general walking, moderate, and vigorous activity), one indicator of total activity, and two questions about walking and cycling for transport. The PA measures were operationalized using multiple categories based on time and estimated energy expenditure that were interpretable with reference to the latest PA recommendations. The association between neighborhood disadvantage and PA was examined using multilevel multinomial logistic regression and Markov Chain Monte Carlo simulation. The contribution of neighborhood disadvantage to between-neighborhood variation in PA was assessed using the 80% interval odds ratio. ---------- RESULTS: After adjustment for sex, age, living arrangement, education, occupation, and household income, reported participation in all measures and levels of PA varied significantly across Brisbane’s neighborhoods, and neighborhood disadvantage accounted for some of this variation. Residents of advantaged neighborhoods reported significantly higher levels of total activity, general walking, moderate, and vigorous activity; however, they were less likely to walk for transport. There was no statistically significant association between neighborhood disadvantage and cycling for transport. In terms of total PA, residents of advantaged neighborhoods were more likely to exceed PA recommendations. ---------- CONCLUSIONS: Neighborhoods may exert a contextual effect on residents’ likelihood of participating in PA. The greater propensity of residents in advantaged neighborhoods to do high levels of total PA may contribute to lower rates of cardiovascular disease and obesity in these areas

Reading the play : adaptation by prediction of agent motion

An adaptive agent improves its performance by learning from experience. This paper describes an approach to adaptation based on modelling dynamic elements of the environment in order to make predictions of likely future state. This approach is akin to an elite sports player being able to “read the play”, allowing for decisions to be made based on predictions of likely future outcomes. Modelling of the agent‟s likely future state is performed using Markov Chains and a technique called “Motion and Occupancy Grids”. The experiments in this paper compare the performance of the planning system with and without the use of this predictive model. The results of the study demonstrate a surprising decrease in performance when using the predictions of agent occupancy. The results are derived from statistical analysis of the agent‟s performance in a high fidelity simulation of a world leading real robot soccer team.

Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.