Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Archaeofacies Analysis: Using Depositional Attributes to Identify Anthropic Processes of Deposition in a Monumental Shell Mound of Santa Catarina State, Southern Brazil

In the coast of Santa Catarina State (southern Brazil), a large population of monumental shell mounds characterizes a highly dynamic coastal setting. In this paper, sedimentary facies analysis was adapted for description, sampling, and interpretation of shell mound complex and repetitive archaeostratigraphic successions. Archaeofacies identification in the field, according to depositional attributes, is tested by contrasting field description with multi-element chemical analyses, total carbon and nitrogen determinations, and micromorphological descriptions. Two vertical sequences at the black deposit of Jabuticabeira II shell mound were studied and preliminary results showed that: (1) depositional attributes are a reliable base for archaeofacies identification in the field, (2) the formation process of this site involved a sequence of anthropic depositional processes, where burned refuse was redeposited over the shell mound following a ritual construction pattern, and (3) the black deposit includes a double palimpsest that refers to provenance and meaning of mound construction material. (C) 2009 Wiley Periodicals, Inc.

Effective carrying capacity and analytical solution of a particular case of the Richards-like two-species population dynamics model

We consider a generalized two-species population dynamic model and analytically solve it for the amensalism and commensalism ecological interactions. These two-species models can be simplified to a one-species model with a time dependent extrinsic growth factor. With a one-species model with an effective carrying capacity one is able to retrieve the steady state solutions of the previous one-species model. The equivalence obtained between the effective carrying capacity and the extrinsic growth factor is complete only for a particular case, the Gompertz model. Here we unveil important aspects of sigmoid growth curves, which are relevant to growth processes and population dynamics. (C) 2011 Elsevier B.V. All rights reserved.

Cartographic notes on the work in the Family Health Strategy: relationships between workers and the population

The objective of this intervention study was to map instituted and instituting movements present in the work of the Family Health Strategy in the development of their care practices. The theoretical framework is based on institutional analysis, using the schizoanalytic approach. Group meetings were carried out with the staff to discuss how they provided collective care in continuing health education. The study subjects were professionals from the team and students who were engaged in academic activity in the service. The average attendance was twelve people per meeting, and there were a total of eight meetings from March to July 2010. Data were grouped into two immanent strata: the relationships of the team and the relationship with clients. The strata point to the intersection of education and legal institutions and the social and technical division of labor. Collective thinking in groups appeared to be effective in denaturalizing established processes and interrogating places, knowledge and practices.

Population size of threatened and endemic birds of the Cerrado in Estacao Ecologica de Itirapina, a fragmented area in the State of Sao Paulo, Brazil

The pressures for land use change have led to an increasing isolation of habitat remnants throughout the world. The goal of this study was to estimate the population size and density of some endemic and threatened species in a nature reserve in the Cerrado biome. One hundred and thirty four point transects were undertaken at the Estacao Ecologica de Itirapina (EEI), one of the last natural grassland savannah remnants in Sao Paulo state, in the south-east of Brazil between September and December 2006 and densities estimated for seven species (four endemic to the Cerrado, one near-endemic and two grassland specialists). Neither species reached the minimum viable population size of 500-5000 individuals. Four species, White-banded Tanager, White-rumped Tanager, Black-throated Saltator and Sharp-tailed Tyrant have populations ranging from 112 to 248 individuals, while the other species have a low population (< 60 individuals). The mean densities of Sharp-tailed Tyrant and Cock-tailed Tyrant in the EEI grassland showed similar values to those observed in larger areas of the Cerrado, which may indicate that the EEL grassland area is well conserved. In spite of the restricted size of the EEI, small areas can maintain some endemic and threatened bird populations, thus contributing to local biodiversity and the ecological processes in the region. The capacity of fragments of Cerrado (similar to 2,000 ha) to maintain populations of endemic and threatened bird species is unlikely to be effective in the long term.

Thioredoxin interacting protein (TXNIP) rs7212 polymorphism is associated with arterial stiffness in the Brazilian general population

Thioredoxin interacting protein plays a pivotal role in several important processes of cardiovascular homeostasis by functioning as a biological sensor for biomechanical and oxidative stress. However, the effects of genetic variants in the modulation of arterial stiffness are unknown. In this scenario, the present study evaluated the relationship between the TXNIP rs7212 polymorphism and arterial stiffness. In the overall sample and in the diabetic group, individuals carrying CG + GG genotypes had higher PWV values compared with CC genotype group ( 10.0 vs 9.8 ms(-1), P = 0.03; 12.3 vs 11.2 ms(-1), P = 0.01; respectively). Our findings indicated that the G allele may contribute to increased arterial stiffness in the Brazilian general population and suggest a possible interaction with diabetes.

A stochastic spatially structured epidemic model with diffusive processes

We developed a stochastic lattice model to describe the vector-borne disease (like yellow fever or dengue). The model is spatially structured and its dynamical rules take into account the diffusion of vectors. We consider a bipartite lattice, forming a sub-lattice of human and another occupied by mosquitoes. At each site of lattice we associate a stochastic variable that describes the occupation and the health state of a single individual (mosquito or human). The process of disease transmission in the human population follows a similar dynamic of the Susceptible-Infected-Recovered model (SIR), while the disease transmission in the mosquito population has an analogous dynamic of the Susceptible-Infected-Susceptible model (SIS) with mosquitos diffusion. The occurrence of an epidemic is directly related to the conditional probability of occurrence of infected mosquitoes (human) in the presence of susceptible human (mosquitoes) on neighborhood. The probability of diffusion of mosquitoes can facilitate the formation of pairs Susceptible-Infected enabling an increase in the size of the epidemic. Using an asynchronous dynamic update, we study the disease transmission in a population initially formed by susceptible individuals due to the introduction of a single mosquito (human) infected. We find that this model exhibits a continuous phase transition related to the existence or non-existence of an epidemic. By means of mean field approximations and Monte Carlo simulations we investigate the epidemic threshold and the phase diagram in terms of the diffusion probability and the infection probability.