Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Contribution to economic evaluation of systems that value animal welfare at farm

The demand for "welfare friendly" products increases as public conscience and perception on livestock production systems grow. The public and policy-makers demand scientific information for education and to guide decision processes. This paper describes some of the last decade contributions made by scientists on the technical, economical and market areas of farm animal welfare. Articles on animal welfare were compiled on the following themes: 1) consumer behavior, 2) technical and economical viability, 3) public regulation, and 4) private certification policies. Most studies on the economic evaluation of systems that promote animal welfare involved species destined to produce export items, such as eggs, beef and pork. Few studies were found on broilers, dairy cows and fish, and data regarding other species, such as horses, sheep and goats were not found. Scientists understand that farm animal welfare is not only a matter of ethics, but also an essential tool to gain and maintain markets. However, it is unfortunate that little attention is paid to species that are not economically important for exports. Studies that emphasize on more humane ways to raise animals and that provide economic incentives to the producer are needed. An integrated multidisciplinary approach is necessary to highlight the benefits of introducing animal welfare techniques to existing production systems.

Cost-effectiveness of introducing the 10-valent pneumococcal conjugate vaccine into the universal immunisation of infants in Brazil

Background Cost-effectiveness studies have been increasingly part of decision processes for incorporating new vaccines into the Brazilian National Immunisation Program. This study aimed to evaluate the cost-effectiveness of 10-valent pneumococcal conjugate vaccine (PCV10) in the universal childhood immunisation programme in Brazil. Methods A decision-tree analytical model based on the ProVac Initiative pneumococcus model was used, following 25 successive cohorts from birth until 5 years of age. Two strategies were compared: (1) status quo and (2) universal childhood immunisation programme with PCV10. Epidemiological and cost estimates for pneumococcal disease were based on National Health Information Systems and literature. A 'top-down' costing approach was employed. Costs are reported in 2004 Brazilian reals. Costs and benefits were discounted at 3%. Results 25 years after implementing the PCV10 immunisation programme, 10 226 deaths, 360 657 disability-adjusted life years (DALYs), 433 808 hospitalisations and 5 117 109 outpatient visits would be avoided. The cost of the immunisation programme would be R$10 674 478 765, and the expected savings on direct medical costs and family costs would be R$1 036 958 639 and R$209 919 404, respectively. This resulted in an incremental cost-effectiveness ratio of R$778 145/death avoided and R$22 066/DALY avoided from the society perspective. Conclusion The PCV10 universal infant immunisation programme is a cost-effective intervention (1-3 GDP per capita/DALY avoided). Owing to the uncertain burden of disease data, as well as unclear long-term vaccine effects, surveillance systems to monitor the long-term effects of this programme will be essential.

Integrating on-line process control and imperfect corrective maintenance: An economical design

Existing studies of on-line process control are concerned with economic aspects, and the parameters of the processes are optimized with respect to the average cost per item produced. However, an equally important dimension is the adoption of an efficient maintenance policy. In most cases, only the frequency of the corrective adjustment is evaluated because it is assumed that the equipment becomes "as good as new" after corrective maintenance. For this condition to be met, a sophisticated and detailed corrective adjustment system needs to be employed. The aim of this paper is to propose an integrated economic model incorporating the following two dimensions: on-line process control and a corrective maintenance program. Both performances are objects of an average cost per item minimization. Adjustments are based on the location of the measurement of a quality characteristic of interest in a three decision zone. Numerical examples are illustrated in the proposal. (c) 2012 Elsevier B.V. All rights reserved.