Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Integrating on-line process control and imperfect corrective maintenance: An economical design

Existing studies of on-line process control are concerned with economic aspects, and the parameters of the processes are optimized with respect to the average cost per item produced. However, an equally important dimension is the adoption of an efficient maintenance policy. In most cases, only the frequency of the corrective adjustment is evaluated because it is assumed that the equipment becomes "as good as new" after corrective maintenance. For this condition to be met, a sophisticated and detailed corrective adjustment system needs to be employed. The aim of this paper is to propose an integrated economic model incorporating the following two dimensions: on-line process control and a corrective maintenance program. Both performances are objects of an average cost per item minimization. Adjustments are based on the location of the measurement of a quality characteristic of interest in a three decision zone. Numerical examples are illustrated in the proposal. (c) 2012 Elsevier B.V. All rights reserved.

The role of evaluation in decision-making in the management of health services

The management of health services is a complex administrative practice due to the breadth of the field of health and the need to reconcile individual, corporate and collective interests that are not always convergent. In this context, the evaluation needs to have specific characteristics in order to fulfill its role. The scope of this study was to establish the characteristics that the evaluation for the management of health services should have to contribute to decision-making. Usefulness, opportunity, feasibility, reliability, objectivity and directionality represent the set of principles upon which the evaluation should be based. Evaluations should lead to decisions that guarantee not only their efficiency and effectiveness but also their implementation. The evaluation process should ensure that decisions involve all stakeholders in order to render the implementation of decisions feasible, and take into account the health needs of the population and the goals set for the services. The scope of this article is to elicit a debate among different stakeholders in the evaluation in the hope that it can contribute to the reflection on the real usefulness of evaluations in which the political component in management has been increasingly prevalent.

Feelings of women regarding end-of-life decision making after ultrasound diagnosis of a lethal fetal malformation

Objective: this study investigated the feelings of women regarding end-of-life decision making after ultrasound diagnosis of a lethal fetal malformation. The aim of this study was to present the decision making process of women that chose for pregnancy termination and to present selected speeches of women about their feelings. Design: open psychological interviews conducted by a psychologist immediately after the diagnosis of fetal malformation by ultrasound. Analysis of the results was performed through a content analysis technique. Setting: the study was carried out at a public university hospital in Brazil. Participants: 249 pregnant women who had received the diagnosis of a severe lethal fetal malformation. Findings: fetal anencephaly was the most frequent anomaly detected in 135 cases (54.3%). Termination of pregnancy was decided by 172 (69.1%) patients and legally authorised by the judiciary (66%). The reason for asking for termination was to reduce suffering in all of them. In the 77 women who chose not to terminate pregnancy (30.9%), the reasons were related to feelings of guilt (74%). Key conclusions: the results support the importance of psychological counselling for couples when lethal fetal malformation is diagnosed. The act of reviewing moral and cultural values and elements of the unconscious provides assurance in the decision-making process and mitigates the risk of emotional trauma and guilt that can continue long after the pregnancy is terminated. (C) 2011 Elsevier Ltd. All rights reserved.

A new method for decision making in multi-objective optimization problems

Many engineering sectors are challenged by multi-objective optimization problems. Even if the idea behind these problems is simple and well established, the implementation of any procedure to solve them is not a trivial task. The use of evolutionary algorithms to find candidate solutions is widespread. Usually they supply a discrete picture of the non-dominated solutions, a Pareto set. Although it is very interesting to know the non-dominated solutions, an additional criterion is needed to select one solution to be deployed. To better support the design process, this paper presents a new method of solving non-linear multi-objective optimization problems by adding a control function that will guide the optimization process over the Pareto set that does not need to be found explicitly. The proposed methodology differs from the classical methods that combine the objective functions in a single scale, and is based on a unique run of non-linear single-objective optimizers.