Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation.

Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Malnutrition and its impact on cost of hospitalization, length of stay, readmission and 3-year mortality

Background & aims: The confounding effect of disease on the outcomes of malnutrition using diagnosis-related groups (DRG) has never been studied in a multidisciplinary setting. This study aims to determine the prevalence of malnutrition in a tertiary hospital in Singapore and its impact on hospitalization outcomes and costs, controlling for DRG. Methods: This prospective cohort study included a matched case control study. Subjective Global Assessment was used to assess the nutritional status on admission of 818 adults. Hospitalization outcomes over 3 years were adjusted for gender, age, ethnicity, and matched for DRG. Results: Malnourished patients (29%) had longer hospital stays (6.9 ± 7.3 days vs. 4.6 ± 5.6 days, p < 0.001) and were more likely to be readmitted within 15 days (adjusted relative risk = 1.9, 95%CI 1.1–3.2, p = 0.025). Within a DRG, the mean difference between actual cost of hospitalization and the average cost for malnourished patients was greater than well-nourished patients (p = 0.014). Mortality was higher in malnourished patients at 1 year (34% vs. 4.1 %), 2 years (42.6% vs. 6.7%) and 3 years (48.5% vs. 9.9%); p < 0.001 for all. Overall, malnutrition was a significant predictor of mortality (adjusted hazard ratio = 4.4, 95% CI 3.3-6.0, p < 0.001). Conclusions: Malnutrition was evident in up to one third of the inpatients and led to poor hospitalization outcomes and survival as well as increased costs of care, even after matching for DRG. Strategies to prevent and treat malnutrition in the hospital and post-discharge are needed.

A cost-outcome description of the septic work-up for bacterial infection in neonates in a tertiary care hospital

Analysis of the septic work-up of 194 neonates at Women's College Hospital, Toronto, showed that the only antepartum condition predicting neonatal sepsis was the mother being on antibiotics. The only postnatal condition predicting sepsis was a maternal postpartum white blood cell count over 11,000. The average cost for tests for a septic work-up in these 194 mother-neonate pairs was $71.48 (Canadian dollars), and the average cost of tests to find a septic case was $1,066.77.

Malnutrition and its impact on cost of hospitalization, length of stay, readmission and 3-year mortality

Background & aims The confounding effect of disease on the outcomes of malnutrition using diagnosis-related groups (DRG) has never been studied in a multidisciplinary setting. This study aims to determine the impact of malnutrition on hospitalisation outcomes, controlling for DRG. Methods Subjective Global Assessment was used to assess the nutritional status of 818 patients within 48 hours of admission. Prospective data were collected on cost of hospitalisation, length of stay (LOS), readmission and mortality up to 3 years post-discharged using National Death Register data. Mixed model analysis and conditional logistic regression matching by DRG were carried out to evaluate the association between nutritional status and outcomes, with the results adjusted for gender, age and race. Results Malnourished patients (29%) had longer hospital stays (6.9±7.3 days vs. 4.6±5.6 days, p<0.001) and were more likely to be readmitted within 15 days (adjusted relative risk = 1.9, 95%CI 1.1–3.2, p=0.025). Within a DRG, the mean difference between actual cost of hospitalisation and the average cost for malnourished patients was greater than well-nourished patients (p=0.014). Mortality was higher in malnourished patients at 1 year (34% vs. 4.1 %), 2 years (42.6% vs. 6.7%) and 3 years (48.5% vs. 9.9%); p<0.001 for all. Overall, malnutrition was a significant predictor of mortality (adjusted hazard ratio = 4.4, 95%CI 3.3-6.0, p<0.001). Conclusions Malnutrition was evident in up to one third of inpatients and led to poor hospitalisation outcomes, even after matching for DRG. Strategies to prevent and treat malnutrition in the hospital and post-discharge are needed.

How Much Does It Cost to Provide Antiretroviral Therapy for HIV/AIDS in Africa?

Background: Many African countries are rapidly expanding HIV/AIDS treatment programs. Empirical information on the cost of delivering antiretroviral therapy (ART) for HIV/AIDS is needed for program planning and budgeting. Methods: We searched published and gray sources for estimates of the cost of providing ART in service delivery (non-research) settings in sub-Saharan Africa. Estimates were included if they were based on primary local data for input prices. Results: 17 eligible cost estimates were found. Of these, 10 were from South Africa. The cost per patient per year ranged from $396 to $2,761. It averaged approximately $850/patient/year in countries outside South Africa and $1,700/patient/year in South Africa. The most recent estimates for South Africa averaged $1,200/patient/year. Specific cost items included in the average cost per patient per year varied, making comparison across studies problematic. All estimates included the cost of antiretroviral drugs and laboratory tests, but many excluded the cost of inpatient care, treatment of opportunistic infections, and/or clinic infrastructure. Antiretroviral drugs comprised an average of one third of the cost of treatment in South Africa and one half to three quarters of the cost in other countries. Conclusions: There is very little empirical information available about the cost of providing antiretroviral therapy in non-research settings in Africa. Methods for estimating costs are inconsistent, and many estimates combine data drawn from disparate sources. Cost analysis should become a routine part of operational research on the treatment rollout in Africa.

Cost of innovation in the pharmaceutical industry.

The research and development costs of 93 randomly selected new chemical entities (NCEs) were obtained from a survey of 12 U.S.-owned pharmaceutical firms. These data were used to estimate the pre-tax average cost of new drug development. The costs of abandoned NCEs were linked to the costs of NCEs that obtained marketing approval. For base case parameter values, the estimated out-of-pocket cost per approved NCE is $114 million (1987 dollars). Capitalizing out-of-pocket costs to the point of marketing approval at a 9% discount rate yielded an average cost estimate of $231 million (1987 dollars).

Cost-effectiveness of electricity energy efficiency programs

We analyze the cost-effectiveness of electric utility ratepayer-funded programs to promote demand-side management (DSM) and energy efficiency (EE) investments. We specify a model that relates electricity demand to previous EE DSM spending, energy prices, income, weather, and other demand factors. In contrast to previous studies, we allow EE DSM spending to have a potential longterm demand effect and explicitly address possible endogeneity in spending. We find that current period EE DSM expenditures reduce electricity demand and that this effect persists for a number of years. Our findings suggest that ratepayer funded DSM expenditures between 1992 and 2006 produced a central estimate of 0.9 percent savings in electricity consumption over that time period and a 1.8 percent savings over all years. These energy savings came at an expected average cost to utilities of roughly 5 cents per kWh saved when future savings are discounted at a 5 percent rate. Copyright © 2012 by the IAEE. All rights reserved.

Cost-effectiveness of ART restorations in elderly adults: a randomized clinical trial

BACKGROUND: As the world population ages, the requirement for cost-effective methods of treating chronic disease conditions increases. In terms of oral health, there is a rapidly increasing number of dentate elderly with a high burden of maintenance. Population surveys indicate that older individuals are keeping their teeth for longer and are a higher caries risk group. Atraumatic Restorative Treatment (ART) could be suitable for patients in nursing homes or house-bound elderly, but very little research has been done on its use in adults.

OBJECTIVES: To compare the cost-effectiveness of ART and a conventional technique (CT) for restoring carious lesions as part of a preventive and restorative programme for older adults.

METHODS: In this randomized clinical trial, 82 patients with carious lesions were randomly allocated to receive either ART or conventional restorations. Treatment costs were measured based on treatment time, materials and labour. For the ART group, the cost of care provided by a dentist was also compared to the cost of having a hygienist to provide treatment. Effectiveness was measured using percentage of restorations that survived after a year.

RESULTS: Eighty-two patients received 260 restorations, that is, 128 ART and 132 conventional restorations. 91.1% of the restorations were on one surface only. After a year, 252 restorations were assessed in 80 patients. The average cost for ART and conventional restorations was €16.86 and €28.71 respectively; the restoration survival percentages were 91.1% and 97.7%, respectively. This resulted in a cost-effectiveness ratio of 0.18 (ART) and 0.29 (CT). When the cost of a hygienist to provide ART was inserted in the analysis, the resulting ratio was 0.14.

CONCLUSIONS: Atraumatic restorative treatment was found to be a more cost-effective alternative to treat older adults after 1 year, compared to conventional restorations, especially in out of surgery facilities and using alternative workforce such as hygienists. Atraumatic restorative treatment can be a useful tool to provide dental care for frail and fearful individuals who might not access dental treatment routinely.

A preliminary report of the cost-effectiveness of tooth replacement strategies for partially dentate elders

OBJECTIVE: To compare the cost-effectiveness of conventional treatment using partial dentures with functionally orientated treatment to replace missing teeth for partially dentate elders using a randomised controlled clinical trial.

BACKGROUND: In many countries, including the Republic of Ireland, the only publically funded treatment option offered to partially dentate older patients is a removable partial denture. However, evidence suggests that these removable prostheses are unpopular with patients and can potentially increase the risk of further dental disease and subsequent tooth loss.

MATERIALS AND METHODS: Fourty-four partially dentate patients aged 65 years and older were recruited. Patients were randomly assigned to the two treatment arms of the study. The conventional treatment group received removable partial dentures to replace all missing natural teeth. The functionally orientated group was restored to a Shortened Dental Arch (SDA) of 10 occluding contacts using resin-bonded bridgework (RBB). The costs associated with each treatment were recorded. Effectiveness was measured in terms of the impact on oral health-related quality of life (OHRQoL) using OHIP-14.

RESULTS: Both groups reported improvements in OHRQoL 1 month after completion of treatment. The conventional treatment group required 8.3 clinic visits as compared to 4.4 visits for the functionally orientated group. The mean total treatment time was 183 min 19 s for the conventional group vs. 124 min 8 s for the functionally orientated group. The average cost of treatment for the conventional group was 487.74 Euros compared to 356.20 Euros for the functional group.

CONCLUSIONS: Functionally orientated treatment was more cost-effective than conventional treatment in terms of treatment effect and opportunity costs to the patients' time.

Tiered Pricing and Cost Functions: Are Equity, Cost Recovery and Economic Efficiency Compatible Goals?

Suppliers of water and energy are frequently natural monopolies, with their pricing regulated by governmental agencies. Pricing schemes are evaluated by the efficiency of the resource allocation they lead to, the capacity of the utilities to capture their costs and the distributional effects of the policies, in particular, impacts on the poor. One pricing approach has been average cost pricing, which guarantees cost recovery and allows utilities to provide their product at relatively low prices. However, average cost pricing leads to economically inefficient consumption levels, when sources of water and energy are limited and increasing the supply is costly. An alternative approach is increasing block rates (hereafter, IBR or tiered pricing), where individuals pay a low rate for an initial consumption block and a higher rate as they increase use beyond that block. An example of IBR is shown in Figure 1 (on next page), which shows a rate structure for residential water use. With the rates in Figure 1, a household would be charged $0.46 and $0.71 per hundred gallons for consumption below and above 21,000 gallons per month, respectively.

Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

Cost and energy density of diet in Brazil, 2008-2009

This study aimed to evaluate the relationship between the cost and energy density of diet consumed in Brazilian households. Data from the Brazilian Household Budget Survey (POF 200812009) were used to identify the main foods and their prices. Similar items were grouped, resulting in a basket of 67 products. Linear programming was applied for the composition of isoenergetic baskets, minimizing the deviation from the average household diet. Restrictions were imposed on the inclusion of items and the energy contribution of the various food groups. A reduction in average cost of diet was applied at intervals of R$0.15 to the lowest possible cost. We identified an inverse association between energy density and cost of diet (p < 0.05), and at the lowest possible cost we obtained the maximum value of energy density Restrictions on the diet's cost resulted in the selection of diets with higher energy density, indicating that cost of diet may lead to the adoption of inadequate diets in Brazil.

Are Indian Firms too Small? A Nonparametric Analysis of Cost Efficiency and Industry Structure of Indian Manufacturing

In this paper we use the 2004-05 Annual Survey of Industries data to estimate the levels of cost efficiency of Indian manufacturing firms in the various states and also get state level measures of industrial organization (IO) efficiency. The empirical results show the presence of considerable cost inefficiency in a majority of the states. Further, we also find that, on average, Indian firms are too small. Consolidating them to attain the optimal scale would further enhance efficiency and lower average cost.

Utilising survey data to inform public policy: Comparison of the cost-effectiveness of treatment of ten mental disorders

Background Mental health survey data are now being used proactively to decide how the burden of disease might best be reduced. Aims To study the cost-effectiveness of current and optimal treatments for mental disorders and the proportion of burden avertable by each. Method Data for three affective, four anxiety and two alcohol use disorders and for schizophrenia were compared in terms of cost, burden averted and efficiency of current and optimal treatment. We then calculated the burden unavertable given current knowledge. The unit of health gain was a reduction in the years lived with disability (YLDs). Results Summing across all disorders, current treatment averted 13% of the burden, at an average cost of AUS$30 000 per YLD gained. Optimal treatment at current coverage could avert 20% of the burden, at an average cost of AUS$18 000 per YLD gained. Optimal treatment at optimal coverage could avert 28% of the burden, at AUS$16 000 per YLD gained. Sixty per cent of the burden of mental disorders was deemed to be unavertable. Conclusions The efficiency of treatment varied more than tenfold across disorders. Although coverage of some of the more efficient treatments should be extended, other factors justify continued use of less-efficient treatments for some disorders. Declaration of interest None. Funding detailed in Acknowledgements.