Validating adult and paediatric nutrition screening and assessment tools in Vietnamese language in two acute hospital in Ho Chi Minh City, Vietnam

Despite being commonly prevalent in acute care hospitals worldwide, malnutrition often goes unidentified and untreated due to a lack in the implementation of a nutrition care pathway. The aim of this study was to validate nutrition screening and assessment tools in Vietnamese language. After converting into Vietnamese, Malnutrition Screening Tool (MST) and Subjective Global Assessment (SGA) were used to identify malnutrition in the adult setting; and the Paediatric Nutrition Screening Tool (PNST) and paediatric Subjective Global Nutritional Assessment (SGNA) were used in the paediatric setting in two acute care hospitals in Vietnam. This cross-sectional observational study sampled 123 adults (median age 78 years [39–96 years], 63% males) and 105 children (median age 20 months [2–100 months], 66% males). In adults, nutrition risk and malnutrition were identified in 29% and 45% of the cohort respectively. Nutrition risk and malnutrition were identified in 71% and 43% of the paediatric cohort respectively. The sensitivity and specificity of the screening tools were: 62% and 99% for the MST compared to the SGA; 89% and 42% for the PNST compared to the SGNA. This study provides a stepping stone to the potential use of evidence-based nutrition screening and assessment tools in Vietnamese language within the adult and paediatric Vietnamese acute care setting. Further work is required into integrating a complete nutrition care pathway within the acute care setting in Vietnamese hospitals.

Change to costs and lengths of stay in the emergency department and the Brisbane protocol: An observational study

- Objective To compare health service cost and length of stay between a traditional and an accelerated diagnostic approach to assess acute coronary syndromes (ACS) among patients who presented to the emergency department (ED) of a large tertiary hospital in Australia. - Design, setting and participants This historically controlled study analysed data collected from two independent patient cohorts presenting to the ED with potential ACS. The first cohort of 938 patients was recruited in 2008–2010, and these patients were assessed using the traditional diagnostic approach detailed in the national guideline. The second cohort of 921 patients was recruited in 2011–2013 and was assessed with the accelerated diagnostic approach named the Brisbane protocol. The Brisbane protocol applied early serial troponin testing for patients at 0 and 2 h after presentation to ED, in comparison with 0 and 6 h testing in traditional assessment process. The Brisbane protocol also defined a low-risk group of patients in whom no objective testing was performed. A decision tree model was used to compare the expected cost and length of stay in hospital between two approaches. Probabilistic sensitivity analysis was used to account for model uncertainty. - Results Compared with the traditional diagnostic approach, the Brisbane protocol was associated with reduced expected cost of $1229 (95% CI −$1266 to $5122) and reduced expected length of stay of 26 h (95% CI −14 to 136 h). The Brisbane protocol allowed physicians to discharge a higher proportion of low-risk and intermediate-risk patients from ED within 4 h (72% vs 51%). Results from sensitivity analysis suggested the Brisbane protocol had a high chance of being cost-saving and time-saving. - Conclusions This study provides some evidence of cost savings from a decision to adopt the Brisbane protocol. Benefits would arise for the hospital and for patients and their families.

Complimentary collaborations: Teachers and researchers co-developing best practices in art education

Australia is currently experiencing a huge cultural shift as it moves from a State-based curriculum, to a national education system. The Australian State-based bodies that currently manage teacher registration, teacher education course accreditation, curriculum frameworks and syllabi are often complex organisations that hold conflicting ideologies about education and teaching. The development of a centralised system, complete with a single accreditation body and a national curriculum can be seen as a reaction to this complexity. At the time of writing, the Australian Curriculum is being rolled out in staggered phases across the states and territories of Australia. Phase one has been implemented, introducing English, Mathematics, History and Science. Subsequent phases (Humanities and Social Sciences, the Arts, Technologies, Health and Physical Education, Languages, and year 9-10 work studies) are intended to follow. Forcing an educational shift of this magnitude is no simple task; not least because the States and Territories have and continue to demonstrate varying levels of resistance to winding down their own curricula in favour of new content with its unfamiliar expectations and organisations. The full implementation process is currently far from over, and far from being fully resolved. The Federal Government has initiated a number of strategies to progress the implementation, such as the development of the Australian Institute for Teaching and School Leadership (AITSL) to aid professional educators to implement the new curriculum. AITSL worked with professional and peak specialist bodies to develop Illustrations of Practice (hereafter IoP) for teachers to access and utilise. This paper tells of the building of one IoP, where a graduate teacher and a university lecturer collaborated to construct ideas and strategies to deliver visual arts lessons to early childhood students in a low Socio- Economic Status [SES] regional setting and discusses the experience in terms of its potential for professional learning in art education.

Species-based and community-level approaches to conservation prioritization

One major reason for the global decline of biodiversity is habitat loss and fragmentation. Conservation areas can be designed to reduce biodiversity loss, but as resources are limited, conservation efforts need to be prioritized in order to achieve best possible outcomes. The field of systematic conservation planning developed as a response to opportunistic approaches to conservation that often resulted in biased representation of biological diversity. The last two decades have seen the development of increasingly sophisticated methods that account for information about biodiversity conservation goals (benefits), economical considerations (costs) and socio-political constraints. In this thesis I focus on two general topics related to systematic conservation planning. First, I address two aspects of the question about how biodiversity features should be valued. (i) I investigate the extremely important but often neglected issue of differential prioritization of species for conservation. Species prioritization can be based on various criteria, and is always goal-dependent, but can also be implemented in a scientifically more rigorous way than what is the usual practice. (ii) I introduce a novel framework for conservation prioritization, which is based on continuous benefit functions that convert increasing levels of biodiversity feature representation to increasing conservation value using the principle that more is better. Traditional target-based systematic conservation planning is a special case of this approach, in which a step function is used for the benefit function. We have further expanded the benefit function framework for area prioritization to address issues such as protected area size and habitat vulnerability. In the second part of the thesis I address the application of community level modelling strategies to conservation prioritization. One of the most serious issues in systematic conservation planning currently is not the deficiency of methodology for selection and design, but simply the lack of data. Community level modelling offers a surrogate strategy that makes conservation planning more feasible in data poor regions. We have reviewed the available community-level approaches to conservation planning. These range from simplistic classification techniques to sophisticated modelling and selection strategies. We have also developed a general and novel community level approach to conservation prioritization that significantly improves on methods that were available before. This thesis introduces further degrees of realism into conservation planning methodology. The benefit function -based conservation prioritization framework largely circumvents the problematic phase of target setting, and allowing for trade-offs between species representation provides a more flexible and hopefully more attractive approach to conservation practitioners. The community-level approach seems highly promising and should prove valuable for conservation planning especially in data poor regions. Future work should focus on integrating prioritization methods to deal with multiple aspects in combination influencing the prioritization process, and further testing and refining the community level strategies using real, large datasets.

Expression of glucose transporter isoforms and the insulin receptor during hamster preimplantation embryo development

During preimplantation development, embryos of many species are known to express up to five isoforms of the facilitative glucose transporter proteins (GLUT). Development of hamster blastocysts is inhibited by glucose. We therefore investigated GLUT isoform and insulin receptor (IR) expression in hamster preimplantation embryos cultured in glucose-free medium from the 8-cell stage onwards. We show that GLUT1, 3 and 8 mRNA are constitutively expressed from the 8-cell to the blastocyst stage. The IR is expressed from the morula stage onwards. Messenger RNA of the insulin-responsive GLUT4 was not detected at any stage. GLUT1 and 3 were localised by immunocytochemistry. GLUT1 was expressed in both embryoblast and trophoblast, in the latter, mainly in basal and lateral membranes directed towards the blastocoel. and embryoblast. GLUT3 was exclusively localised in the apical. membrane of trophoblast cells. We show that hamster preimplantation embryos express several GLUT isoforms thus closely resembling embryos of other mammalian species. Despite endogenous IR expression, the insulin-sensitive isoform GLUT4 was not expressed, indicating that the insulin-mediated glucose uptake known from classical insulin target cells may not be relevant for hamster blastocysts.

The research capacity and culture of Australian podiatrists

Background Best practice clinical health care is widely recognised to be founded on evidence based practice. Enhancing evidence based practice via the rapid translation of new evidence into every day clinical practice is fundamental to the success of health care and in turn health care professions. There is little known about the collective research capacity and culture of the podiatry profession across Australia. Thus, the aim of this study was to investigate the research capacity and culture of the podiatry profession within Australia and determine if there were any differences between podiatrists working in different health sectors and workplaces. Method All registered podiatrists were eligible to participate in a cross-sectional online survey. The Australian Podiatry Associations disseminated the survey and all podiatrists were encouraged to distribute it to colleagues. The Research Capacity and Culture (RCC) tool was used to collect all research capacity and culture item variables using a 10-point scale (1 = lowest; 10 = highest). Additional demographic, workplace and health sector data variables were also collected. Mann–Whitney-U, Kruskal–Wallis and logistic regression analyses were used to determine any difference between health sectors and workplaces. Word cloud analysis was used for qualitative responses of individual motivators and barriers to research culture. Results There were 232 fully completed surveys (6% of Australian registered podiatrists). Overall respondents reported low success or skills (Median rating < 4) on the majority of individual success or skill items. Podiatrists working in multi-practitioner workplaces reported higher individual success or skills in the majority of items compared with sole practitioners (p < 0.05). Non-clinical and public health sector podiatrists reported significantly higher post-graduate study enrolment or completion, research activity participation, provisions to undertake research and individual success or skill than those working privately. Conclusions This study suggests that podiatrists in Australia report similar low levels of research success or skill to those reported in other allied health professions. The workplace setting and health sector seem to play key roles in self reported research success and skills. This is important knowledge for podiatrists and researchers aiming to translate research evidence into clinical practice.

Auger-emitter radiochemotherapy in squamous cell cancers : in vitro and in vivo experiments with In-111-BLMC

Head and neck squamous cell cancer (HNSCC) is the sixth most common cancer worldwide. Despite advances in combined modality therapy (surgery, radiotherapy, chemotherapy) the 5-year survival rate in stage III and IV disease remains at 40% - 60%. Short-range Auger-electron emitters, such as In-111 and In-114m, tagged with a drug, molecule, peptide, protein or nanoparticles brought in close proximity to nuclear DNA represent a fascinating alternative for treating cancer. In this thesis, we studied the usefulness of Indium-111-bleomycin complex (In-111-BLMC) in the diagnostics and potential therapy of HNSCC using in vitro HNSCC cell lines, in vivo nude mice, and in vivo HNSCC patients. In in vitro experiments with HNSCC cell lines, the sensitivity to external beam radiation, BLM, In-111-BLMC, and In-111-Cl3 was studied using the 96-well plate clonogenic assay. The influence of BLM and In-111-BLMC on the cell cycle was measured with flow cytometry. In in vivo nude mice xenograft studies, the activity ratios of In-111-BLMC were obtained in gamma camera images. The effect of In-111-BLMC in HNSCC xenografts was studied. In in vivo patient studies, we determined the tumor uptake of In-111-BLMC with gamma camera and the radioactivity from tumor samples using In-111-BLMC with specific activity of 75, 175, or 375 MBq/mg BLM. The S values, i.e. absorbed dose in a target organ per cumulated activity in a source organ, were simulated for In-111 and In-114m. In vitro studies showed the variation of sensitivity for external beam radiation, BLM, and In-111-BLMC between HNSCC cell lines. IC50 values for BLM were 1.6-, 1.8-, and 2.1-fold higher than In-111-BLMC (40 MBq/mg BLM) in three HNSCC cell lines. Specific In-111 activity of 40 MBq/mgBLM was more effective in killing cells than specific In-111 activity of 195MBq/mgBLM (p=0.0023). In-111-Cl3 alone had no killing effect. The percentage of cells in the G2/M phase increased after exposure to BLM and especially to In-111-BLMC in the three cell lines studied, indicating a G2/M block. The tumor-seeking behavior was shown in the in vivo imaging study of xenografted mice. BLM and In-111-BLMC were more effective than NaCl in reducing xenografted tumor size in HNSCC. The uptake ratios received from gamma images in the in vivo patient study varied from 1.2 to 2.8 in malignant tumors. However, the uptake of In-111-BLMC was unaffected by increasing the injected activity. A positive correlation existed between In-111-BLMC uptake, Ki-67/MIB activity, and number of mitoses. Regarding the S values, In-114m delivered a 4-fold absorbed radiation dose into the tumor compared with In-111, and thus, In-114m-BLMC might be more effective than In-111-BLMC at the DNA level. Auger-electron emitters, such as In-111 and In-114m, might have potential in the treatment of HNSCC. Further studies are needed to develop a radiopharmaceutical agent with appropriate physical properties of the radionuclide and a suitable carrier to bring it to the targeted tissue.

2nd ESMO Consensus Conference in Lung Cancer: Locally advanced stage III non-small-cell lung cancer

To complement the existing treatment guidelines for all tumour types, ESMO organises consensus conferences to focus on specific issues in each type of tumour. The 2nd ESMO Consensus Conference on Lung Cancer was held on 11–12 May 2013 in Lugano. A total of 35 experts met to address several questions on non-small-cell lung cancer (NSCLC) in each of four areas: pathology and molecular biomarkers, first-line/second and further lines of treatment in advanced disease, early-stage disease and locally advanced disease. For each question, recommendations were made including reference to the grade of recommendation and level of evidence. This consensus paper focuses on locally advanced disease.

The effects of stall surfaces and milk yield on the lying behavior of dairy cow

The importance of lying behavior to dairy cows and the feasible definition of lying has attracted many studies on the subject. Cattle show both behavioral and physiological stress responses when subjected to thwarting of their lying behavior. If cows are unable to lie down they later compensate for lost lying time when possible. Environmental factors such as housing and bedding systems have been noted to affect the time spent lying, but there is usually large variation in lying time between individuals. Internal factors such as the reproductive stage, age and health of cows affect their lying time and can cause variation. However, the effect of higher milk production on behavior has not previously been illuminated. The objective of this study was to provide data applicable for the improvement of resting conditions of cows. The preference of stall surface material, differences in normal behavior per unit time and various health measures were observed. The aim was to evaluate lying behavior and cow comfort on different stall bedding materials. In addition, the effect of milk yield on behavior was examined in a tie stall experiment. The preferences for surface materials were investigated in 5 experiments using 3 surface materials with bedding manipulations. According to the results, the cows preferred abundant straw bedding and soft rubber mats. However, they showed an aversion to sand bedding. Some individuals even refused to use stalls with sand when no organic bedding material was present. However, this study was unable to determine the reason for the avoidance, as neither the sand particle size nor thermal properties appeared critical. However, previous exposure to particular surface materials increased the preference for them. The amount of straw bedding was found to be an important factor affecting the preferences for stalls, and the lying time in stalls increased when the flooring softness was improved by applying straw or by installing elastic mats. Despite sand being the least preferred flooring material in preference tests, the health of legs improved during exposure to sand-floored stalls. Moreover cows using sand were cleaner than those that used straw stalls. Thus, sand bedding entailed some health benefits despite the contradictory results of preference tests, which more strongly reflected the perceptions of individual animals. Milk yield was observed to affect behavior by reducing the lying time, possibly due to factors other than longer duration of eating. High yielding cows seemed to intensify their lying bouts, as they were observed to lie with the neck muscles relaxed sooner after lying down than lower yielding cows. In conclusion, cows were found to prefer softer stall surface materials and organic bedding material. In addition, the lying time was reduced by a high milk yield, although the lying time seemed to be important for resting. Cows might differ in the needs for their lying environment. The management of dairy cows should eliminate any unnecessary prevention of lying, as even in tie-stalls high yielding cows seem to be affected by time constraints. Adding fresh bedding material to stalls increases the comfort of any stall flooring material.

Mechanical and Thermal Properties of Eva Blended with Biodegradable Ethyl Cellulose

In this study, biodegradable blend of Poly (Ethylene-co-Vinyl Acetate) (EVA) and Ethyl Cellulose (EC) were prepared. Ethylene vinyl alcohol (EVOH) copolymer was used as an interfacial compatibilizer to enhance adhesion between EVA and EC. The melt blended compatibilized biocomposites were examined for mechanical and thermal properties as per the ASTM standards. It has been found that the EC has a reinforcing effect on EVA leading to enhanced tensile strength and also impart biodegradability. Thus, a high loading of 50% EC could be added without compromising Much on the mechanical properties. Analysis of the tensile data using predictive theories showed an enhanced interaction of the dispersed phase (EC) and the matrix (EVA). The compatibilizing effects of EVOH on these blends were confirmed by the significant improvement in the mechanical properties comparable with neat EVA as also observed by SEM microscopy. The TGA thermograms exhibits two-stage degradation and as EC content increases, the onset temperature for thermal degradation reduces. (C) 2009 Wiley Periodicals, Inc. J Appl Polym Sci 116: 1044-1056, 2010

Angel Investing in Finland: An Analysis Based on Agency Theory and the Incomplete Contracting Theory

Wealthy individuals - business angels who invest a share of their net worth in entrepreneurial ventures - form an essential part of an informal venture capital market that can secure funding for entrepreneurial ventures. In Finland, business angels represent an untapped pool of capital that can contribute to fostering entrepreneurial development. In addition, business angels can bridge knowledge gaps in new business ventures by means of making their human capital available. This study has two objectives. The first is to gain an understanding of the characteristics and investment behaviour of Finnish business angels. The strongest focus here is on the due diligence procedures and their involvement post investment. The second objective is to assess whether agency theory and the incomplete contacting theory are useful theoretical lenses in the arena of business angels. To achieve the second objective, this study investigates i) how risk is mitigated in the investment process, ii) how uncertainty influences the comprehensiveness of due diligence as well as iii) how control is allocated post investment. Research hypotheses are derived from assumptions underlying agency theory and the incomplete contacting theory. The data for this study comprise interviews with 53 business angels. In terms of sample size this is the largest on Finnish business angels. The research hypotheses in this study are tested using regression analysis. This study suggests that the Finnish informal venture capital market appears to be comprised of a limited number of business angels whose style of investing much resembles their formal counterparts’. Much focus is placed on managing risks prior to making the investment by strong selectiveness and by a relatively comprehensive due diligence. The involvement is rarely on a day-to-day basis and many business angels seem to see board membership as a more suitable alternative than involvement in the operations of an entrepreneurial venture. The uncertainty involved does not seem to drive an increase in due diligence. On the contrary, it would appear that due diligence is more rigorous in safer later stage investments and when the business angels have considerable previous experience as investors. Finnish business angels’ involvement post investment is best explained by their degree of ownership in the entrepreneurial venture. It seems that when investors feel they are sufficiently rewarded, in terms of an adequate equity stake, they are willing to involve themselves actively in their investments. The lack of support for a relationship between increased uncertainty and the comprehensiveness of due diligence may partly be explained by an increasing trend towards portfolio diversification. This is triggered by a taxation system that favours investments through investment companies rather than direct investments. Many business angels appear to have substituted a specialization strategy that builds on reducing uncertainty for a diversification strategy that builds on reducing firm specific (idiosyncratic) risk by holding shares in ventures whose returns are not expected to exhibit a strong positive correlation.

Associations of interleukin-1 (IL-1) gene variation and IL-1 receptor antagonist phenotypes with immunological responses, metabolic dysregulation and type 2 diabetes

The upstream proinflammatory interleukin-1 (IL-1) cytokines, together with a naturally occurring IL-1 receptor antagonist (IL-1Ra), play a significant role in several diseases and physiologic conditions. The IL-1 proteins affect glucose homeostasis at multiple levels contributing to vascular injuries and metabolic dysregulations that precede diabetes. An association between IL-1 gene variations and IL-1Ra levels has been suggested, and genetic studies have reported associations with metabolic dysregulation and altered inflammatory responses. The principal aims of this study were to: 1) examine the associations of IL-1 gene variation and IL-1Ra expression in the development and persistence of thyroid antibodies in subacute thyroiditis; 2) investigate the associations of common variants in the IL-1 gene family with plasma glucose and insulin concentrations, glucose homeostasis measures and prevalent diabetes in a representative population sample; 3) investigate genetic and non-genetic determinants of IL-1Ra phenotypes in a cross-sectional setting in three independent study populations; 4) investigate in a prospective setting (a) whether variants of the IL-1 gene family are predictors for clinically incident diabetes in two population-based observational cohort studies; and (b) whether the IL-1Ra levels predict the progression of metabolic syndrome to overt diabetes during the median follow-up of 10.8 and 7.1 years. Results from on patients with subacte thyroiditis showed that the systemic IL-1Ra levels are elevated during a specific proinflammatory response and they correlated with C-reactive protein (CRP) levels. Genetic variation in the IL-1 family seemed to have an association with the appearance of thyroid peroxidase antibodies and persisting local autoimmune responses during the follow-up. Analysis of patients suffering from diabetes and metabolic traits suggested that genetic IL-1 variation and IL-1Ra play a role in glucose homeostasis and in the development of type 2 diabetes. The coding IL-1 beta SNP rs1143634 was associated with traits related to insulin resistance in cross-sectional analyses. Two haplotype variants of the IL-1 beta gene were associated with prevalent diabetes or incident diabetes in a prospective setting and both of these haplotypes were tagged by rs1143634. Three variants of the IL-1Ra gene and one of the IL-1 beta gene were consistently identified as significant, independent determinants of the IL-1Ra phenotype in two or three populations. The proportion of the phenotypic variation explained by the genetic factors was modest however, while obesity and other metabolic traits explained a larger part. Body mass index was the strongest predictor of systemic IL-1Ra concentration overall. Furthermore, the age-adjusted IL-1Ra concentrations were elevated in individuals with metabolic syndrome or diabetes when compared to those free of metabolic dysregulation. In prospective analyses the systemic IL-1Ra levels were found as independent predictors for the development of diabetes in people with metabolic syndrome even after adjustment for multiple other factors, including plasma glucose and CRP levels. The predictive power of IL-1Ra was better than that of CRP. The prospective results also provided some evidence for a role of common IL-1 alpha promoter SNP rs1800587 in the development of type 2 diabetes among men and suggested that the role may be gender specific. Likewise, common variations in the IL-1 beta coding region may have a gender specific association with diabetes development. Further research on the potential benefits of IL-1Ra measurements in identifying individuals at high risk for diabetes, who then could be targeted for specific treatment interventions, is warranted. It has been reported in the recent literature that IL-1Ra secreted from adipose tissue has beneficial effects on glucose homeostasis. Furthermore, treatment with recombinant human IL-1Ra has been shown to have a substantial therapeutic potential. The genetic results from the prospective analyses performed in this study remain inconclusive, but together with the cross-sectional analyses they suggest gender-specific effects of the IL-1 variants on the risk of diabetes. Larger studies with more extensive genotyping and resequencing may help to pinpoint the exact variants responsible and to further elucidate the biological mechanisms for the observed associations. This would improve our understanding of the pathways linking inflammation and obesity with glucose and insulin metabolism.

Voluntary Energy Harvesting Relays and Selection in Cooperative Wireless Networks

The use of energy harvesting (EH) nodes as cooperative relays is a promising and emerging solution in wireless systems such as wireless sensor networks. It harnesses the spatial diversity of a multi-relay network and addresses the vexing problem of a relay's batteries getting drained in forwarding information to the destination. We consider a cooperative system in which EH nodes volunteer to serve as amplify-and-forward relays whenever they have sufficient energy for transmission. For a general class of stationary and ergodic EH processes, we introduce the notion of energy constrained and energy unconstrained relays and analytically characterize the symbol error rate of the system. Further insight is gained by an asymptotic analysis that considers the cases where the signal-to-noise-ratio or the number of relays is large. Our analysis quantifies how the energy usage at an EH relay and, consequently, its availability for relaying, depends not only on the relay's energy harvesting process, but also on its transmit power setting and the other relays in the system. The optimal static transmit power setting at the EH relays is also determined. Altogether, our results demonstrate how a system that uses EH relays differs in significant ways from one that uses conventional cooperative relays.

Molecular cloning and expression pattern of a Cubitus interruptus homologue from the mulberry silkworm Bombyx mori

A homologue of the segment polarity gene Cubitus interruptus from Bombyx Mori, (BmCi) has been cloned and characterized. This region harbouring Zn2+ finger motif is highly conserved across species. In B. Mori, BmCi RNA expression was first detected at stage 6 of embryogenesis, which reached maximum levels at stage 21C and was maintained until larval hatching. The segmentally reiterated striped pattern of transcript distribution in stage 21C embryos was in conformity with its predicted segment polarity nature. BmCi was expressed in the fore- and hind-wing discs, ovaries, testes and gut during fifth larval intermolt, reminiscent of its expression domains in Drosophila. Besides, BmCi expression was seen in the. anterior part of the middle silkglands in late embryonic stages, and this pattern was maintained during larval development. The transition from third to fourth and fifth larval intermolts was accompanied by an increase in the transcript levels in the middle silkglands. Our results demonstrate the presence of a novel expression domain for Ci in Bombyx. (C) 2002 Elsevier Science Ireland Ltd. All rights reserved.

An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.