Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

A Raman spectroscopic study of the different vanadate-groups in solid-state compounds, model case : mineral phases vésigniéite, BaCu3(VO4)2(OH)2, and volborthite, Cu3V2O7(OH)2•2H2O

Raman spectroscopy has been used to study vanadates in the solid state. The molecular structure of the vanadate minerals vésigniéite [BaCu3(VO4)2(OH)2] and volborthite [Cu3V2O7(OH)2·2H2O] have been studied by Raman spectroscopy and infrared spectroscopy. The spectra are related to the structure of the two minerals. The Raman spectrum of vésigniéite is characterized by two intense bands at 821 and 856 cm−1 assigned to ν1 (VO4)3− symmetric stretching modes. A series of infrared bands at 755, 787 and 899 cm−1 are assigned to the ν3 (VO4)3− antisymmetric stretching vibrational mode. Raman bands at 307 and 332 cm−1 and at 466 and 511 cm−1 are assigned to the ν2 and ν4 (VO4)3− bending modes. The Raman spectrum of volborthite is characterized by the strong band at 888 cm−1, assigned to the ν1 (VO3) symmetric stretching vibrations. Raman bands at 858 and 749 cm−1 are assigned to the ν3 (VO3) antisymmetric stretching vibrations; those at 814 cm−1 to the ν3 (VOV) antisymmetric vibrations; that at 508 cm−1 to the ν1 (VOV) symmetric stretching vibration and those at 442 and 476 cm−1 and 347 and 308 cm−1 to the ν4 (VO3) and ν2 (VO3) bending vibrations, respectively. The spectra of vésigniéite and volborthite are similar, especially in the region of skeletal vibrations, even though their crystal structures differ.

Surveying the state of higher education in energy efficiency, in Australian engineering curriculum

This paper presents the method and results of a survey of 27 of the 33 Australian universities teaching engineering education in late 2007, undertaken by The Natural Edge Project (hosted by Griffith University and the Australian National University) and supported by the National Framework for Energy Efficiency. This survey aimed to ascertain the extent of energy efficiency (EE) education, and to identify preferred methods to assist in increasing the extent to which EE education is embedded in engineering curriculum. In this paper the context for the survey is supported by a summary of the key results from a variety of surveys undertaken over the last decade internationally. The paper concludes that EE education across universities and engineering disciplines in Australia is currently highly variable and ad hoc. Based on the results of the survey, this paper highlights a number of preferred options to support educators to embed sustainability within engineering programs, and future opportunities for monitoring EE, within the context of engineering education for sustainable development (EESD).

Nursing Sans Frontieres: A Three Year Case Study of Multi-State Registration of Support Nursing Practice Using Information Technology

Objective: To highlight the registration issues for nurses who wish to practice nationally, particularly those practicing within the telehealth sector. Design: As part of a national clinical research study, applications were made to every state and territory for mutual recognition of nursing registration and fee waiver for telenursing cross boarder practice for a period of three years. These processes are described using a case study approach. Outcome: The aim of this case study was to achieve registration in every state and territory of Australia without paying multiple fees by using mutual recognition provisions and the cross-border fee waiver policy of the nurse regulatory authorities in order to practice telenursing. Results: Mutual recognition and fee waiver for cross-border practice was granted unconditionally in two states: Victoria (Vic) and Tasmania (Tas), and one territory: the Northern Territory (NT). The remainder of the Australian states and territories would only grant temporary registration for the period of the project or not at all, due to policy restrictions or nurse regulatory authority (NRA) Board decisions. As a consequence of gaining fee waiver the annual cost of registration was a maximum of $145 per annum as opposed to the potential $959 for initial registration and $625 for annual renewal. Conclusions: Having eight individual nurses Acts and NRAs for a population of 265,000 nurses would clearly indicate a case for over regulation in this country. The structure of regulation of nursing in Australia is a barrier to the changing and evolving role of nurses in the 21st century and a significant factor when considering workforce planning.