Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

PTZ camera pose estimation by tracking a 3D target

We present a technique for estimating the 6DOF pose of a PTZ camera by tracking a single moving target in the image with known 3D position. This is useful in situations where it is not practical to measure the camera pose directly. Our application domain is estimating the pose of a PTZ camerso so that it can be used for automated GPS-based tracking and filming of UAV flight trials. We present results which show the technique is able to localize a PTZ after a short vision-tracked flight, and that the estimated pose is sufficiently accurate for the PTZ to then actively track a UAV based on GPS position data.

The WIL (Work Integrated Learning) report : a national scoping study [Final Report]

This report provides an account of the first large-scale scoping study of work integrated learning (WIL) in contemporary Australian higher education. The explicit aim of the project was to identify issues and map a broad and growing picture of WIL across Australia and to identify ways of improving the student learning experience in relation to WIL. The project was undertaken in response to high levels of interest in WIL, which is seen by universities both as a valid pedagogy and as a means to respond to demands by employers for work-ready graduates, and demands by students for employable knowledge and skills. Over a period of eight months of rapid data collection, 35 universities and almost 600 participants contributed to the project. Participants consistently reported the positive benefits of WIL and provided evidence of commitment and innovative practice in relation to enhancing student learning experiences. Participants provided evidence of strong partnerships between stakeholders and highlighted the importance of these relationships in facilitating effective learning outcomes for students. They also identified a range of issues and challenges that face the sector in growing WIL opportunities; these issues and challenges will shape the quality of WIL experiences. While the majority of comments focused on issues involved in ensuring quality placements, it was recognised that placements are just one way to ensure the integration of work with learning. Also, the WIL experience is highly contextualised and impacted by the expectations of students, employers, the professions, the university and government policy.

School-based drug prevention programs: a review of what works

This article examines the effectiveness of school-based drug prevention programs in preventing illicit drug use. Our article reports the results of a systematic review of the evaluation literature to answer three fundamental questions: (1) do school-based drug prevention programs reduce rates of illicit drug use? (2) what features are characteristic of effective programs? and (3) do these effective program characteristics differ from those identified as effective in reviews of school-based drug prevention of licit substance use (such as alcohol and tobacco)? Using systematic review and meta-analytic techniques, we identify the characteristics of schoolbased drug prevention programs that have a significant and beneficial impact on ameliorating illicit substance use (i.e., narcotics) among young people. Successful intervention programs typically involve high levels of interactivity, time-intensity, and universal approaches that are delivered in the middle school years. These program characteristics aligned with many of the effective program elements found in previous reviews exploring the impact of school-based drug prevention on licit drug use. Contrary to these past reviews, however, our analysis suggests that the inclusion of booster sessions and multifaceted drug prevention programs have little impact on preventing illicit drug use among school-aged children. Limitations of the current review and policy implications are discussed.

Domains of rural social work practice

Even though there is substantial agreement about the nature of rural contexts, practice principles, and factors influencing practice we still do not have a framework for organising this knowledge in a way that can directly inform the practitioner in their day-to-day work. In this paper, we introduce the concepts 'practice domains', 'domain location', and 'domain alignment' that, taken together, provide such a framework. We suggest that each practitioner works within a number of practice domains. A domain is a discourse about practice comprising narratives about how a social worker should practise and which factors they should take most account of in their practice decision making. Each practitioner, and each practice process, can be located somewhere within each domain (domain location) and also situated amongst domains according to their relative alignment with each of them (domain alignment). In this paper, we present this framework and show how it is useful for practitioners in understanding practice, identifying factors influencing it, and making practice decisions in immediate, concrete situations.

The symbiosis of improvised and rehearsed elements in the creation of contemporary jazz ensemble music

This research investigates the symbiotic relationship between composition and improvisation and the notion of improvisation itself. With a specific interest in developing, extending and experimenting with the relationship of improvisation within predetermined structures, the creative work component of this research involved composing six new works with varying approaches for The Andrea Keller Quartet and guest improvisers, for performance on a National Australian tour. This is documented in the CD recording Galumphing Round the Nation - Collaborations Tour 2009. The exegesis component is intended to run alongside the creative work and discusses the central issues surrounding improvisation in an ensemble context and the subject of composing for improvisers. Specifically, it questions the notion that when music emphasises a higher ratio of spontaneous to pre-determined elements, and is exposed to the many variables of a performance context, particularly through its incorporation of visitant improvisers, the resultant music should potentially be measurably altered with each performance. This practice-led research demonstrates the effect of concepts such as individuality, variability within context, and the interactive qualities of contemporary jazz ensemble music. Through the analysis and comparison of the treatment of the six pieces over thirteen performances with varying personnel, this exegesis proposes that, despite the expected potential for spontaneity in contemporary jazz music, the presence of established patterns, the desire for familiarity and the intuitive tendency towards accepted protocols ensure that the music which emerges is not as mutable as initially anticipated.

Margin-adaptive model selection in statistical learning

A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one. Requiring this “strong margin adaptivity” makes the model selection problem more challenging. We first prove, in a general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.

What it means to become a public relations professional: student perceptions of professional identity through real-world learning

Public relations educators need new solutions to prepare students to become tomorrow's practitioner today. Managers and employers in the new creative workforce (McWilliam, 2008) expect graduates to be problem solvers, critical and creative thinkers, reflective, and self reliant (Barrie, 2008; David, 2004). Enabling students to develop these attributes requires a collaborative and creative approach to pedagogy (Jeffrey & Craft, 2001, 2004). A model for the next generation of public relations education was developed to integrate industry partnerships as a way to bridge pedagogy and professional practice. The model suggests (a) that industry partnerships be embedded in learning activities, (b) that assessment items be considered on a continuum and delivered incrementally across a course of study, and (c) that connections between classroom and workplace activities are clearly signposted for students.

The development of a new signal processing program at the Queensland University of Technology

The School of Electrical and Electronic Systems Engineering at Queensland University of Technology, Brisbane, Australia (QUT), offers three bachelor degree courses in electrical and computer engineering. In all its courses there is a strong emphasis on signal processing. A newly established Signal Processing Research Centre (SPRC) has played an important role in the development of the signal processing units in these courses. This paper describes the unique design of the undergraduate program in signal processing at QUT, the laboratories developed to support it, and the criteria that influenced the design.

On the computation and interpretation of auto- and cross-trispectra

This paper discusses the principal domains of auto- and cross-trispectra. It is shown that the cumulant and moment based trispectra are identical except on certain planes in trifrequency space. If these planes are avoided, their principal domains can be derived by considering the regions of symmetry of the fourth order spectral moment. The fourth order averaged periodogram will then serve as an estimate for both cumulant and moment trispectra. Statistics of estimates of normalised trispectra or tricoherence are also discussed.