Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

Learning and teaching from the edge to centre stage : Critical factors in embedding sustainable university-wide engagement in external awards and grants funding initiatives.

Teaching awards, grants and fellowships are strategies used to recognise outstanding contributions to learning and teaching, encourage innovation, and to shift learning and teaching from the edge to centre stage. Examples range from school, faculty and institutional award and grant schemes to national schemes such as those offered by the Australian Learning and Teaching Council (ALTC), the Carnegie Foundation for the Advancement of Teaching in the United States, and the Fund for the Development of Teaching and Learning in higher education in the United Kingdom. The Queensland University of Technology (QUT) has experienced outstanding success in all areas of the ALTC funding since the inception of the Carrick Institute for Learning and Teaching in 2004. This paper reports on a study of the critical factors that have enabled sustainable and resilient institutional engagement with ALTC programs. As a lens for examining the QUT environment and practices, the study draws upon the five conditions of the framework for effective dissemination of innovation developed by Southwell, Gannaway, Orrell, Chalmers and Abraham (2005, 2010): 1. Effective, multi-level leadership and management 2. Climate of readiness for change 3. Availability of resources 4. Comprehensive systems in institutions and funding bodies 5. Funding design The discussion on the critical factors and practical and strategic lessons learnt for successful university-wide engagement offer insights for university leaders and staff who are responsible for learning and teaching award, grant and associated internal and external funding schemes.

Effect of external high pressures on the clay mineral sodium montmorillonite intercalated with methylated octadecylammonium bromide surfactants

Raman microprobe spectra of the clay mineral Wyoming SWy-2-sodium montmorillonite intercalated with the surfactants, methyltrioctadecylammonium bromide (TOMA) dimethyldiotadecylammonium bromide (DODMA) and octadecyl-trimethylammonium bromide (ODTMA), have been measured in the CH2 stretching region at external pressures up to ~40 kbar with the aid of a diamond-anvil cell. In the case of the intercalated clays containing TOMA and DODMA, the Raman data afford evidence for gauche to trans conformational changes in the orientation of the CH2 chains in the surfactants with increasing pressure. These conformational changes are reversed completely upon the release of pressure.

An analysis of the nature and effectiveness of corporate governance in smaller listed Australian companies

The objective of this thesis is to investigate the corporate governance attributes of smaller listed Australian firms. This study is motivated by evidence that these firms are associated with more regulatory concerns, the introduction of ASX Corporate Governance Recommendations in 2004, and a paucity of research to guide regulators and stakeholders of smaller firms. While there is an extensive body of literature examining the effectiveness of corporate governance, the literature principally focuses on larger companies, resulting in a deficiency in the understanding of the nature and effectiveness of corporate governance in smaller firms. Based on a review of agency theory literature, a theoretical model is developed that posits that agency costs are mitigated by internal governance mechanisms and transparency. The model includes external governance factors but in many smaller firms these factors are potentially absent, increasing the reliance on the internal governance mechanisms of the firm. Based on the model, the observed greater regulatory intervention in smaller companies may be due to sub-optimal internal governance practices. Accordingly, this study addresses four broad research questions (RQs). First, what is the extent and nature of the ASX Recommendations that have been adopted by smaller firms (RQ1)? Second, what firm characteristics explain differences in the recommendations adopted by smaller listed firms (RQ2), and third, what firm characteristics explain changes in the governance of smaller firms over time (RQ3)? Fourth, how effective are the corporate governance attributes of smaller firms (RQ4)? Six hypotheses are developed to address the RQs. The first two hypotheses explore the extent and nature of corporate governance, while the remaining hypotheses evaluate its effectiveness. A time-series, cross-sectional approach is used to evaluate the effectiveness of governance. Three models, based on individual governance attributes, an index of six items derived from the literature, and an index based on the full list of ASX Recommendations, are developed and tested using a sample of 298 smaller firms with annual observations over a five-year period (2002-2006) before and after the introduction of the ASX Recommendations in 2004. With respect to (RQ1) the results reveal that the overall adoption of the recommendations increased from 66 per cent in 2004 to 74 per cent in 2006. Interestingly, the adoption rate for recommendations regarding the structure of the board and formation of committees is significantly lower than the rates for other categories of recommendations. With respect to (RQ2) the results reveal that variations in rates of adoption are explained by key firm differences including, firm size, profitability, board size, audit quality, and ownership dispersion, while the results for (RQ3) were inconclusive. With respect to (RQ4), the results provide support for the association between better governance and superior accounting-based performance. In particular, the results highlight the importance of the independence of both the board and audit committee chairs, and of greater accounting-based expertise on the audit committee. In contrast, while there is little evidence that a majority independent board is associated with superior outcomes, there is evidence linking board independence with adverse audit opinion outcomes. These results suggest that board and chair independence are substitutes; in the presence of an independent chair a majority independent board may be an unnecessary and costly investment for smaller firms. The findings make several important contributions. First, the findings contribute to the literature by providing evidence on the extent, nature and effectiveness of governance in smaller firms. The findings also contribute to the policy debate regarding future development of Australia’s corporate governance code. The findings regarding board and chair independence, and audit committee characteristics, suggest that policy-makers could consider providing additional guidance for smaller companies. In general, the findings offer support for the “if not, why not?” approach of the ASX, rather than a prescriptive rules-based approach.

Investigating the development of the internal and external service tasks of non-executive directors: the case of the Netherlands (1997-2005)

During the last decade, globalisation and liberalisation of financial markets, changing societal expectations and corporate governance scandals have increased the attention for the fiduciary duties of non-executive directors. In this context, recent corporate governance reform initiatives have emphasised the control task and independence of non-executive directors. However, little attention has been paid to their impact on the external and internal service tasks of non-executive directors. Therefore, this paper investigates how the service tasks of non-executive directors have evolved in the Netherlands. Data on corporate governance at the top-100 listed companies in the Netherlands between 1997 and 2005 show that the emphasis on non-executive directors' external service task has shifted to their internal service task, i.e. from non-executive directors acting as boundary spanners to non-executive directors providing advice and counselling to executive directors. This shift in board responsibilities affects non-executive directors' ability to generate network benefits through board relationships and has implications for non-executive directors' functional requirements.

The Maori social science academy and evidence-based policy

This paper focuses on recent moves to forge stronger linkages between the Māori social science academy and the policy industry. A critical appraisal of this development is offered, with particular attention given to the desirability of enhancing the academy’s role in the policy process, given the policy industry’s continued privileging of Eurocentric theory and research methodologies within the developing evidence-based environment. The paper ends with a discussion of the possibilities and problems associated with engagement with the policy industry, particularly as these relate to the various roles members can (or are forced to) take; either as ‘insiders’ (such as policy workers and contract researchers), or independent, critical ‘outsiders’. The author concludes that the best that insiders can hope for are incremental, largely ineffective changes to Māori policy, while independent members of the academy are best placed to speak on behalf of Māori, Māori communities, hapu and iwi.

Collaborative inquiries into literacy, place and identity in changing policy contexts : implications for teacher development

In this chapter we describe a history of collaboration between university-based literacy researchers and school-based teachers in teacher development programs and practitioner inquiries designed to improve literacy outcomes for students living in low-socioeconomic circumstances. We consider how an inquiry stance has informed teachers working for social justice through curriculum and pedagogy designed to connect children’s developing literacy repertoires with their changing material, social and linguistic contexts. We use examples from the practices of two of our long-term teacher-collaborators to show what has been possible to achieve, even in radically different policy contexts, because of teachers’ continued commitment to themes of place and belonging, and language and identity.

Having a yarn about smoking : using action research to develop a 'no smoking' policy within an Aboriginal health organisation

Objectives: This article reports on a culturally appropriate process of development of a smoke-free workplace policy within the peak Aboriginal Controlled Community Health Organisation in Victoria, Australia. Smoking is acknowledged as being responsible for at least 20% of all deaths in Aboriginal communities in Australia, and many Aboriginal health workers smoke. Methods: The smoke-free workplace policy was developed using the iterative, discursive and experience-based methodology of Participatory Action Research, combined with the culturally embedded concept of ‘having a yarn’. Results: Staff members initially identified smoking as a topic to be avoided within workplace discussions. This was due, in part, to grief (everyone had suffered a smoking related bereavement). Further, there was anxiety that discussing smoking would result in culturally difficult conflict. The use of yarning opened up a safe space for discussion and debate,enabling development of a policy that was accepted across the organisation. Conclusions: Within Aboriginal organisations, it is not sufficient to focus on the outcomes of policy development. Rather, due attention must be paid to the process employed in development of policy, particularly when that policy is directly related to an emotionally and communally weighted topic such as smoking.

External auditors' reliance on internal auditing : further evidence.

Purpose – The purpose of this paper is to explore whether internal audit’s reporting relationship with the audit committee and the client’s business risk environment impact external auditors’ reliance on the work of internal audit. Design/methodology/approach – An experiment is conducted using a 2 £ 2 between-subjects design where we manipulate the above two factors at strong and weak levels. Participants are 66 audit partners, managers and seniors, all experienced with clients having internal audit functions. Findings – The results indicate that both factors affect external auditors’ reliance on work already undertaken by internal audit and their use of internal auditors (IA) as assistants. The results also indicate that external auditors are more likely to use internal audit for control evaluation tasks than for substantive tests of balances. The study does not find any significant interaction effects between the two factors. Originality/value – No prior studies have examined the influence of reporting relationship and client business risk on external auditors’ reliance decisions in the current governance environment. Further, the paper examines the impact of these factors on reliance on work already undertaken by internal audit and on using IA as assistants, with respect to both control evaluation work and substantive testing of balances.

Connecting online with external students : do we really know how to?

Students are often time poor and find it difficult to manage their time in relation to study and other external factors including work. Online study is no exception to this and in many cases where the student is studying in an online only environment, they are also working in full time employment. Higher Education institutions are now offering an abundance of courses online to attract more under-graduate and post-graduate students. It is in this sense that there is an ever-increasing need to understand the student of today and find ways to connect with them and support them in their studies. This paper will report on a small-scale case study of an undergraduate online-only group of first year education students and their associated online experiences in developing a sense of community whilst interacting with a learning management system and its associated tools. Further the paper will explore the mis-conceptions that are widely held by course designers and lecturers involved with online courses.