Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

The Maori social science academy and evidence-based policy

This paper focuses on recent moves to forge stronger linkages between the Māori social science academy and the policy industry. A critical appraisal of this development is offered, with particular attention given to the desirability of enhancing the academy’s role in the policy process, given the policy industry’s continued privileging of Eurocentric theory and research methodologies within the developing evidence-based environment. The paper ends with a discussion of the possibilities and problems associated with engagement with the policy industry, particularly as these relate to the various roles members can (or are forced to) take; either as ‘insiders’ (such as policy workers and contract researchers), or independent, critical ‘outsiders’. The author concludes that the best that insiders can hope for are incremental, largely ineffective changes to Māori policy, while independent members of the academy are best placed to speak on behalf of Māori, Māori communities, hapu and iwi.

Responsibility to Protect and Sovereignty

The responsibility to protect ('R2P') principle articulates the obligations of the international community to prevent conflict occurring, to intervene in conflicts, and to assist in rebuilding after conflicts. The doctrine is about protecting civilians in armed conflicts from four mass atrocity crimes: genocide, war crimes, crimes against humanity and ethnic cleansing. This book examines interventions in East Timor, Sri Lanka, Sudan and Kosovo. The chapters explore and question UN debates with respect to the doctrine both before and after its adoption in 2005; contrasting state attitudes to international military intervention; and what takes place after intervention. It also discusses the ability of the Security Council to access reliable information and credible and transparent processes to enable it to make a determination on the occurrence of atrocities in a Member State. Questioning whether there is a need to find a closer operational link between the responsibilities to prevent and react and a normative link between R2P and principles of international law, the contributions examine the effectiveness of the framework of R2P for international decision-making in response to mass atrocity crimes and ask how an international system to deal with threats and mass atrocities can be developed in the absence of a central authority. This book will be valuable to those interested in international law, human rights, and security, peace and conflict studies

Collaborative inquiries into literacy, place and identity in changing policy contexts : implications for teacher development

In this chapter we describe a history of collaboration between university-based literacy researchers and school-based teachers in teacher development programs and practitioner inquiries designed to improve literacy outcomes for students living in low-socioeconomic circumstances. We consider how an inquiry stance has informed teachers working for social justice through curriculum and pedagogy designed to connect children’s developing literacy repertoires with their changing material, social and linguistic contexts. We use examples from the practices of two of our long-term teacher-collaborators to show what has been possible to achieve, even in radically different policy contexts, because of teachers’ continued commitment to themes of place and belonging, and language and identity.

Having a yarn about smoking : using action research to develop a 'no smoking' policy within an Aboriginal health organisation

Objectives: This article reports on a culturally appropriate process of development of a smoke-free workplace policy within the peak Aboriginal Controlled Community Health Organisation in Victoria, Australia. Smoking is acknowledged as being responsible for at least 20% of all deaths in Aboriginal communities in Australia, and many Aboriginal health workers smoke. Methods: The smoke-free workplace policy was developed using the iterative, discursive and experience-based methodology of Participatory Action Research, combined with the culturally embedded concept of ‘having a yarn’. Results: Staff members initially identified smoking as a topic to be avoided within workplace discussions. This was due, in part, to grief (everyone had suffered a smoking related bereavement). Further, there was anxiety that discussing smoking would result in culturally difficult conflict. The use of yarning opened up a safe space for discussion and debate,enabling development of a policy that was accepted across the organisation. Conclusions: Within Aboriginal organisations, it is not sufficient to focus on the outcomes of policy development. Rather, due attention must be paid to the process employed in development of policy, particularly when that policy is directly related to an emotionally and communally weighted topic such as smoking.

Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.

National Cultural Policy submission

This was my submission to the Australian Federal Government’s call for submissions in response to the National Cultural Policy Discussion Paper.

Social outcomes in the construction industry : the case of the Western Australia ‘Percent for Art’ policy

Social outcomes, in particular intangible social outcomes, are generally difficult to achieve in the construction industry due to the predominantly episodic, fragmented and heavily regulated nature of construction that presupposes a tendency towards mainstream construction processes and design. The Western Australian ‘Percent for Art’ policy is recognized for stimulating social outcomes, by creating richer and more aesthetically pleasing social environments through the incorporation of artwork into public buildings. A case study of four Percent for Art projects highlights the role of the Artwork Selection Committee in incorporating artwork into construction. A total of 20 semi-structured interviews were conducted with committee members and policy officers. Data analysis involved a combination of pattern coding and matrix categorization, and resulted in the identification of the committee’s three key elements of collaborative communication, democratic decision-making and project champions. The findings suggest these key elements foster the interaction, communication and relationships needed to facilitate feedback, enhance relationships, create cross-functional teams and lower project resistance, which are all necessary to overcome constraints to social outcomes in construction. The findings provide greater insight into the mechanisms for achieving social outcomes and a basis for future discussion about the processes for achieving social outcomes in the construction industry.

Preventing injuries on a state-wide basis : researchers and parliamentary committees are partners in turning injury prevention and safety research into practice and policy

Context: Parliamentary committees established in Westminster parliaments, such as Queensland, provide a cross-party structure that enables them to recommend policy and legislative changes that may otherwise be difficult for one party to recommend. The overall parliamentary committee process tends to be more cooperative and less adversarial than the main chamber of parliament and, as a result, this process permits parliamentary committees to make recommendations more on the available research evidence and less on political or party considerations. Objectives: This paper considers the contributions that parliamentary committees in Queensland have made in the past in the areas of road safety, drug use as well as organ and tissue donation. The paper also discusses the importance of researchers actively engaging with parliamentary committees to ensure the best evidence based policy outcomes. Key messages: In the past, parliamentary committees have successfully facilitated important safety changes with many committee recommendations based on research results. In order to maximise the benefits of the parliamentary committee process it is essential that researchers inform committees about their work and become key stakeholders in the inquiry process. Researchers can keep committees informed by making submissions to their inquiries, responding to requests for information and appearing as witnesses at public hearings. Researchers should emphasise the key findings and implications of their research as well as considering the jurisdictional implications and political consequences. It is important that researchers understand the differences between lobbying and providing informed recommendations when interacting with committees. Discussion and conclusions: Parliamentary committees in Queensland have successfully assisted in the introduction of evidence based policy and legislation. In order to present best practice recommendations, committees rely on the evidence presented to them including the results of researchers. Actively engaging with parliamentary committees will help researchers to turn their results into practice with a corresponding decrease in injuries and fatalities. Developing an understanding of parliamentary committees, and the typical inquiry process used by these committees, will help researchers to present their research results in a manner that will encourage the adoption of their ideas by parliamentary committees, the presentation of these results as recommendations within the report and the subsequent enactment of the committee’s recommendations by the government.

Facilitating Malaysia towards innovative society : arguing the case for open access policy

The Malaysian National Innovation Model blueprint states that there is an urgent need to pursue an innovation-oriented economy to improve the nation’s capacity for knowledge, creativity and innovation. In nurturing a pervasive innovation culture, the Malaysian government has declared the year 2010 as an Innovative Year whereby creativity among its population is highly celebrated. However, while Malaysian citizens are encouraged to be creative and innovative, scientific data and information generated from publicly funded research in Malaysia is locked up because of rigid intellectual property licensing regimes and traditional publishing models. Reflecting on these circumstances, this paper looks at, and argue why, scientific data and information should be made available, accessible and re-useable freely to promote the grassroots level of innovation in Malaysia. Using innovation theory as its platform of argument, this paper calls for an open access policy for publicly funded research output to be adopted and implemented in Malaysia. Simultaneously, a normative analytic approach is used to determine the types of open access policy that ought to be adopted to spur greater innovation among Malaysians.