Multiple-choice versus short-response items: Differences in omit behavior

The overall rate of omission of items for 28,331 17 year old Australian students on a high stakes test of achievement in the common elements or cognitive skills of the senior school curriculum is reported for a subtest in multiple choice format and a subtest in short response format. For the former, the omit rates were minuscule and there was no significant difference by gender or by type of school attended. For the latter, where an item can be 'worth' up to five times that of a single multiple choice item, the omit rates were between 10 and 20 times that for multiple choice and the difference between male and female omit rate was significant as was the difference between students from government and non-government schools. For both formats, females from single sex schools omitted significantly fewer items than did females from co-educational schools. Some possible explanations of omit behaviour are alluded to.

Testing for accountability

Current graduates in education are entering a very different profession to the one in which most of their “baby-boomer” colleagues started. It is a profession in which accountability and national high-stakes testing (e.g. NAPLAN) have become catch-cries, and where the interpretation and use of educational data is an additional challenge. This has led to schools focusing on performance, and teachers now have to analyse test data and apply the findings to their teaching.

Parents’ risky choices around the IB Diploma

In recent years there has been a rapid growth in the International Baccalaureate Diploma(IBD), a secondary curriculum administered by the International Baccalaureate Organisation(IBO), as an alternative to the local curriculum in Australian schools in some schools. This growth is indicative of an increasing demand from Australian families for new educational structures, practices and processes. With more curriculum options and pathways such as the IBD available in the secondary education system, parents are faced with a more complex high stakes decision when it comes to choosing the optimal education path for their offspring, one which requires a careful assessment of potential outcomes and risks. This paper reports on the responses of 184 parents to an online survey conducted in 26 Australian schools that offer the IBD as a curricular alternative. It examines which parents either chose, or chose not to, enrol their children in the program, why, and what risks they perceived to be associated with that choice. The paper will compare the choice behaviour of the two groups of parents from a sociological perspective, framing the enquiry with reference to globalisation and neo-liberal education policy and its effect on parental choice of schooling. This paper will make evident how parental choice of educational alternatives has become a more complicated process for Australian families.

Interactional competence in a paired speaking test : features salient to raters

Paired speaking tests are now commonly used in both high-stakes testing and classroom assessment contexts. The co-construction of discourse by candidates is regarded as a strength of paired speaking tests, as candidates have the opportunity to display a wider range of interactional competencies, including turn taking, initiating topics and engaging in extended discourse with a partner, rather than an examiner. However, the impact of the interlocutor in such jointly negotiated discourse and the implications for assessing interactional competence are areas of concern. This article reports on the features of interactional competence that were salient to four trained raters of 12 paired speaking tests through the analysis of rater notes, stimulated verbal recalls and rater discussions. Findings enabled the identification of features of the performance noted by raters when awarding scores for interactional competence, and the particular features associated with higher and lower scores. A number of these features were seen by the raters as mutual achievements, which raises the issue of the extent to which it is possible to assess individual contributions to the co-constructed performance. The findings have implications for defining the construct of interactional competence in paired speaking tests and operationalising this in rating scales.

Interaction in a Paired Speaking Test : the Rater's Perspective

Paired speaking tests are increasingly used in both low-and high-stakes second language assessment contexts. Until recently, very little was known about the way in which raters interpret and apply descriptors relating to interactional competence to a performance that is co-constructed. This book presents a study which explores the interactional features of a paired speaking test that were sailient to raters and the extent to which raters viewed the performance as separable. The study shows that raters use their own frames of reference to interpret descriptors and that they viewed certain features of the performance as mutual accomplishments. The book takes us 'beyond scores', and in doing so, contributes to the growing body of research on paired speaking tests.

Data-driven accountability in Australia : an unfolding story with lessons for leaders

Throughout the world, state and nation standardised testing of children, has become a "huge industry" (English, 2002). Although English is referring to the American system which has been involved in standardised testing for over half a century, the same could be said of many other countries, including Australia. It has been only in recent years that Australia has embraced national testing as part of a wider reform effort to bring about increased accountability in schooling. The results of high-stakes tests in Australia are now published in newspapers and electronically on the Australian federal government's MySchool website (www.myschoold.edu.au). MySchool provides results on the National Assessment Program - Literacy and Numeracy (NAPLAN) for students in Years 3,5, 7 and 9. Data are available that compare schools to statistically similar schools. This more recent publication of national testing results in Australia is a visible example of "contractual accountability", described by Mulford, Edmunds, Kendall, Kendall and Bishop (2008) as " the degree to which [actors] are fulfilling the expectations of particular audiences in terms of standards, outcomes and results" (p.20).

The ascendancy of oracy in university courses: Implications for teachers and second language users

This chapter reports on a study of oracy in a first-year university Business course, with particular interest in the oracy demands for second language-using international students. The research is relevant at a time when Higher Education is characterised by the confluence of increased international enrolments, more dialogic teaching and learning, and imperatives for teamwork and collaboration. Data sources for the study included videotaped lectures and tutorials, course documents, student surveys, and an interview with the lecturer. The findings pointed to a complex, oracy-laden environment where interactive talk fulfilled high-stakes functions related to social inclusion, the co-construction of knowledge, and the accomplishment of assessment tasks. The salience of talk posed significant challenges for students negotiating these core functions in their second language. The study highlights the oracy demands in university courses and foregrounds the need for university teachers, curriculum writers and speaking test developers to recognise these demands and explicate them for the benefit of all students.

Data analysis skills for teachers

Although accountability in the form of high stakes testing is in favour in the contemporary Australian educational context, this practice remains a highly contested source of debate. Proponents for high stakes tests claim that higher standards in teaching and learning result from their implementation, whereas others believe that this type of testing regime is not required and may even in fact be counterproductive. Regardless of what side of the debate you sit on, the reality is that at present, high stakes testing appears to be here to stay. It could therefore be argued it is essential that teachers understand accountability and possess the specific skills to interpret and use test data beneficially.

Creativity is as important as literacy : making a film trailer responding to literature

The first part of the title is from Sir Ken Robinson (Robinson, 2009), the esteemed educator and champion of creativity in schools. Sometimes in the face of meeting the demands of time, timetabling, demanding administration tasks and teaching for high stakes testing accountability, we can find ourselves desperate for time to remember that English has always been one of the places in schools where creativity can flourish. English is a place for the play of the imagination. English teachers are the purveyors of narrative; the keepers and teachers of stories. The new Australian Curriculum: English, (Australian Curriculum and Assessment Reporting Authority (ACARA), 2012) asks us to be using ICT technologies in creative ways to tell those stories. The curriculum asks students to access texts receptively and to then speak about, write and create texts productively. There are so many interesting things to do with texts beyond word processing of print based resources. Responding to literature through media is always an alternative option to writing or simply speaking about it. In this paper my QUT pre-service student Chrystal Armitage describes how she made a mini story via a film trailer in response to a short story, ‘Turned’ (Gilman, 1987) in the unit, Literature in Secondary Teaching.

Assessing the damage caused by deepwater horizon : not just another Exxon Valdez

In light of the high stakes of the deepwater horizon civil trial and the important precedent-setting role that the case will have on the assessment of future marine disasters, the methodologies underpinning the calculations of damage on both sides will be subjected to considerable scrutiny. Despite the importance of the case, however, there seems to be a pronounced lack of convergence about it in the academic literature. Contributions from scientific journals frequently make comparisons to the Ixtoc I oil spill off the coast of Mexico in 1979; the legal literature, by stark contrast, seems to be much more focused on the Exxon Valdez spill that occurred off the shores of Alaska in 1989. This paper accordingly calls for a more thorough consideration of other analogs beyond the Exxon Valdez spill—most notably, the Ixtoc I incident—in arriving at an assessment of the damage caused by the Deepwater Horizon disaster.

The spatialized practices of teaching writing in Australian elementary schools : diverse students shaping discoursal selves

This paper discusses the teaching of writing within the competing and often contradictory spaces of high-stakes testing and the practices and priorities around writing pedagogy in diverse school communities. It uses socio-spatial theory to examine the real-and-imagined spaces (Soja, 1996) that influence and are influenced by teachers’ pedagogical priorities for writing in two linguistically diverse elementary school case studies. Methods of critical discourse analysis are used to examine rich data sets to make visible the discourses and power relations at play in the case schools. Findings show that when teachers’ practices focus on the teaching of structure and skills alongside identity building and voice, students with diverse linguistic backgrounds can produce dramatic, authoritative and resonant texts. The paper argues that “thirdspaces” can be forged that both attend to accountability requirements, yet also give the necessary attention to more complex aspects of writing necessary for students from diverse and multilingual backgrounds to invest in writing as a creative and critical form of communication for participation in society and the knowledge economy.

Bilingualism, literacy and NAPLAN : ongoing challenges for EAL/D education [Editorial]

Advocacy is integral to the work of many TESOL specialists. For several decades, ACTA and the state TESOL associations, along with other professional associations, and individual teachers, researchers and administrators have all engaged with conversations about EAL/D education in public forums. These advocates have drawn attention to implications of policy developments for EAL/D students; they have proffered alternative forms of curriculum, pedagogy and assessment to better account for the particularity of EAL/D learning pathways; they have argued the necessity of specialist EAL/D teaching. In response to the Australian Language and Literacy Policy of the early 1990s, for example, there was “a frenzy of writing responses… a conference… and attempts to publicise what was going on through the press and television” (Moore, 1995, p. 6). It is in this spirit that this double issue of TESOL in Context has been compiled...

Reframing Conceptions of Contemporary Literacy Capabilities in Pre-Service Primary Teacher Education

This chapter describes the challenges of integrating new technologies with literacy education in pre-service primary teacher education in Australia. The authors describe the policy context and regulatory mechanisms controlling pre-service education, including a national set of professional standards for graduate teachers, a new national curriculum for school students, the introduction of high stakes national assessment for school students, and the looming threat of decontextualized back-to-the-basics professional entry tests for aspiring teachers. The chapter includes three case studies of the authors’ pedagogical practices that attempt to reframe conceptions of the literacy capabilities of pre-service teachers to reflect the complex and sophisticated requirements of teachers in contemporary schooling. The authors conclude the chapter with a discussion of the implications of these case studies as they illustrate the ways that pre-service teachers can be scaffolded and supported to develop creative capacity and critical awareness of the kinds of literacies required in the digital age despite restrictive regimes.

Clinical perceptions of radiation therapy undergraduate competency standards

Introduction The multifactorial nature of clinical skills development makes assessment of undergraduate radiation therapist competence level by clinical mentors challenging. A recent overhaul of the clinical assessment strategy at Queensland University of Technology has moved away from the high-stakes Observed Structured Clinical Examination (OSCE) to encompass a more continuous measure of competence. This quantitative study aimed to gather stakeholder evidence to inform development of standards by which to measure student competence for a range of levels of progression. Methods A simple anonymous questionnaire was distributed to all Queensland radiation therapists. The tool asked respondents to assign different levels of competency with a range of clinical tasks to different levels of student. All data were anonymous and was combined for analysis using Microsoft Excel. Results Feedback indicated good agreement with tasks that specified amount of direction required and this has been incorporated into the new clinical achievements record that the students need to have signed off. Additional puzzling findings suggested higher expectations with planning tasks than with treatment-based tasks. Conclusion The findings suggest that the amount of direction required by students is a valid indicator of their level and has been adopted into the clinical assessment scheme. Further work will build on this to further define standards of competency for undergraduates.

Interrogating policy contradictions in literacy reforms about persuasive texts

Objectives Our overarching objective is to demonstrate the political contradictions about about how persuasive texts should be taught in the middle years of schooling, analysing two contradictory Australian wide educational reforms. We consider the complexities of power and access to literacy for students in relation to these reforms about the privileged genre of persuasion. Our work is framed by our appreciation of literacy as a social justice issue, and the notion of students’ pedagogic rights (Bernstein, 2000). Specifically, we introduce and analyse the knowledge and skills about persuasive text sanctioned by the Australian high-stakes test, the National Assessment Program for Literacy and Numeracy (NAPLAN), for students in the middle years of schooling (ACARA, 2013). We compare this to the contemporary emphasis on multimodal persuasive texts sanctioned by the recently released Australian Curriculum English (ACARA, 2014). We conclude our analysis by identifying biases in the structure of particular knowledges and the inherent threats to democracy.