Developing speaking assessment tasks to reflect the 'social turn' in language testing

Interactional competence has emerged as a focal point for language testing researchers in recent years. In spoken communication involving two or more interlocutors, the co-construction of discourse is central to successful interaction. The acknowledgement of co-construction has led to concern over the impact of the interlocutor and the separability of performances in speaking tests involving interaction. The purpose of this article is to review recent studies of direct relevance to the construct of interactional competence and its operationalisation by raters in the context of second language speaking tests. The review begins by tracing the emergence of interaction as a criterion in speaking tests from a theoretical perspective, and then focuses on research salient to interactional effectiveness that has been carried out in the context of language testing interviews and group and paired speaking tests.

The relationship between listening and other language skills in International English Language Testing System

Listening comprehension is the primary channel of learning a language. Yet of the four dominant macro-skills (listening, speaking, reading and writing), it is often difficult and inaccessible for second and foreign language learners due to its implicit process. The secondary skill, speaking, proceeds listening cognitively. Aural/oral skills precede the graphic skills, such as reading and writing, as they form the circle of language learning process. However, despite the significant relationship with other language skills, listening comprehension is treated lightly in the applied linguistics research. Half of our daily conversation and three quarters of classroom interaction are virtually devoted to listening comprehension. To examine the relationship of listening skill with other language skills, the outcome of 1800 Iranian participants undertaking International English Language Testing System (IELTS) in Tehran indicates the close correlation between listening comprehension and the overall language proficiency.

Listening skill requires a further look into second/foreign language learning

Current English-as-a-second and foreign-language (ESL/EFL) research has encouraged to treat each communicative macroskill separately due to space constraint, but the interrelationship among these skills (listening, speaking, reading, and writing) is not paid due attention. This study attempts to examine first the existing relationship among the four dominant skills, second the potential impact of reading background on the overall language proficiency, and finally the relationship between listening and overall language proficiency as listening is considered an overlooked/passive skill in the pedagogy of the second/foreign language classroom. However, the literature in language learning has revealed that listening skill has salient importance in both first and second language learning. The purpose of this study is to investigate the role of each of four skills in EFL learning and their existing interrelationships in an EFL setting. The outcome of 701 Iranian applicants undertaking International English Language Testing System (IELTS) in Tehran demonstrates that all communicative macroskills have varied correlations from moderate (reading and writing) to high (listening and reading). The findings also show that the applicants’ reading history assisted them in better performing at high stakes tests, and what is more, listening skill was strongly correlated with the overall language proficiency.

Co-constructed interaction in a paired speaking test : the rater's perspective

The definition and operationalisation of interactional competence in speaking tests that entail co-construction of discourse is an area of language testing requiring further research. This article explores the reactions of four trained raters to paired candidates who oriented to asymmetric patterns of interaction in a discussion task. Through an analysis of candidate discourse combined with rater notes, stimulated verbal recalls, rater discussions and scores awarded for interactional effectiveness, the article examines the extent to which raters compensate or penalise candidates for their role in co-constructing asymmetric interactional patterns. The article argues that key features of the interaction are perceived by the raters as mutual achievements, and it further suggests that the awarding of shared scores for interactional competence is one way of acknowledging the inherently co-constructed nature of interaction in a paired speaking test.

Discourse and assessment

Many contemporary currents in applied linguistics have favored discourse studies within assessment; there have been calls for cross-fertilization with other areas within applied linguistics, critiques of the positivist tradition within language testing research, and the growing impact of Conversation Analysis (CA) and sociocultural theory. This chapter focuses on the resulting increase in discourse-based studies of oral proficiency assessment techniques. These studies initially focused on the traditional oral proficiency interview but have since been extended to new test formats, including paired and group interaction. We discuss the research carried out on a number of factors in the assessment setting, including the role of the interlocutor, candidate, and rater, and the impact of tasks, task performance conditions, and rating criteria. Recent research has also concentrated more specifically on the assessment of pragmatic competence and on the applications of technology within the assessment of spoken language, including the comparability of semidirect and direct methods for such assessment and the use of computer corpora.

Interaction in a Paired Speaking Test : the Rater's Perspective

Paired speaking tests are increasingly used in both low-and high-stakes second language assessment contexts. Until recently, very little was known about the way in which raters interpret and apply descriptors relating to interactional competence to a performance that is co-constructed. This book presents a study which explores the interactional features of a paired speaking test that were sailient to raters and the extent to which raters viewed the performance as separable. The study shows that raters use their own frames of reference to interpret descriptors and that they viewed certain features of the performance as mutual accomplishments. The book takes us 'beyond scores', and in doing so, contributes to the growing body of research on paired speaking tests.

Consumer guarantees and the supply of educational services by higher education providers

The higher education sector is undergoing a number of significant changes, the implications of which have yet to emerge. One such change is the increasing reliance by higher education providers on the revenue generated by full fee paying international students to fund their operating expenses. The report by the Victorian Ombudsman, Investigation into how Universities Deal with International Students ('Victorian Ombudsman's Report') tabled in the Victorian Parliament on 27 October 2011, provides evidence that Australian higher education providers may be failing to meet their legal obligations to international students. The Victorian Ombudsman's Report is the result of an investigation into four Victorian universities teaching international students with a focus on accounting and nursing schools. The report contains evidence that the universities were admitting students with scores below, or at the lower end of, the International English Language Testing System ('IELTS') score considered acceptable. Alternatively, they were relying upon their own language testing admission standards and not on an independent test like the IELTS test. While the universities provided English language support services for their international students after they had been admitted, the Ombudsman was concerned that the universities 'have not dedicated sufficient resources to meet the level of need amongst international students'.

Less-skilled learners benefit more from metacognitive instruction to develop listening comprehension

A growing interest in using metacognitive instruction to develop listening comprehension has emerged for almost two decades. This paper investigates the impact of metacognitive instruction on less-skilled and more-skilled learners’ listening comprehension. Thirty-two female adult, Iranian, intermediate level English as a Foreign Language (EFL) learners participated in a ‘strategy-based’ instruction, planning, monitoring and evaluation. Each of three metacognitive strategies focused on promoting learners’ comprehension of International English Language Testing System (IELTS) listening texts. A comparison of pre- and post-test scores showed that the less-skilled learners benefited more from metacognitive instruction than more-skilled learners in IELTS listening tests.

The role of metacognition in the development of EFL learners’ listening skill

This study investigated the impact of metacognitive instruction on the listening skill, and metacognitive knowledge of a group of male students (N = 30) who were learning English as a Foreign Language (EFL) in Iran. The study participants spoke Persian as a first language and were high-intermediate EFL learners. The participants received a guided lesson plan in metacognition (planning, monitoring, and evaluation) through a pedagogical cycle approach over a semester (eight weeks). International English Language Testing System listening tests (practice) were used to track the participants’ listening performance. Participants also completed a Metacognitive Awareness Listening Questionnaire, which examined their use of metacognitive awareness when engaged in listening tasks. Results indicated that the students improved their listening skill after being taught about metacognition; however, no significant use of metacognitive awareness was reported. This study concludes with a discussion of some potential implications, and provides scope for future research.

Analysing children’s persuasive texts : the use of hedging and intensification

Research Quality This is a dialogue between two Australian literacy scholars about two persuasive writing techniques that posed difficulty for the students in our research. This dialogue flows from the analysis of Year 6 writing samples from an ARC Linkage Project, URLearning (2009-2013) - the focus of the symposium. We use vivid examples of writing from students’ handwritten persuasive texts on topics that were chosen by teachers. The persuasive structure in the texts followed the Toulmin (2003) model: a thesis statement, three arguments with evidence, and a conclusion. The findings show that to realise the effective power of rhetorical persuasion, students need an expanded lexicon that does not rely on intensifiers, and which employs a greater range of advanced hedging techniques to use to their advantage. National & International Importance The study is potentially of national and international relevance, given that argumentation or persuasion is a key life skill in many professional, personal, and discourses. It is also a requirement in the International English Language Testing Systems (IELTS) tests, which are a critical gateway for tertiary studies in many English-speaking countries (Coffin, 2004). Timeliness The research is timely given the Australian Curriculum English, in which persuasive texts figure prominently from Preparatory to Year 10 (ACARA, 2014). The recommendations are also timely in the context of educational policies in other parts of the world. For example, in the United States, the Common Core Standards: English Language Arts, mandates the teaching of persuasive texts (Council of Chief State School Officers & National Governors Association, 2013) Implications for practice/policy The findings of the study have specific practical implications for teachers, who can address the persuasive writing techniques of hedging and intensification with which children need targeted support and explicit instruction. The presentation is positioned at the nexus of teacher practice to better address the national priorities of the Australian Curriculum: English (ACARA, 2014), while having implications for applied linguistics research by identifying common problems in students' persuasive writing.

Developing and testing readability measurements for second language learners

This research constructed a readability measurement for French speakers who view English as a second language. It identified the true cognates, which are the similar words from these two languages, as an indicator of the difficulty of an English text for French people. A multilingual lexical resource is used to detect true cognates in text, and Statistical Language Modelling to predict the predict the readability level. The proposed enhanced statistical language model is making a step in the right direction by improving the accuracy of readability predictions for French speakers by up to 10% compared to state of the art approaches. The outcome of this study could accelerate the learning process for French speakers who are studying English. More importantly, this study also benefits the readability estimation research community, presenting an approach and evaluation at sentence level as well as innovating with the use of cognates as a new text feature.

Assessment and curriculum reform in China : the college English test and tertiary English as a foreign language education.

Educational assessment was a worldwide commonplace practice in the last century. With the theoretical underpinnings of education shifting from behaviourism and social efficiency to constructivism and cognitive theories in the past two decades, the assessment theories and practices show a widespread changing movement. The emergent assessment paradigm, with a futurist perspective, indicates a deviation away from the prevailing large scale high-stakes standardised testing and an inclination towards classroom-based formative assessment. Innovations and reforms initiated in attempts to achieve better education outcomes for a sustainable future via more developed learning and assessment theories have included the 2007 College English Reform Program (CERP) in Chinese higher education context. This paper focuses on the College English Test (CET) - the national English as a Foreign Language (EFL) testing system for non-English majors at tertiary level in China. It seeks to explore the roles that the CET played in the past two College English curriculum reforms, and the new role that testing and assessment assumed in the newly launched reform. The paper holds that the CET was operationalised to uplift the standards. However, the extended use of this standardised testing system brings constraints as well as negative washback effects on the tertiary EFL education. Therefore in the newly launched reform -CERP, a new assessment model which combines summative and formative assessment approaches is proposed. The testing and assessment, assumed a new role - to engender desirable education outcomes. The question asked is: will the mixed approach to formative and summative assessment provide the intended cure to the agony that tertiary EFL education in China has long been suffering - spending much time, yet achieving little effects? The paper reports the progresses and challenges as informed by the available research literature, yet asserts a lot needs to be explored on the potential of the assessment mix in this examination tradition deep-rooted and examination-obsessed society.

Testing for the non-separability of bi-ambiguous compounds

Separability is a concept that is very difficult to define, and yet much of our scientific method is implicitly based upon the assumption that systems can sensibly be reduced to a set of interacting components. This paper examines the notion of separability in the creation of bi-ambiguous compounds that is based upon the CHSH and CH inequalities. It reports results of an experiment showing that violations of the CHSH and CH inequality can occur in human conceptual combination.

Curriculum change in testing times

In these new, testing times, we need a new controlled rapid approach to curriculum change. Judy Smeed and TerrI Bourke explain.

The impact of high stakes testing: the Australian story

High stakes testing in Australia was introduced in 2008 by way of the National Assessment Program – Literacy and Numeracy (NAPLAN). Currently, every year all students in Years 3, 5, 7 and 9 are assessed on the same days using national tests in Reading, Writing, Language Conventions (Spelling, Grammar and Punctuation) and Numeracy. In 2010 the NAPLAN results were published on the Federal Government MySchool website. The impact of these high stakes tests on jurisdictions, school principals, parents and students is considered in this article. We draw on reported observations from the Australian Primary Principals Association during 2009–10 testing periods across the country and published Australian research on the impact of high stakes literacy and numeracy testing. We also examine alternative approaches that include the use of assessment evidence for learning improvement purposes and for accountability purposes. In considering alternatives to the current large-scale testing approach we draw on key insights from research on teacher judgement, achievement standards and social moderation in the context of national curriculum and assessment reform in support of the suggested directions forward.