Developing speaking assessment tasks to reflect the 'social turn' in language testing

Interactional competence has emerged as a focal point for language testing researchers in recent years. In spoken communication involving two or more interlocutors, the co-construction of discourse is central to successful interaction. The acknowledgement of co-construction has led to concern over the impact of the interlocutor and the separability of performances in speaking tests involving interaction. The purpose of this article is to review recent studies of direct relevance to the construct of interactional competence and its operationalisation by raters in the context of second language speaking tests. The review begins by tracing the emergence of interaction as a criterion in speaking tests from a theoretical perspective, and then focuses on research salient to interactional effectiveness that has been carried out in the context of language testing interviews and group and paired speaking tests.

The relationship between listening and other language skills in International English Language Testing System

Listening comprehension is the primary channel of learning a language. Yet of the four dominant macro-skills (listening, speaking, reading and writing), it is often difficult and inaccessible for second and foreign language learners due to its implicit process. The secondary skill, speaking, proceeds listening cognitively. Aural/oral skills precede the graphic skills, such as reading and writing, as they form the circle of language learning process. However, despite the significant relationship with other language skills, listening comprehension is treated lightly in the applied linguistics research. Half of our daily conversation and three quarters of classroom interaction are virtually devoted to listening comprehension. To examine the relationship of listening skill with other language skills, the outcome of 1800 Iranian participants undertaking International English Language Testing System (IELTS) in Tehran indicates the close correlation between listening comprehension and the overall language proficiency.

Standard setting in specific-purpose language testing: what can a qualitative study add?

This paper explores the views of nursing and medical domain experts in considering the standards for a specific-purpose English language screening test, the Occupational English Test (OET), for professional registration for immigrant health professionals. Since individuals who score performances in the test setting are often language experts rather than domain experts, there are possible tensions between what is being measured by a language test and what is deemed important by domain experts. Another concern is a lack of qualitative research on the process of the standard setting. To date, no published qualitative work has been identified about the contributions of domain experts in the standard setting for healthcare communication. In this study, a standard-setting exercise was conducted for the speaking component of the OET, using judgements of nursing and medical clinical educators and supervisors. In all, 13 medical and 18 nursing clinical educators and supervisors rated medical and nursing candidate performances respectively. These performances were audio-recorded OET role-plays that were selected across a range of proficiency levels. Domain experts were invited to comment on the basis of their decisions and the extent of alignment between these decisions and the criteria used to assess performance on the OET. Nursing and medical domain experts showed that they attended to all of the OET criteria in making their decisions about standards. However, clinical scenario simulation also invited judgements of clinical competence from participants, even where they knew that clinical competence should be excluded from their decision-making. Another concern related to the authenticity limitations of the role-play tasks as evidence of readiness to handle communication in the workplace. Overall, findings support the value of qualitative evidence from the standard setting in providing insight into the factors informing and impeding decision-making.

Listening skill requires a further look into second/foreign language learning

Current English-as-a-second and foreign-language (ESL/EFL) research has encouraged to treat each communicative macroskill separately due to space constraint, but the interrelationship among these skills (listening, speaking, reading, and writing) is not paid due attention. This study attempts to examine first the existing relationship among the four dominant skills, second the potential impact of reading background on the overall language proficiency, and finally the relationship between listening and overall language proficiency as listening is considered an overlooked/passive skill in the pedagogy of the second/foreign language classroom. However, the literature in language learning has revealed that listening skill has salient importance in both first and second language learning. The purpose of this study is to investigate the role of each of four skills in EFL learning and their existing interrelationships in an EFL setting. The outcome of 701 Iranian applicants undertaking International English Language Testing System (IELTS) in Tehran demonstrates that all communicative macroskills have varied correlations from moderate (reading and writing) to high (listening and reading). The findings also show that the applicants’ reading history assisted them in better performing at high stakes tests, and what is more, listening skill was strongly correlated with the overall language proficiency.

The generation effect and word learning: a test of the effect of the language experience approach versus the text approach in the acquisition of new reading vocabulary

A class of twenty-two grade one children was tested to determine their reading levels using the Stanford Diagnostic Reading Achievement Test. Based on these results and teacher input the students were paired according to reading ability. The students ages ranged from six years four months to seven years four months at the commencement of the study. Eleven children were assigned to the language experience group and their partners became the text group. Each member of the language experience group generated a list of eight to be learned words. The treatment consisted of exposing the student to a given word three times per session for ten sessions, over a period of five days. The dependent variables consisted of word identification speed, word identification accuracy, and word recognition accuracy. Each member of the text group followed the same procedure using his/her partner's list of words. Upon completion of this training, the entire process was repeated with members of the text group from the first part becoming members of the language experience group and vice versa. The results suggest that generally speaking language experience words are identified faster than text words but that there is no difference in the rate at which these words are learned. Language experience words may be identified faster because the auditory-semantic information is more readily available in them than in text words. The rate of learning in both types of words, however, may be dictated by the orthography of the to be learned word.

Syntactic complexity measures and their relation to oral proficiency in Japanese as a foreign language

The study reported in this article is a part of a large-scale study investigating syntactic complexity in second language (L2) oral data in commonly taught foreign languages (English, German, Japanese, and Spanish; Ortega, Iwashita, Rabie, & Norris, in preparation). In this article, preliminary findings of the analysis of the Japanese data are reported. Syntactic complexity, which is referred to as syntactic maturity or the use of a range of forms with degrees of sophistication (Ortega, 2003), has long been of interest to researchers in L2 writing. In L2 speaking, researchers have examined syntactic complexity in learner speech in the context of pedagogic intervention (e.g., task type, planning time) and the validation of rating scales. In these studies complexity is examined using measures commonly employed in L2 writing studies. It is assumed that these measures are valid and reliable, but few studies explain what syntactic complexity measures actually examine. The language studied is predominantly English, and little is known about whether the findings of such studies can be applied to languages that are typologically different from English. This study examines how syntactic complexity measures relate to oral proficiency in Japanese as a foreign language. An in-depth analysis of speech samples from 33 learners of Japanese is presented. The results of the analysis are compared across proficiency levels and cross-referenced with 3 other proficiency measures used in the study. As in past studies, the length of T-units and the number of clauses per T-unit is found to be the best way to predict learner proficiency; the measure also had a significant linear relation with independent oral proficiency measures. These results are discussed in light of the notion of syntactic complexity and the interfaces between second language acquisition and language testing. Adapted from the source document

Co-constructed interaction in a paired speaking test : the rater's perspective

The definition and operationalisation of interactional competence in speaking tests that entail co-construction of discourse is an area of language testing requiring further research. This article explores the reactions of four trained raters to paired candidates who oriented to asymmetric patterns of interaction in a discussion task. Through an analysis of candidate discourse combined with rater notes, stimulated verbal recalls, rater discussions and scores awarded for interactional effectiveness, the article examines the extent to which raters compensate or penalise candidates for their role in co-constructing asymmetric interactional patterns. The article argues that key features of the interaction are perceived by the raters as mutual achievements, and it further suggests that the awarding of shared scores for interactional competence is one way of acknowledging the inherently co-constructed nature of interaction in a paired speaking test.

Discourse and assessment

Many contemporary currents in applied linguistics have favored discourse studies within assessment; there have been calls for cross-fertilization with other areas within applied linguistics, critiques of the positivist tradition within language testing research, and the growing impact of Conversation Analysis (CA) and sociocultural theory. This chapter focuses on the resulting increase in discourse-based studies of oral proficiency assessment techniques. These studies initially focused on the traditional oral proficiency interview but have since been extended to new test formats, including paired and group interaction. We discuss the research carried out on a number of factors in the assessment setting, including the role of the interlocutor, candidate, and rater, and the impact of tasks, task performance conditions, and rating criteria. Recent research has also concentrated more specifically on the assessment of pragmatic competence and on the applications of technology within the assessment of spoken language, including the comparability of semidirect and direct methods for such assessment and the use of computer corpora.

Interaction in a Paired Speaking Test : the Rater's Perspective

Paired speaking tests are increasingly used in both low-and high-stakes second language assessment contexts. Until recently, very little was known about the way in which raters interpret and apply descriptors relating to interactional competence to a performance that is co-constructed. This book presents a study which explores the interactional features of a paired speaking test that were sailient to raters and the extent to which raters viewed the performance as separable. The study shows that raters use their own frames of reference to interpret descriptors and that they viewed certain features of the performance as mutual accomplishments. The book takes us 'beyond scores', and in doing so, contributes to the growing body of research on paired speaking tests.

Consumer guarantees and the supply of educational services by higher education providers

The higher education sector is undergoing a number of significant changes, the implications of which have yet to emerge. One such change is the increasing reliance by higher education providers on the revenue generated by full fee paying international students to fund their operating expenses. The report by the Victorian Ombudsman, Investigation into how Universities Deal with International Students ('Victorian Ombudsman's Report') tabled in the Victorian Parliament on 27 October 2011, provides evidence that Australian higher education providers may be failing to meet their legal obligations to international students. The Victorian Ombudsman's Report is the result of an investigation into four Victorian universities teaching international students with a focus on accounting and nursing schools. The report contains evidence that the universities were admitting students with scores below, or at the lower end of, the International English Language Testing System ('IELTS') score considered acceptable. Alternatively, they were relying upon their own language testing admission standards and not on an independent test like the IELTS test. While the universities provided English language support services for their international students after they had been admitted, the Ombudsman was concerned that the universities 'have not dedicated sufficient resources to meet the level of need amongst international students'.

Less-skilled learners benefit more from metacognitive instruction to develop listening comprehension

A growing interest in using metacognitive instruction to develop listening comprehension has emerged for almost two decades. This paper investigates the impact of metacognitive instruction on less-skilled and more-skilled learners’ listening comprehension. Thirty-two female adult, Iranian, intermediate level English as a Foreign Language (EFL) learners participated in a ‘strategy-based’ instruction, planning, monitoring and evaluation. Each of three metacognitive strategies focused on promoting learners’ comprehension of International English Language Testing System (IELTS) listening texts. A comparison of pre- and post-test scores showed that the less-skilled learners benefited more from metacognitive instruction than more-skilled learners in IELTS listening tests.

The role of metacognition in the development of EFL learners’ listening skill

This study investigated the impact of metacognitive instruction on the listening skill, and metacognitive knowledge of a group of male students (N = 30) who were learning English as a Foreign Language (EFL) in Iran. The study participants spoke Persian as a first language and were high-intermediate EFL learners. The participants received a guided lesson plan in metacognition (planning, monitoring, and evaluation) through a pedagogical cycle approach over a semester (eight weeks). International English Language Testing System listening tests (practice) were used to track the participants’ listening performance. Participants also completed a Metacognitive Awareness Listening Questionnaire, which examined their use of metacognitive awareness when engaged in listening tasks. Results indicated that the students improved their listening skill after being taught about metacognition; however, no significant use of metacognitive awareness was reported. This study concludes with a discussion of some potential implications, and provides scope for future research.

Analysing children’s persuasive texts : the use of hedging and intensification

Research Quality This is a dialogue between two Australian literacy scholars about two persuasive writing techniques that posed difficulty for the students in our research. This dialogue flows from the analysis of Year 6 writing samples from an ARC Linkage Project, URLearning (2009-2013) - the focus of the symposium. We use vivid examples of writing from students’ handwritten persuasive texts on topics that were chosen by teachers. The persuasive structure in the texts followed the Toulmin (2003) model: a thesis statement, three arguments with evidence, and a conclusion. The findings show that to realise the effective power of rhetorical persuasion, students need an expanded lexicon that does not rely on intensifiers, and which employs a greater range of advanced hedging techniques to use to their advantage. National & International Importance The study is potentially of national and international relevance, given that argumentation or persuasion is a key life skill in many professional, personal, and discourses. It is also a requirement in the International English Language Testing Systems (IELTS) tests, which are a critical gateway for tertiary studies in many English-speaking countries (Coffin, 2004). Timeliness The research is timely given the Australian Curriculum English, in which persuasive texts figure prominently from Preparatory to Year 10 (ACARA, 2014). The recommendations are also timely in the context of educational policies in other parts of the world. For example, in the United States, the Common Core Standards: English Language Arts, mandates the teaching of persuasive texts (Council of Chief State School Officers & National Governors Association, 2013) Implications for practice/policy The findings of the study have specific practical implications for teachers, who can address the persuasive writing techniques of hedging and intensification with which children need targeted support and explicit instruction. The presentation is positioned at the nexus of teacher practice to better address the national priorities of the Australian Curriculum: English (ACARA, 2014), while having implications for applied linguistics research by identifying common problems in students' persuasive writing.