A case for addressing the literacy demands of student assessment

The development and implementation of the Australian Curriculum together with national testing of students and the publication of school results place new demands on teachers. In this article we address the importance of teachers becoming attuned to the silent assessors in assessment generally and in the National Literacy and Numeracy Program (NAPLAN) more specifically. Using the concept of literacies, we develop a method to conduct a literacy audit of assessment tasks that teachers can use to help both themselves and their students. Providing assistance to students as a consequence of such an audit is imperative to improve the outcomes for students and to address issues of equity.

Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.

…Where did I lose you? Accessing the literacy demands of assessment

In this study we sought to find out how teachers could make assessment fairer for Indigenous students in learning mathematics, given the context of the high stakes of the National Assessment Program Literacy and Numeracy (NAPLAN). Today, teachers are experiencing the full range of demands from their own students who require individual attention, through to system level expectations of improved performances for all students. Many staff experience reform fatigue with limited time for critical reflection and a reduction in support for the use and the analysis of the overwhelming amount of data that has become available in recent years. Over the past three years we worked with teachers in seven schools to gradually refine our research focus to centre on how we might best support teachers in this demanding context with the important outcome of improved teaching and learning of mathematics with particular consideration of how to respond to the cultural needs of Indigenous (Aboriginal and Torres Strait Islander) students.

Researching quality and equity in assessment for improved learner outcomes

Quality and equity issues as they relate to assessment practices and policy are becoming increasingly important as Australia introduces a national curriculum and achievement standards. In a context of high-stakes accountability, concerns regarding equity and quality have important implications for teachers‘ practice for the improvement of learner outcomes. This article is based on three research projects that were conducted over the past four years and were funded by the Australian Federal Government. The research focus, emergent questions, the educational contexts, and the rationale for the studies are discussed prior to the presentation of the analysis of the research findings and the implications for teachers‘ practice and policy reform.

The effects of standardised assessment (NAPLAN) on teacher pedagogy at two Queensland schools

The purpose of this qualitative interpretative case study was to explore how the National Assessment Program – Literacy and Numeracy (NAPLAN) requirements may be affecting pedagogies of two Year 3, Year 5 and Year 7 teachers at two Queensland schools. The perceived problem was that standardised assessment NAPLAN practices and its growing status as a key measure of education quality throughout Australia has the potential to limit the everyday literacy and numeracy practices of teachers to instructional methods primarily focused on teaching to the test. The findings demonstrate how increased explicit teaching of NAPLAN content and procedural knowledge prior to testing has the potential to negatively impact on the teaching of everyday literacy and numeracy skills and knowledge that extend beyond those concerned with NAPLAN. Such teaching limited opportunity for what teachers reported as valued collaborative learning contexts aiming for long-term literacy and numeracy results.

Towards fairer assessment

Drawing on the largest Australian collection and analysis of empirical data on multiple facets of Aboriginal and Torres Strait Islander education in state schools to date, this article critically analyses the systemic push for standardized testing and improved scores, and argues for a greater balance of assessment types by providing alternative, inclusive, participatory approaches to student assessment. The evidence for this article derives from a major evaluation of the Stronger Smarter Learning Communities. The first large-scale picture of what is occurring in classroom assessment and pedagogy for Indigenous students is reported in this evaluation yet the focus in this article remains on the issue of fairness in student assessment. The argument presented calls for “a good balance between formative and summative assessment” (OECD, Synergies for Better Learning An International Perspective on Evaluation and Assessment, Pointers for Policy Development, 2013) at a time of unrelenting high-stakes, standardized testing in Australia with a dominance of secondary as opposed to primary uses of NAPLAN data by systems, schools and principals. A case for more “intelligent accountability in education” (O’Neill, Oxford Review of Education 39(1):4–16, 2013) together with a framework for analyzing efforts toward social justice in education (Cazden, International Journal of Educational Psychology 1(3):178–198, 2012) and fairer assessment make the case for more alternative assessment practices in recognition of the need for teachers’ pedagogic practice to cater for increased diversity.

An exploratory study of the current assessment practices for improving the learning of English as a foreign language (EFL) in two Vietnamese universities

This case study investigated EFL assessment practices at one public and one private university to explore the support of assessment for English language learning (ELL) within the Vietnamese sociocultural context. Findings demonstrated the potential of assessment to engage students in learning; enhance their understanding of the learning objectives; and facilitate their learning reflection. Findings also identified strong influences of contextual factors such as teachers' language assessment literacy, high-stakes testing and institutional administrative policies on the practices of assessment for ELL. This study contributes to research on Assessment for Learning and EFL education at tertiary level in Vietnam and other similar sociocultural contexts.

Written numeracy assessment in the early years: The challenges of pronouns and noun groups

We examine the challenges of pronouns and noun groups in the Australian Curriculum, Assessment and Reporting Authority’s (ACARA) Year 3 National Assessment Program - Literacy and Numeracy (NAPLAN) Numeracy Example Test (ACARA, 2015a). Framed by discourses of Literacy as a General Capability (ACARA, 2015b) in the Australian Curriculum: Mathematics (ACARA, 2015c), the pronouns and noun groups used in the Example Test are mapped onto the Australian Curriculum: English Content Descriptions (ACARA, 2015d). The findings are that some of the pronouns and noun groups used in the Example Test are more complex than what Year 3 children are expected to achieve in the Australian Curriculum: English. We thus stress the need for early years teachers of mathematics to account for the unique mathematical register of written mathematics problems so that young children are not only prepared for high-stakes written numeracy assessments, but are also enabled as a numerically literate citizen.

Children’s perceptions of primary science assessment in England and Wales

This study builds on and contributes to work on assessment of children in primary school, particularly in science. Previous research has examined primary science assessment from different standpoints, but no studies have speci?cally addressed children’s perspectives. This article provides additional insight into issues surrounding children’s assessment in primary school and how the assessment of science might develop in England after the science SATs (Standard Assessment Tests) were abolished in 2009. Some research suggests that primary science assessment via SATs is a major reason for the observed decline in children’s engagement with science in upper primary and lower secondary school. The analytic focus on engaging children as coresearchers to assist in the process of gathering informed views and interpreting ?ndings from a large sample of children’s views enables another contribution. The study, based on a survey of 1000 children in primary and secondary schools in England and Wales, reveals that despite being assessed under two different regimes (high-stakes national tests in England and moderated teacher assessment in Wales), children’s views of science assessment are remarkably consistent. Most appreciate the usefulness of science assessment and value frequent, non-SATs testing for monitoring/improving science progress. There was a largely negative impact, however, of science
assessment on children’s well-being, particularly due to stress. The paper demonstrates that children provide an important perspective on assessment and that including their views can improve policy-making in relation to primary science assessment.

Approaches to learning across cultures : The role of assessment

In the context of a broader research study on the intercultural understanding of teachers in Australia, Japan and Thailand, this paper focuses on approaches to learning and the role of assessment in shaping such approaches. Popular contrasts portray Asian learners as compliant and favouring rote memorisation and Western learners as independent and favouring deep, conceptual learning. Yet Asian students frequently outperform their Western counterparts in competitive tests purported to measure higher cognitive skills. Biggs and his associates have challenged the stereotypical view of Asian students as rote learners as a Western misperception. But data from the present cross-cultural study suggest it is more than a Western misperception, being shared by teachers in Japan and Thailand. With this background, this paper then explores the role of assessment through an analysis of examination papers in the three countries at the high stakes, year 12 level. This analysis of the ways in which knowledge and comprehension are assessed identifies different practices across cultures but not ones corresponding to the rhetoric on contrasting approaches to learning. Rather it concludes that assessment tasks classified superficially as comprehension can be approached through memorisation and conversely, those often classified as memorisation can require careful reading, thought and interpretation, while drawing from an extensive knowledge base. A shared understanding of the nature of assessment tasks in different cultures thus has the potential to dissolve the demarcation of culturally embedded learning styles and to enhance deep learning grounded in specialist knowledge for scholars, be they students or teachers, in all cultures.

Mobile texts and migrant audiences : rethinking literacy and assessment in a new media age

This article highlights the importance of rethinking literacy assessment in a digital and global world. Although pressures currently abound to narrow conceptions and practices of literacy, especially in an era of high stakes testing, digital multimodality and connectivity offer the potential for new ways of thinking, representing, and communicating, as well as new avenues for participating in relationships across social, geographic, and cultural difference. We explore the challenges of redesigning assessments so that they better take into account children's multiliterate capabilities. In so doing, we offer examples from our work in afterschool contexts that demonstrate how we have grappled with the complexities of assessment in new times.

Student knowledge: curriculum, assessment and reporting

At a time when national and international high-stakes testing has assumed such prominence, one might begin to wonder about the status of teacher judgement when assessing and reporting on children’s knowledge and skills against the descriptors specified in curriculum standards. Were standardised test results congruent with the judgements that teachers make when reporting on students’ achievement, concern about how one type of judgement might compare with another would perhaps be unwarranted. This article draws on research that has investigated whether standardised assessments in the state of Victoria, Australia are actually comparable with teacher’s judgements about their students’ work to illustrate that discrepancies do exist. These results have been interpreted within an analytical framework that derives from Aristotle’s (350BC/2000) distinction between three types of knowledge, namely epistemic, technical and phronetic knowledge.

FAIR_OSCE – Feedback structure for assessment of interactive roleplay in Objective Structured Clinical Exams

Introduction Since the quality of patient portrayal of standardized patients (SPs) during an Objective Structured Clinical Exam (OSCE) has a major impact on the reliability and validity of the exam, quality control should be initiated. Literature about quality control of SPs’ performance focuses on feedback [1, 2] or completion of checklists [3, 4]. Since we did not find a published instrument meeting our needs for the assessment of patient portrayal, we developed such an instrument after being inspired by others [5] and used it in our high-stakes exam. Project description SP trainers from five medical faculties collected and prioritized quality criteria for patient portrayal. Items were revised twice, based on experiences during OSCEs. The final instrument contains 14 criteria for acting (i.e. adequate verbal and non-verbal expression) and standardization (i.e. verbatim delivery of the first sentence). All partners used the instrument during a high-stakes OSCE. SPs and trainers were introduced to the instrument. The tool was used in training (more than 100 observations) and during the exam (more than 250 observations). Outcome High quality of SPs’ patient portrayal during the exam was documented. More than 90% of SP performances were rated to be completely correct or sufficient. An increase in quality of performance between training and exam was noted. For example, the rate of completely correct reaction in medical tests increased from 88% to 95%. Together with 4% of sufficient performances these 95% add up to 99% of the reactions in medical tests meeting the standards of the exam. SP educators using the instrument reported an augmentation of SPs’ performance induced by the use of the instrument. Disadvantages mentioned were the high concentration needed to observe all criteria and the cumbersome handling of the paper-based forms. Discussion We were able to document a very high quality of SP performance in our exam. The data also indicates that our training is effective. We believe that the high concentration needed using the instrument is well invested, considering the observed enhancement of performance. The development of an iPad-based application for the form is planned to address the cumbersome handling of the paper.

English language learners and standardized testing in an urban senior high school

The primary purpose of this study was to examine the influences of literacy variables on high-stakes test performance including: (a) student achievement on the Metropolitan Achievement Test, Seventh Edition (MAT-7) as correlated to the high-stakes test such as the FCAT examination and (b) the English language proficiency attained by English Language Learners (ELL) students when participating in, or exiting from English Speakers of Other Languages (ESOL) program as determined by the Limited English Proficient (LEP) committee. ^ Two one-sample Chi-square tests were conducted to investigate the relationship between passing the MAT-7 Reading and Language examinations and the FCAT-SSS Reading Comprehension and FCAT-NRT examinations. In addition, 2x2 Analyses of Variance (ANOVAs) were conducted to address the relationship between the time ELL students spent in the ESOL program and the level of achievement on MAT-7 Reading and Language examinations and the FCAT-SSS Reading Comprehension and FCAT-NRT. ^ Findings of this study indicated that more ELL students exit the program based on the LEP committee decisions than by passing the MAT-7. The majority of ELL students failed the 10th grade FCAT, the passing of which is needed for graduation. A significant number of ELL students failed, even when passing the MAT-7 or being duly exited through the decision of the LEP committee. The data also indicated that ELL students who exited the ESOL program in six semesters or fewer had higher FCAT scores than those who exited the program in seven semesters or more. The MAT-7 and the decision of the LEP committee were shown to be ineffective as predictors of success on the FCAT. ^ Further research to determine the length of time a student in the ESOL program uses English to read, write, and speak should be conducted. Additionally, the development of a new assessment instrument to better predict student success should be considered. However, it should be noted that the results of this study are limited to the context in which it was conducted and does not warrant generalizations beyond that context. ^

Proving My Competency One Test at a Time: Internationally Educated Nurses and the Canadian English Language Benchmark Assessment for Nurses

Abstract Professional language assessment is a new concept that has great potential to benefit Internationally Educated Professionals and the communities they serve. This thesis reports on a qualitative study that examined the responses of 16 Canadian English Language Benchmark Assessment for Nurses (CELBAN) test-takers on the topic of their perceptions of the CELBAN test-taking experience in Ontario in the winter of 2015. An Ontario organization involved in registering participants distributed an e-mail through their listserv. Thematic analyses of focus group and interview transcripts identified 7 themes from the data. These themes were used to inform conclusions to the following questions: (1) How do IENs characterize their assessment experience? (2) How do IENs describe the testing constructs measured by the CELBAN? (3) What, if any, potential sources of construct irrelevant variance (CIV) do the test-takers describe based on their assessment experience? (4) Do IENs feel that the CELBAN tasks provide a good reflection of the types of communicative tasks required of a nurse? Overall, participants reported positive experiences with the CELBAN as an assessment of their language skills, and noted some instances in which they felt some factors external to the assessment impacted their demonstration of their knowledge and skill. Lastly, some test-takers noted the challenge of completing the CELBAN where the types of communicative nursing tasks included in the assessment differed from nursing tasks typical of an IENs country or origin. The findings are discussed in relation to literature on high-stakes large-scale assessment and IEPs, and a set of recommendations are offered to future CELBAN administration. These recommendations include (1) the provision of a webpage listing all licensure requirements (2) monitoring of CELBAN location and dates in relation to the wider certification timeline for applicants (3) The provision of additional CELBAN preparatory materials (4) Minor changes to the CELBAN administrative protocols. Given that the CELBAN is a relatively new assessment format and its widespread use for high-stakes decisions (a component of nursing certification and licensure), research validating IEN-test-taker responses to construct representation and construct irrelevant variance is critical to our understanding of the role of competency testing for IENs.