Mandated literacy assessment and the reorganisation of teachers’ work : federal policy, local effects

This paper explores how mandated literacy assessment is reorganising teachers’ work in the context of Australia’s National Assessment Program – Literacy and Numeracy (NAPLAN), which was implemented in 2008. Students in Years 3, 5, 7 and 9 are tested annually, with school results publicly available. The wider policy context and the emergence of different forms of interconnected educational work associated with the testing phenomenon are described. Taking an Institutional Ethnography approach, the local effects of the federal policy regime are examined through a case study of one school. What mandated literacy assessment does to educators’ work in a culturally diverse low socioeconomic school community is discussed. Key themes include strategic exclusions of students from the testing process, appropriations and adaptations of literacy theory, work intensification, and ethical mediation of results. Questions concerning equity are raised about the differential effects of policy in different school contexts.

Literacy assessment that counts : mediating, interpreting and contesting translocal policy in a primary school

In Australia, as in many western education systems over the last two decades, discourses of accountability and performativity have reshaped education policy that has in turn reorganised the work of school leaders and teachers. One of the effects of this reorganisation is increased attention to the production, analysis and display of student achievement data. In this paper we examine in detail a sequence of the production and reading of literacy assessment data in a small Catholic school. Our analysis uses institutional ethnography’s concept of the ‘active text’, the text as occurring in a specific place and time even as it is articulated to social relations beyond its immediate context. Through this process we learn from those involved how their everyday work brings into being formalised, textually authorised processes in a local site that ensure the school meets accountability requirements while enabling teachers to resist standardisation of literacy teaching and assessment.

The librarian is in: a consultation model for information literacy formative assessment

Formative assessment was introduced in Rehabilitation Therapy students’ information literacy programs in Fall Term 2006 in the course OT/PT 892: Evidence-Based Practice. It was subsequently employed in the Winter Term 2008 and again in the Spring Term 2008. Formative assessment during student/librarian face-to-face consultations was one of a variety of teaching techniques used in the program. Other techniques included: a required reading; an interactive hands-on searching session; and a summative assessment of the final revised search strategy assignment (these techniques varied somewhat over the 3 classes). With the 2008 entrance class, this course content moved to OT/PT 898: Critical Enquiry, largely in Module 3: Reviewing the Literature. One of the Critical Enquiry’s learning objectives is: “recognize and reflect on the complexity of gathering evidence to inform decision-making.”

Short Assessment of Health Literacy for Portuguese-Speaking Adults

OBJECTIVE: To develop and validate a short health literacy assessment tool for Portuguese-speaking adults. METHODS: The Short Assessment of Health Literacy for Portuguese-speaking Adults is an assessment tool which consists of 50 items that assess an individual's ability to correctly pronounce and understand common medical terms. We evaluated the instrument's psychometric properties in a convenience sample of 226 Brazilian older adults. Construct validity was assessed by correlating the tool scores with years of schooling, self-reported literacy, and global cognitive functioning. Discrimination validity was assessed by testing the tool's accuracy in detecting inadequate health literacy, defined as failure to fully understand standard medical prescriptions. RESULTS: Moderate to high correlations were found in the assessment of construct validity (Spearman's coefficients ranging from 0.63 to 0.76). The instrument showed adequate internal consistency (Cronbach's alpha=0.93) and adequate test-retest reliability (intraclass correlation coefficient=0.95). The area under the receiver operating characteristic curve for detection of inadequate health literacy was 0.82. A version consisting of 18 items was tested and showed similar psychometric properties. CONCLUSIONS: The instrument developed showed good validity and reliability in a sample of Brazilian older adults. It can be used in research and clinical settings for screening inadequate health literacy.

Short Assessment of Health Literacy for Portuguese-speaking Adults

OBJECTIVE: To develop and validate a short health literacy assessment tool for Portuguese-speaking adults. METHODS: The Short Assessment of Health Literacy for Portuguese-speaking Adults is an assessment tool which consists of 50 items that assess an individual's ability to correctly pronounce and understand common medical terms. We evaluated the instrument's psychometric properties in a convenience sample of 226 Brazilian older adults. Construct validity was assessed by correlating the tool scores with years of schooling, self-reported literacy, and global cognitive functioning. Discrimination validity was assessed by testing the tool's accuracy in detecting inadequate health literacy, defined as failure to fully understand standard medical prescriptions. RESULTS: Moderate to high correlations were found in the assessment of construct validity (Spearman's coefficients ranging from 0.63 to 0.76). The instrument showed adequate internal consistency (Cronbach's alpha=0.93) and adequate test-retest reliability (intraclass correlation coefficient=0.95). The area under the receiver operating characteristic curve for detection of inadequate health literacy was 0.82. A version consisting of 18 items was tested and showed similar psychometric properties. CONCLUSIONS: The instrument developed showed good validity and reliability in a sample of Brazilian older adults. It can be used in research and clinical settings for screening inadequate health literacy.

Assessment of markers of medication literacy in a US-Mexico border community

Each year an estimated 180,000 people in the United States (U.S.) die as a result of medication errors, now considered a major public health problem. If a patient cannot correctly act on information related to medication use or "Medication Literacy" there is an increased potential for error. Medication use issues are unique on the US-Mexico border because they include high rates of herbal products and medication products from Mexico as well as issues related to the preferred language (English or Spanish) of the patient. To evaluate the medication literacy in a US-Mexico border community, this retrospective study evaluates 180 subjects representing four diverse economic segments of a metropolitan US-Mexico Border community who have taken a Medication Literacy Assessment. The assessment tool has been created to understand how patients interpret medication information for prescription, over-the-counter, herbal, and Mexican medication product use, and how they problem-solve medication questions. The Medication Literacy Assessment tool specifically assesses document literacy (e.g., prescription labels), prose literacy (e.g., patient leaflets), numeracy (e.g., calculations and measurements) as well as qualitative data related to medication use practices. The main hypothesis of this study is that the ability to interpret and use medications will vary based on education, language (Spanish or English), and recruitment sites (economically diverse communities). The results will provide information to better characterize medication use in a primarily Hispanic population on the US-Mexico border and may be used to influence policy decisions regarding prescription and over-the-counter product information.^

A dog of a QCAT : collateral effects of mandated English assessment in the Torres Strait

This paper critiques a 2008 Queensland Studies Authority (QSA) assessment initiative known as Queensland Comparable Assessment Tasks, or QCATs. The rhetoric is that these centrally devised assessment tasks will provide information about how well students can apply what they know, understand and can do in different contexts (QSA, 2009). The QCATs are described as ‘authentic, performance-based assessment’ that involves a ‘meaningful problem’, ‘emphasises critical thinking and reasoning’ and ‘provides students with every opportunity to do their best work’ (QSA, 2009). From my viewpoint as a teacher, I detail my professional concerns with implementing the 2008 middle primary English QCAT in one case study Torres Strait Island community. Specifically I ask ‘QCATs: Comparable with what?’ and ‘QCATs: Whose authentic assessment?’. I predict the possible collateral effects of implementing this English assessment in this remote Indigenous community, concluding, rather than being an example of quality assessment, colloquially speaking, it is nothing more than a ‘dog’.

Social studies disciplinary knowledge: Tensions between state curriculum and national assessment requirements

In this chapter, we are particularly concerned with making visible the general principles underlying the transmission of Social Studies curriculum knowledge, and considering it in light of a high-stakes mandated national assessment task. Specifically, we draw on Bernstein’s theoretical concept of pedagogic models as a tool for analysing orientations to teaching and learning. We introduce a case in point from the Australian context: one state Social Studies curriculum vis-a-vis one part of the Year Three national assessment measure for reading. We use our findings to consider the implications for the disciplinary knowledge of Social Studies in the communities in which we are undertaking our respective Australian Research Council Linkage project work (Glasswell et al.; Woods et al.). We propose that Social Studies disciplinary knowledge is being constituted, in part, through power struggles between different agencies responsible for the production and relay of official forms of state curriculum and national literacy assessment. This is particularly the case when assessment instruments are used to compare and contrast school results in highly visible web based league tables (see, for example, http://myschoolaustralia.ning.com/).

Rethinking the literacy capabilities of pre-service primary teachers in testing times

As a group of committed literacy teacher educators from five universities across three Australian states, the authors bring professional critique to the problematic issue of what counts in current and possible future measures of pre-service teachers’ literacy capacity. In times when normalising models of literacy assessment ignore innovative developments in technologies, we provide an example of what is happening at the ‘chalk-face’ of literacy teacher education. This paper describes a study that demonstrates how responsible alignment of teacher accreditation requirements with a scholarly impetus to incorporate digital literacies to prepare pre-service teachers will help address changing educational needs and practices (AITSL 2012; Gillen & Barton 2010; Hattie 2003; Johnson, Smith, Willis, Levine & Haywood 2011; Klein 2006; Masny & Cole 2012; OECD 2011).

Designing Assessment for Quality Learning

- Considers broad-scale assessment approaches and how they impact on educational opportunity and outcomes. - Brings together internationally recognised scholars providing new insights into assessment for learning improvement and accountability. - Presents different theoretical and methodological perspectives influential in the field of assessment, learning and social change. - Contributes to the theorising of assessment in contexts characterised by heightened accountability requirements and constant change. This book brings together internationally recognised scholars with an interest in how to use the power of assessment to improve student learning and to engage with accountability priorities at both national and global levels. It includes distinguished writers who have worked together for some two decades to shift the assessment paradigm from a dominant focus on assessment as measurement towards assessment as central to efforts to improve learning. These writers have worked with the teaching profession and, in so doing, have researched and generated key insights into different ways of understanding assessment and its relationship to learning. The volume contributes to the theorising of assessment in contexts characterised by heightened accountability requirements and constant change. The book’s structure and content reflect already significant and growing international interest in assessment as contextualised practice, as well as theories of learning and teaching that underpin and drive particular assessment approaches. Learning theories and practices, assessment literacies, teachers’ responsibilities in assessment, the role of leadership, and assessment futures are the organisers within the book’s structure and content. The contributors to this book have in common the view that quality assessment, and quality learning and teaching are integrally related. Another shared view is that the alignment of assessment with curriculum, teaching and learning is linchpin to efforts to improve both learning opportunities and outcomes for all. Essentially, the book presents new perspectives on the enabling power of assessment. In so doing, the writers recognise that validity and reliability - the traditional canons of assessment – remain foundational and therefore necessary. However, they are not of themselves sufficient for quality education. The book argues that assessment needs to be radically reconsidered in the context of unprecedented societal change. Increasingly, communities are segregating more by wealth, with clear signs of social, political, economic and environmental instability. These changes raise important issues relating to ethics and equity, taken to be core dimensions in enabling the power of assessment to contribute to quality learning for all. This book offers readers new knowledge about how assessment can be used to re/engage learners across all phases of education.

The literacy curriculum: A critical review

This is a critical review of the scope of the literacy curriculum in the twenty-first century, uncovering the strengths, controversies, and silences that have divided literacy researchers and educators. It conceptualizes the literacy curriculum as a particular set of socially organized symbolic practices that are always selective, and which are inextricably connected to the function and organization of schooling. We trace the political, historical, and ideological antecedents of literacy curriculum and schooling as a form of cultural apparatus of the nation-state, before tracing some of the major interpretive paradigms that have influenced the shape of the literacy curriculum in many parts of the world. These include debates about skills-based approaches, whole language, systemic functional grammar, and critical literacy. It then draws attention to noteworthy advances and shifts in the field over recent decades: debates about the role of orality in the literacy curriculum, home-school community literacy practices, teacher and student knowledge of language and grammar, and the role of curriculum area literacies. It anticipates the future of the literacy curriculum in online textual environments and the changing sensorial and material nature of literacy practices, while acknowledging that curriculum innovation is always limited in complex ways by historically established pedagogic discourses of schooling.