Mandated literacy assessment and the reorganisation of teachers’ work : federal policy, local effects

This paper explores how mandated literacy assessment is reorganising teachers’ work in the context of Australia’s National Assessment Program – Literacy and Numeracy (NAPLAN), which was implemented in 2008. Students in Years 3, 5, 7 and 9 are tested annually, with school results publicly available. The wider policy context and the emergence of different forms of interconnected educational work associated with the testing phenomenon are described. Taking an Institutional Ethnography approach, the local effects of the federal policy regime are examined through a case study of one school. What mandated literacy assessment does to educators’ work in a culturally diverse low socioeconomic school community is discussed. Key themes include strategic exclusions of students from the testing process, appropriations and adaptations of literacy theory, work intensification, and ethical mediation of results. Questions concerning equity are raised about the differential effects of policy in different school contexts.

Literacy assessment that counts : mediating, interpreting and contesting translocal policy in a primary school

In Australia, as in many western education systems over the last two decades, discourses of accountability and performativity have reshaped education policy that has in turn reorganised the work of school leaders and teachers. One of the effects of this reorganisation is increased attention to the production, analysis and display of student achievement data. In this paper we examine in detail a sequence of the production and reading of literacy assessment data in a small Catholic school. Our analysis uses institutional ethnography’s concept of the ‘active text’, the text as occurring in a specific place and time even as it is articulated to social relations beyond its immediate context. Through this process we learn from those involved how their everyday work brings into being formalised, textually authorised processes in a local site that ensure the school meets accountability requirements while enabling teachers to resist standardisation of literacy teaching and assessment.

Focus group discussions of literacy assessment and world wide web-based reading intervention

Focus groups provide a means for participants in research to take on a greater role in the evaluation of the research and planning of its various stages. The role and outcomes of focus groups conducted with participants in a World Wide Web Project in literacy assessment and intervention are reported. Thirteen individuals with severe communication impairments participated in one of three focus groups. The groups were held after an assesment of reading skills and a trial period of intervention as part of the Web Project. The aim of the focus groups was to obtain feedback from participants about their involvement in the project and to discuss strategies for the next stage of the intervention. The focus groups offered a forum for participants to talk about what they did and did not like about the assessment and trial intervention. The discussions provided information about issues of both a practical and emotional nature that might have otherwise been unavailable to the researchers. Brainstorming of strategies provided valuable input for the next stage of the project and involved both the researchers and participants in a form of participatory action research.

The librarian is in: a consultation model for information literacy formative assessment

Formative assessment was introduced in Rehabilitation Therapy students’ information literacy programs in Fall Term 2006 in the course OT/PT 892: Evidence-Based Practice. It was subsequently employed in the Winter Term 2008 and again in the Spring Term 2008. Formative assessment during student/librarian face-to-face consultations was one of a variety of teaching techniques used in the program. Other techniques included: a required reading; an interactive hands-on searching session; and a summative assessment of the final revised search strategy assignment (these techniques varied somewhat over the 3 classes). With the 2008 entrance class, this course content moved to OT/PT 898: Critical Enquiry, largely in Module 3: Reviewing the Literature. One of the Critical Enquiry’s learning objectives is: “recognize and reflect on the complexity of gathering evidence to inform decision-making.”

Mobile texts and migrant audiences : rethinking literacy and assessment in a new media age

This article highlights the importance of rethinking literacy assessment in a digital and global world. Although pressures currently abound to narrow conceptions and practices of literacy, especially in an era of high stakes testing, digital multimodality and connectivity offer the potential for new ways of thinking, representing, and communicating, as well as new avenues for participating in relationships across social, geographic, and cultural difference. We explore the challenges of redesigning assessments so that they better take into account children's multiliterate capabilities. In so doing, we offer examples from our work in afterschool contexts that demonstrate how we have grappled with the complexities of assessment in new times.

Short Assessment of Health Literacy for Portuguese-Speaking Adults

OBJECTIVE: To develop and validate a short health literacy assessment tool for Portuguese-speaking adults. METHODS: The Short Assessment of Health Literacy for Portuguese-speaking Adults is an assessment tool which consists of 50 items that assess an individual's ability to correctly pronounce and understand common medical terms. We evaluated the instrument's psychometric properties in a convenience sample of 226 Brazilian older adults. Construct validity was assessed by correlating the tool scores with years of schooling, self-reported literacy, and global cognitive functioning. Discrimination validity was assessed by testing the tool's accuracy in detecting inadequate health literacy, defined as failure to fully understand standard medical prescriptions. RESULTS: Moderate to high correlations were found in the assessment of construct validity (Spearman's coefficients ranging from 0.63 to 0.76). The instrument showed adequate internal consistency (Cronbach's alpha=0.93) and adequate test-retest reliability (intraclass correlation coefficient=0.95). The area under the receiver operating characteristic curve for detection of inadequate health literacy was 0.82. A version consisting of 18 items was tested and showed similar psychometric properties. CONCLUSIONS: The instrument developed showed good validity and reliability in a sample of Brazilian older adults. It can be used in research and clinical settings for screening inadequate health literacy.

Short Assessment of Health Literacy for Portuguese-speaking Adults

OBJECTIVE: To develop and validate a short health literacy assessment tool for Portuguese-speaking adults. METHODS: The Short Assessment of Health Literacy for Portuguese-speaking Adults is an assessment tool which consists of 50 items that assess an individual's ability to correctly pronounce and understand common medical terms. We evaluated the instrument's psychometric properties in a convenience sample of 226 Brazilian older adults. Construct validity was assessed by correlating the tool scores with years of schooling, self-reported literacy, and global cognitive functioning. Discrimination validity was assessed by testing the tool's accuracy in detecting inadequate health literacy, defined as failure to fully understand standard medical prescriptions. RESULTS: Moderate to high correlations were found in the assessment of construct validity (Spearman's coefficients ranging from 0.63 to 0.76). The instrument showed adequate internal consistency (Cronbach's alpha=0.93) and adequate test-retest reliability (intraclass correlation coefficient=0.95). The area under the receiver operating characteristic curve for detection of inadequate health literacy was 0.82. A version consisting of 18 items was tested and showed similar psychometric properties. CONCLUSIONS: The instrument developed showed good validity and reliability in a sample of Brazilian older adults. It can be used in research and clinical settings for screening inadequate health literacy.

Assessment of markers of medication literacy in a US-Mexico border community

Each year an estimated 180,000 people in the United States (U.S.) die as a result of medication errors, now considered a major public health problem. If a patient cannot correctly act on information related to medication use or "Medication Literacy" there is an increased potential for error. Medication use issues are unique on the US-Mexico border because they include high rates of herbal products and medication products from Mexico as well as issues related to the preferred language (English or Spanish) of the patient. To evaluate the medication literacy in a US-Mexico border community, this retrospective study evaluates 180 subjects representing four diverse economic segments of a metropolitan US-Mexico Border community who have taken a Medication Literacy Assessment. The assessment tool has been created to understand how patients interpret medication information for prescription, over-the-counter, herbal, and Mexican medication product use, and how they problem-solve medication questions. The Medication Literacy Assessment tool specifically assesses document literacy (e.g., prescription labels), prose literacy (e.g., patient leaflets), numeracy (e.g., calculations and measurements) as well as qualitative data related to medication use practices. The main hypothesis of this study is that the ability to interpret and use medications will vary based on education, language (Spanish or English), and recruitment sites (economically diverse communities). The results will provide information to better characterize medication use in a primarily Hispanic population on the US-Mexico border and may be used to influence policy decisions regarding prescription and over-the-counter product information.^

Renewing assessment practices: literacy teaching and learning in digital environments

Interpretations of “literacy” and approaches to literacy pedagogy and assessment are under renewal as meaning-making and learning are increasingly situated in digitized environments. While the implications of these shifts are in part technological, they are also relational, as students are increasingly positioned as interactive with participatory roles in self-knowledge and increased responsibility for their learning. However, while shifts are occurring in understandings of literacy and approaches to literacy pedagogy, the same cannot be said for the way in which assessments of digital literacies are undertaken. There is a lack of valid, reliable, and practical assessments of new literacies to inform and help students to become better prepared for study, work, and citizenship in digital environments. This article maps five characteristics of effective formative assessment in print-based classrooms with seven affordancesin digital learning and assessment to suggest an analytical framework for examining teacher and student assessment in digital environments. Drawing on data from a research project in which a team of teachers introduced a one-to-one computing program and worked to renew their literacy assessment practices, this article discusses how each of the seven affordances are enacted in the assessment practices in a years five and six primary school classroom. The findings from this research project show that educational technologies have the potential to enable new approaches to teaching, learning, and assessment that better align with the needs of twenty-first century literacy learners. The findings alsosupport approaches to formative assessment that value print and multimodality and engage students in more flexible and differentiated ways. They can enable teachers and students to be re-positioned as designers, knowledge producers, and collaborative learners. The seven affordances provide a framework that holds rich possibilities for teacher learning and planning as prompts to support reflection on formative assessment practices, critique habitual practices, and considernew opportunities.

A dog of a QCAT : collateral effects of mandated English assessment in the Torres Strait

This paper critiques a 2008 Queensland Studies Authority (QSA) assessment initiative known as Queensland Comparable Assessment Tasks, or QCATs. The rhetoric is that these centrally devised assessment tasks will provide information about how well students can apply what they know, understand and can do in different contexts (QSA, 2009). The QCATs are described as ‘authentic, performance-based assessment’ that involves a ‘meaningful problem’, ‘emphasises critical thinking and reasoning’ and ‘provides students with every opportunity to do their best work’ (QSA, 2009). From my viewpoint as a teacher, I detail my professional concerns with implementing the 2008 middle primary English QCAT in one case study Torres Strait Island community. Specifically I ask ‘QCATs: Comparable with what?’ and ‘QCATs: Whose authentic assessment?’. I predict the possible collateral effects of implementing this English assessment in this remote Indigenous community, concluding, rather than being an example of quality assessment, colloquially speaking, it is nothing more than a ‘dog’.

Social studies disciplinary knowledge: Tensions between state curriculum and national assessment requirements

In this chapter, we are particularly concerned with making visible the general principles underlying the transmission of Social Studies curriculum knowledge, and considering it in light of a high-stakes mandated national assessment task. Specifically, we draw on Bernstein’s theoretical concept of pedagogic models as a tool for analysing orientations to teaching and learning. We introduce a case in point from the Australian context: one state Social Studies curriculum vis-a-vis one part of the Year Three national assessment measure for reading. We use our findings to consider the implications for the disciplinary knowledge of Social Studies in the communities in which we are undertaking our respective Australian Research Council Linkage project work (Glasswell et al.; Woods et al.). We propose that Social Studies disciplinary knowledge is being constituted, in part, through power struggles between different agencies responsible for the production and relay of official forms of state curriculum and national literacy assessment. This is particularly the case when assessment instruments are used to compare and contrast school results in highly visible web based league tables (see, for example, http://myschoolaustralia.ning.com/).