X-ray your data with Rasch

By using the Rasch model, much detailed diagnostic information is available to developers of survey and assessment instruments and to the researchers who use them. We outline an approach to the analysis of data obtained from the administration of survey instruments that can enable researchers to recognise and diagnose difficulties with those instruments and then to suggest remedial actions that can improve the measurement properties of the scales included in questionnaires. We illustrate the approach using examples drawn from recent research and demonstrate how the approach can be used to generate figures that make the results of Rasch analyses accessible to non-specialists.

Connecting pre-service teachers to 'real world' teaching practices through microteaching

Action research proved a useful strategy for monitoring the evolution of microteaching task as an authentic assessment for post-graduate pre-service teachers. Through four iterations of continually reflecting on the structure, purpose and outcomes of utilising microteaching as assessment, unit coordinators implemented an authentic assessment task that simulated real world experience.

Redefining the student-teacher relationship through online sharing activities

The student-teacher relationship should be a critical factor for successful teaching and learning in design education. In tradition, the relationship is defined as a master-apprentice, so design teachers’ visual assessment capability and technical standards significantly affect students’ quality of learning and achievements. However, there are some negative aspects of the master-apprentice relationship in design education that it may restrict student experiences to cultural diversity and interdisciplinary learning through various interactions with other students. A visual design subject was designed to adapt a new learning method that is to share students’ work and assessment through an asynchronous communication tool. This method was expected to reduce the negative aspects of the master-apprentice relationship and enhance peer-to-peer interactions and individualistic collaboration. A survey with two types of student groups in terms of their levels of participation was conducted to evaluate student experiences to this method. The outcomes implicate that online peer assessment is helpful to reduce the negative aspects of master-apprentice relation and can be useful for achieving the ultimate purpose of design education.

Developing authentic achievement for disengaged young people using an electronic portfolio system

Credentials are a salient form of cultural capital and if a student’s learning and productions are not assessed, they are invisible in current social systems of education and employment. In this field, invisible equals non-existent. This paper arises from the context of an alternative education institution where conventional educational assessment techniques currently fail to recognise the creativity and skills of a cohort of marginalised young people. In order to facilitate a new assessment model an electronic portfolio system (EPS) is being developed and trialled to capture evidence of students’ learning and their productions. In so doing a dynamic system of arranging, exhibiting, exploiting and disseminating assessment data in the form of coherent, meaningful and valuable reports will be maintained. The paper investigates the notion of assessing development of creative thinking and skills through the means of a computerised system that operates in an area described as the efield. A model of the efield is delineated and is explained as a zone existing within the internet where free users exploit the cloud and cultivate social and cultural capital. Drawing largely on sociocultural theory and Bourdieu’s concepts of field, habitus and capitals, the article positions the efield as a potentially productive instrument in assessment for learning practices. An important aspect of the dynamics of this instrument is the recognition of teachers as learners. This is seen as an integral factor in the sociocultural approach to assessment for learning practices that will be deployed with the EPS. What actually takes place is argued to be assessment for learning as a field of exchange. The model produced in this research is aimed at delivering visibility and recognition through an engaging instrument that will enhance the prospects of marginalised young people and shift the paradigm for assessment in a creative world.

How can flexible learning centres use social networking to improve their ongoing educational assessments?

Most online assessment systems now incorporate social networking features, and recent developments in social media spaces include protocols that allow the synchronisation and aggregation of data across multiple user profiles. In light of these advances and the concomitant fear of data sharing in secondary school education this papers provides important research findings about generic features of online social networking, which educators can use to make sound and efficient assessments in collaboration with their students and colleagues. This paper reports on a design experiment in flexible educational settings that challenges the dichotomous legacy of success and failure evident in many assessment activities for at-risk youth. Combining social networking practices with the sociology of education the paper proposes that assessment activities are best understood as a negotiable field of exchange. In this design experiment students, peers and educators engage in explicit, "front-end" assessment (Wyatt-Smith, 2008) to translate digital artefacts into institutional, and potentiality economic capital without continually referring to paper based pre-set criteria. This approach invites students and educators to use social networking functions to assess “work in progress” and final submissions in collaboration, and in doing so assessors refine their evaluative expertise and negotiate the value of student’s work from which new criteria can emerge. The mobile advantages of web-based technologies aggregate, externalise and democratise this transparent assessment model for most, if not all, student work that can be digitally represented.

Self-narratives of children : using digital journals

This paper takes the position that children are at risk of being marginalised when research methods are not tailored to their requirements. In particular, children who are negotiating early adolescence are presented as an ideal group for involvement with narrative research approaches that attempt to be flexible and creative. With the premise that the need to juggle multiple realities within complex societal structures is challenging and isolating for such children, narrative methods offer a promising mode of access to their individual realities. Children's own self-narratives in the form of email journal entries are proposed as research tools that can help to minimise issues arising from resistance to adults and problems of shared vocabulary that may occur using more traditional methods. Digital journaling, as a means of capturing self-narratives, can provide a convenient space for young people to generate and share their own personal accounts of their lives and their experiences that can also serve to inform others. Guidelines are offered for how to manage a journaling project that is not reliant on children's physical presence within school settings. Digital journals are thus described as multi-function mechanisms that can support personal growth as well as promote shared understandings and social fairness between adults and children.

Testing for accountability

Current graduates in education are entering a very different profession to the one in which most of their “baby-boomer” colleagues started. It is a profession in which accountability and national high-stakes testing (e.g. NAPLAN) have become catch-cries, and where the interpretation and use of educational data is an additional challenge. This has led to schools focusing on performance, and teachers now have to analyse test data and apply the findings to their teaching.

Building bridges to participation : sociocultural case studies of AfL

Assessment for Learning(AfL) case studies in a North Queensland school highlight the significance of the teacher-student relationship in creating a supportive culture within which students can negotiate new learner identities. AfL practices are school based evaluative practices that occur within the regular flow of teaching and learning with the purpose of informing and improving student learning to enhance learner autonomy. The identity of an autonomous learner is socially negotiated through participation in the community of practice of the class. Underpinned by a sociocultural perspective this research shows how AfL is manifested in action in its complexity and how positive teacher-student interactions can build bridges for students to move towards full participation.

Assessing the capstone experience : conceptualising a capstone experience for law students

Assessment Principle -- -- The capstone experience should include assessment that: 1: Enables students to apply their knowledge skills and capabilities in an authentic context ; 2: Tests whether or not students are able to apply knowledge skills and capabilities in unfamiliar contexts ; 3: Incorporates feedback from a multitude of sources including peers and self‐reflection to enable students to become self‐reliant and to exercise their own professional judgment ; 4: Recognises the culminating nature of the capstone experience.

Teamwork protocol

This protocol represents an attempt to assist in the instruction of teamwork assessment for first-year students across QUT. We anticipate that teaching staff will view this protocol as a generic resource in teamwork instruction, processes and evaluation. Teamwork has been acknowledged as a problematic practice at QUT while existing predominantly in importance amongst graduate capabilities for all students at this institution. This protocol is not an extensive document on the complexities and dynamics of teamwork processes, but instead presents itself as a set of best practice guidelines and recommendations to assist in team design, development, management, support and assessment. It is recommended that this protocol be progressively implemented across QUT, not only to attain teamwork teaching consistency, but to address and deal with the misconceptions and conflict around the importance of the teamwork experience. The authors acknowledge the extensive input and contributions from a Teamwork Steering Committee selected from academic staff and administrative members across the institution. As well, we welcome feedback and suggestions to both fine tune and make inclusive those strategies that staff believe add to optimal teamwork outcomes.

Assessing students in senior science: an analysis of questions in contextualised chemistry exams

This study explores the development of a coding system for analysing test questions in two context-based chemistry exams. We describe our unique analytical procedures before contrasting the data from both tests. Our findings indicate that when a new curriculum is developed such as a context-based curriculum, teachers are required to combine the previously separate domains of context and concept to develop contextualised assessment. We argue that constructing contextualised assessment items requires teachers to view concepts and context as interconnected rather than as separate entities that may polarise scientific endeavour. Implications for practice, curriculum and assessment-development in context-based courses are proposed.

An investigation into online moderation

The purpose of this paper is to understand how teachers’ identity as an assessor in a standards-referenced assessment system may be developed through their participation in online social moderation meetings. In these meetings teachers negotiate and share their understandings of assessment standards and judgement decisions. In particular, the paper focuses on the relationship between the technology, the moderation processes and teachers’ development in this assessment system. This paper draws on sociocultural theories of learning to analyse the qualitative data collected through observations of eleven online moderation meetings and interviews of the teachers involved in these meetings. The results provide insights into the mediating role of the technology with regard to teachers’ development of shared meanings and common practices within a standards-referenced assessment system.

Towards an understanding of teacher judgement in the context of social moderation

Social moderation involves teachers gathering together to discuss their judgements of the quality of student work and to reach agreement regarding the standard awarded. This qualitative study conducted over a three-year period investigated the social practice of moderation and the influence on teachers’ judgements of students work. An initial survey of teachers’ understandings of moderation and standards, pre-interviews of teachers who participated in the moderation meetings, observations of these meetings with a particular focus on one teacher (focus teachers) comprised the data collection methods. Data analysis involved organising, matching, coding, identifying patterns and themes using a constant comparative method. Socio-cultural theories of learning and assessment underpinned the approach to data analysis and proved helpful in explaining the diverse influences on teachers’ judgements beyond the task criteria, and the progressive development of shared understandings through engaging in professional discussions of students’ work. The study revealed that the process is not clear and linear and is influenced by factors such as the representation of the standards and the knowledge base of the teachers.

The impact of high stakes testing: the Australian story

High stakes testing in Australia was introduced in 2008 by way of the National Assessment Program – Literacy and Numeracy (NAPLAN). Currently, every year all students in Years 3, 5, 7 and 9 are assessed on the same days using national tests in Reading, Writing, Language Conventions (Spelling, Grammar and Punctuation) and Numeracy. In 2010 the NAPLAN results were published on the Federal Government MySchool website. The impact of these high stakes tests on jurisdictions, school principals, parents and students is considered in this article. We draw on reported observations from the Australian Primary Principals Association during 2009–10 testing periods across the country and published Australian research on the impact of high stakes literacy and numeracy testing. We also examine alternative approaches that include the use of assessment evidence for learning improvement purposes and for accountability purposes. In considering alternatives to the current large-scale testing approach we draw on key insights from research on teacher judgement, achievement standards and social moderation in the context of national curriculum and assessment reform in support of the suggested directions forward.

The development of the graphics-decoding proficiency instrument

The Graphics-Decoding Proficiency (G-DP) instrument was developed as a screening test for the purpose of measuring students’ (aged 8-11 years) capacity to solve graphics-based mathematics tasks. These tasks include number lines, column graphs, maps and pie charts. The instrument was developed within a theoretical framework which highlights the various types of information graphics commonly presented to students in large-scale national and international assessments. The instrument provides researchers, classroom teachers and test designers with an assessment tool which measures students’ graphics decoding proficiency across and within five broad categories of information graphics. The instrument has implications for a number of stakeholders in an era where graphics have become an increasingly important way of representing information.