Revisiting the impact of formative assessment opportunities in student learning

This project developed as a result of some inconclusive data from an investigation of whether a relationship existed between the use of formative assessment opportunities and performance, as measured by final grade. We were expecting to show our colleagues and students that use of formative assessment resources had the potential to improve performance of first year students. This first study, undertaken in semester 1 2002, indicated that there was no apparent relationship between the two, even though the students reported how useful they found the formative assessment resources. This led us to ask if there was a transition effect such that students were not yet working in an independent way and making full use of the resources, and/or whether in order to see an effect we needed to persuade non-users of the resources to become users, before investigating if use can be correlated with improvement in performance. With the 2002-3 NextEd ASCILITE Research Grant we set out to repeat our project and to look at use and usefulness of resources in both first and second semester, to encourage non-users to become users and to investigate the relationship between use and performance. Now our story has a different ending.

Being a female PE teacher in an all-boys' school

The relationship between physical education (PE) and masculinity has been widely considered.  Since its inception in the Australian school curriculum around the turn of the 20th Century, PE has a history of privileging the so-called masculine desires to be active and compete.  In the contemporary climate where the educational performance of boys has become a considerable source of social anxiety, the literature points to factors such as a lack of male role models, the feminisation of curriculum and the lack of 'boy friendly' pedagogies as key factors.  This backdrop poses some interesting questions about the contributions that female PE teachers make to the development of contemporary masculinity.  Drawing on a narrative research methodology, this research set out to explore the experiences of female PE teachers working on all-boys' schools.

Assessment in experiential learning : the case of a public policy internship

Assessment is a significant issue for learning in the workplace. In some professions there are key indicators of success shared by workplace and academic supervisors alike. Beyond specific professions, however, assessment becomes more diffuse in workplaces that do not have explicit criteria established to judge performance of students in experiential learning. Assessing learning in these workplaces may be associated with methods that rely more upon student self appraisal and workplace supervisor reports. This article reports on the approach used for assessment in a public policy internship program in one Australian university - Deakin University in Victoria. The article argues that assessment, rather than being an add-on or a test of pre-ordained information, is central to the process of learning itself. This means that before students embark upon a policy internship they need to build their critical thinking abilities; i.e. a process of purposeful, self- regulatory judgment. Secondly they need to discuss how to negotiate their tasks in different workplaces and how to produce the criteria to be used in their evaluation.

Australian law students' perceptions of their values: interim results in the first year--2001--of a three--year empirical assessment

This study attempts to answer an essential social policy question: what values are characteristic of the mass of Australian lawyers' in their last year of law school and their early careers, and how do these values develop or degrade over time? This question is important because of the concern felt in the community as to the activities of lawyers. In recent years the Australian legal profession has sustained more scrutiny by governments, regulators and consumer movements than in any previous period of our history. The perception that practitioners' competencies and ethics are deficient and materially linked both to reduced standards of performance and to higher levels of public complaints, has received attention from academics, law societies, parliamentary committees and the Australian Competition and Consumer Commission.

Changing places : testing the relevance of educational research and teacher education

Frequent reviews of teacher education in recent times seem to indicate that those charged with the responsibility for teacher education have little understanding of the contemporary needs of teachers. Just how does educational research inform teachers in their day-to-day practice and how is its relevance tested? As an educational researcher Professor Shirley Grundy challenges her own notions of 'relevance' by returning from academia to the school system in Western Australia. This experience expanded her understanding of how conceptual frameworks and the realities of schools complemented each other. She reflects on the agenda for change faced by education systems and the ensuing transformation of curriculum planning, pedagogy, assessment and reporting procedures in some schools while not in others.

Her reference to Habermas' work relating to 'system' and 'lifeworld' provided her with an explanation for the social conditions of schooling, while her exploration of discourse theory provides some understanding of the practices related to the exercise of power in school settings. Interchanges such as that experienced by Grundy are to the mutual benefit of both parties.

Teacher or trainer? The shaping of teacher identity in adult education contexts

In response to changing government funding priorities there has been a shift away from the provision of needs based language and personal development courses for adults in community based contexts towards the delivery of vocational education. Much vocational education is characterized by competency-based curriculum and outcomes influenced by the needs of the current labour market as well as economic driven initiatives such as competitive tendering for short-term course funding. These trends have resulted in changes to the nature of curriculum, assessment, and the purpose and nature of the delivery of courses to adult learners. In turn, these changes affect the ways in which teachers see themselves and carry out their roles as professionals. This paper explores the ways in which the current discourses of vocational education shape teacher identities across a variety of vocational education contexts and the ways in which teacher identities are played out through training and teaching practices.

Issues arising from the implementation of a comprehensive assessment approach in a large, multi-modal Bachelor of Commerce Unit

Recent literature in higher education argues university assessment has been too narrow and hasn’t adequately reflected the quality, breadth and depth of students’ learning. Research shows students often prioritise and learn what they need to know for formal, graded assessment and disregard other academic content seen as less relevant to those requirements. The predominance of essays and examinations has therefore tended to constrain learning. The case for a more comprehensive approach has been clearly articulated. So what happens when staff take up the unique challenge of designing fair and uniform assessment for a large, core, multi-modal, multi-campus unit offered nationally and internationally?
When developing an undergraduate Bachelor of Commerce unit at Deakin University, staff considered the most appropriate ways to assess a range of conceptual understandings and communication skills. This resulted in the mapping and adoption of a comprehensive approach incorporating teacher, peer, and self-assessment aspects, individual and group work, oral and written presentations, and the use of portfolios and journals. Particular practices were adopted to control workloads, ensure fairness in marking, and overcome some problems generally associated with group work. When implementing the approach, practical issues arose that demanded adjustments. This paper details the approach taken, outlines research activities, and discusses the practical implications of issues that arose.

The reliability of performance of healthy adults on three visual perception tests

Problems with visual perceptual skills have been shown to have a negative impact on the daily living skills of individuals and are, therefore, commonly assessed by occupational therapists. The purpose of this study was to examine two types of reliability (internal consistency and test-retest reliability) of three newly revised or developed adult visual perception tests. The participants were 50 healthy adults, aged 18 to 55 years, from Melbourne, Victoria, Australia. The participants completed the Developmental Test of Visual Perception - Adolescent and Adult (DTVP-A), the Motor-Free Visual Perception Test - Third Edition (MVPT-3) and the Test of Visual Perceptual Skills (non-motor) - Third Edition (TVPS-3). Internal consistency was examined using Cronbach's alpha calculations and test-retest reliability was analysed using Spearman rho non-parametric correlation coefficients.

The results indicated that the DTVP-A, the MVPT-3 and the TVPS-3 had total scale internal consistency correlation scores of 0.60 or higher (0.60, 0.69 and 0.63 respectively). The majority of the subscales of each test had lower correlation coefficients than the overall scores (ranging from 0.22 to 0.49). For the DTVP-A, MVPT-3 and TVPS-3 total scale scores, the test-retest reliability correlation coefficients were statistically significant (rho = 0.46, p<0.05; rho = 0.62, p<0.01; and rho = 0.59, p<0.01, respectively). Overall, the three visual perceptual tests exhibited low to moderate levels of internal consistency and test-retest reliability.

Teacher education in a global context : towards a defensible theory of teacher education

This paper examines some of the effects of globalisation on education and teacher education. In particular it considers the contradictory demands of economic and cultural forms of globalisation, and between globalisation and localisation. Attempts to construct an 'education space' in Europe and Asia are considered and various responses of teacher education systems are outlined. A defensible theory of teacher education is presented around the transformation of curriculum, pedagogy, assessment and the practicum: one that might allow a creative response to the contradictions of globalisation.

The effects of time pressure and experience on nurses' risk assessment decisions : a signal detection analysis

Background: Time pressure and, occasionally, suboptimal assessment decisions are features of nursing in acute care.

Objectives: To explore the effect of generic and specialist clinical experience on the ability to detect the need to take action in acute care and the impact of time pressure on nurses' decision-making performance.

Methods: Experienced acute care registered nurses (n = 241) were presented with 50 vignettes of real clinical risk assessments. Each vignette contained seven information cues. In response to these vignettes, nurses had to decide whether to intervene or not. The 26 vignettes were time limited and mixed randomly into the 50 cases. Signal detection analysis was used to establish nurses' performance, personal decision thresholds ([beta]), and their abilities (d') to distinguish a signal of clinical risk from the clinical noise of noncontributory information.

Results: Nurses had significantly lower d' and were significantly less likely to indicate intervening under time pressure. For ability-but not threshold-there was a significant interaction of time pressure and years of experience in acute care. With no time pressure, d' increased in line with years of experience. Under time pressure, there was no effect.

Discussion: Time pressure reduced nurses' ability to detect the need and the tendency to report intervening. Thus, there were more failures to report appropriate intervention under time pressure, and the positive effects of clinical experience were negated under time pressure. More and larger scale research on the effect on clinical outcomes of time pressured nursing choices is required.

The relative and absolute reliability of two balance performance measures in chronic stroke patients

Purpose. To examine the relative reliability and absolute reliability of the Berg Balance Scale (BBS) and the Postural Assessment Scale for Stroke Patients (PASS) in chronic stroke patients. Method. A total of 52 mild to moderate stroke patients, who had a stroke more than 6 months previously, participated in the study. Both balance measures were administered twice, seven days apart, to the patients. A relative reliability index (intra-class correlation coefficient, ICC2,1) was used to examine the level of agreement between test and retest. Absolute reliability indices, including the Bland and Altman method, the standard error of measurement (SEM), and the smallest real differences (SRD), were used to define the extent to which a balance score varies on test-retest measurements. Results. Test-retest agreements were high (ICC2,1: BBS = 0.98; PASS = 0.97), indicating excellent agreement from a relative perspective. The SEM of the BBS and PASS, representing the smallest change threshold that indicates a real improvement for a group of individuals, were 2.4 and 1.1, respectively. The SRD of the BBS and PASS were 6.7 and 3.2, respectively, exhibiting the smallest change threshold that indicates a real improvement for a single individual. Conclusions. The test-retest agreements of the BBS and PASS were high in mild to moderate chronic stroke patients. The thresholds of both measures to detect real change are acceptable in research and clinical settings.

The use of scoring rubrics to determine clinical performance in the operating suite

Summary : This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite.

Method : Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College of Operating Room Nurses Standard NR4, 2004. ACORN Competency Standards for Perioperative Nurses: Standard NR4: The Instrument Nurse in the Perioperative Environment. Australian College of Operating Room Nurses Ltd, Adelaide] and underpinned by the Dreyfus model (1981). Three video clips that captured varying performance of nurses performing as instrument nurses in the operating suite were recorded and used as prompts by expert raters, who judged the performance using the rubrics.

Results : The study found that the holistic rubrics led to more consistent judgments than the analytical rubrics, yet the latter provided more diagnostic information for intervention purposes. Despite less consistency, the Analytical Observation Form had sufficient construct validity to satisfy the requirements of criterion referencing as determined by the Item Separation Index (Rasch, 1960), including high internal consistency and greater inter-rater reliability when average ratings were used.

Conclusion : The study was an empirical investigation of the use of concomitant Analytical and Holistic Rubrics to determine various levels of performance in the operating suite including inter-rater reliability. The methodology chosen was theoretically sound and sufficiently flexible to be used to develop other competencies within the operating suite.

Parsimonious and efficient assessment of health-related quality of life in osteoarthritis research: validation of the Assessment of Quality of Life (AQoL) instrument

Background: The Assessment of Quality of Life (AQoL) utility instrument was psychometrically developed for the general population. This study aimed to explore its potential as an osteoarthritis (OA) outcome measure.

Methods: WOMAC, Lequesne index, SF-36, Visual analogue scales and the AQoL were administered to 222 people with OA. The ability of each questionnaire to detect differences between groups was based on (i) self-rated health (SRH) and, (ii) differences between people on an orthopedic waiting list (WL) vs people with OA in the community (C). Comparisons included effect size, relative efficiency and receiver operator characteristic curves.

Results: All instruments detected differences between groups; however no one instrument exhibited superior efficiency. The AQoL demonstrated strong psychometric properties.

Conclusion: The AQoL has equivalent performance to comparator questionnaires commonly used in OA research and would be a useful adjunct to well-established disease specific scales. The AQoL has important advantages; brevity (12 items), facilitates comparisons between disease groups, and delivers a utility score that can be used in health economic evaluations.

Reconfiguring the spatial in the pre-service classroom

Teaching and researching through art opens multiple, distinct and/or overlapping understandings with positionings that shift according to the angle of repose of the percipient. The performance of teaching includes both technical skills and professional knowledge that pervade the literature of teacher education and teacher learning. These are strongly linked to the weight of attention teachers give to standards and assessment. Teaching, however, is more than this twodimensional technicist construction. Seventy years ago Dewey (1934) identified the distinction between an assessment based education with narrowly defined objectives, and active, open ended constructivist approaches best exemplified in the arts.

Primary school teacher perceived self-efficacy to teach fundamental motor skills

Fundamental Movement Skills (FMS) are a part of the school curricula, yet many Australian primary-age children are not mastering FMS. One reason may be a lack of perceived self-efficacy of primary teachers to teach FMS. This study investigated the level of perceived self-efficacy of primary school teachers to teach FMS in Victoria, Australia. A cross-sectional survey, based on the Victorian Institute of Teaching Standards of Professional Practice, was used to sample sixty-five pre-service and forty-six in-service teachers. Most primary school teachers were self-efficacious in teaching FMS (67.59 per cent); almost one-third (32.41 per cent) were not. Male teachers had higher perceived self-efficacy than female teachers, and a positive relationship was found between perceived self-efficacy to teach FMS and interest in, and participation in, physical activity (r = 0.52 and r = 0.31 respectively). Implications for practice include providing FMS teaching resources and professional training. Further research should explore the effect of perceived self-efficacy on teaching performance.