Pachler N (2008) SCOPING A VISION FOR FORMATIVE E-ASSESSMENT - FEEDBACK Findings from the literature review

Pachler N (2008) SCOPING A VISION FOR FORMATIVE E-ASSESSMENT - FEEDBACK Findings from the literature review

Brand formative design - development and assessment of product design from a future, brand and consumer perspective

With this dissertation research we investigate intersections between design and marketing and in this respect, which factors do contribute that a product design becomes brand formative. We have developed a Brand Formative Design (BFD) framework, which investigates individual design features in a holistic, comparable, brand relevant, and consumer specific context. We discuss what kinds of characteristics contribute to BFD but also illuminate how they should be applied and examine: rnA holistic framework leading to Brand Formative Design. Identification and assessment of BFD Drivers. The dissection of products into three Distinctive Design Levels. The detection of surprising design preferences. The appropriate degree of scheme deviation with evolutionary design. Simulated BFD development processes with three different products and the integration of consumers. Future oriented objectification, comparability and assessment of design. Recommendations for the management of design in a brand specific context. Design is a product feature, which contributes significantly to the success of products. However, the development of new design contains challenges. Design can hardly be objectified; many people have an opinion concerning the attractiveness of new products but cannot formulate their future preferences. Product design is widely developed based on intuition, which can be difficult for the management of design. Here the concept of Brand Formative Design can provide a framework which contributes to structure, objectify, develop and assess new evolutionary design in brand and future relevant contexts, but also integrates consumers and their preferences without restricting creativity too much.

The Comprehensive Assessment of Leadership for Learning: A Next Generation Formative Evaluation and Feedback System

Recent developments in federal policy have prompted the creation of state evaluation frameworks for principals and teachers that hold educators accountable for effective practices and student outcomes. These changes have created a demand for formative evaluation instruments that reflect current accountability pressures and can be used by schools to focus school improvement and leadership development efforts. The Comprehensive Assessment of Leadership for Learning (CALL) is a next generation, 360-degree on-line assessment and feedback system that reflect best practices in feedback design. Some unique characteristics of CALL include a focus on: leadership distributed throughout the school rather than as carried out by an individual leader; assessment of leadership tasks rather than perceptions of leadership practice; a focus on larger complex systems of middle and high school; and transparency of assessment design. This paper describes research contributing to the design and validation of the CALL survey instrument.

Authentic and conventional assessment in Singapore schools : an empirical study of teacher assignments and student work

This is an empirical examination of the quality of teacher assignments and student work in Singapore schools. Using a theoretical framework based on principles of authentic assessment and intellectual quality, two sets of criteria and scoring rubrics were developed for the training of expert teachers to judge the quality of assignments and student work. Following rigorous training, the inter-rater reliability of expert teacher scoring was high. Samples of teacher assignments and student work were collected in English, social studies, mathematics, and science subject areas from a random stratified sample of 30 elementary schools and 29 high schools. For both grade levels, there were significant differences for the authentic intellectual quality of teachers’ assignments by subject area. Likewise, the differences of authentic intellectual quality for student work were significant and varied by subject area. Subject area effect was large. The correlations between the quality of teachers’ assignment tasks and student work were strong and significant at both grade levels. Where teachers set more intellectually demanding tasks, students were more likely to generate work or artefacts judged to be of higher quality. The findings suggest that teacher professional development in authentic intellectual assessment task design can contribute to the improvement of student learning and performance. It is argued that this will be a key requisite of educational systems like Singapore that are seeking to expand pedagogy and student outcomes beyond a focus on factual and rote knowledge.

Is implementing criterion-referenced assessment worth the effort with gen Y?

Well implemented criterion-referenced assessment (CRA) requires dedicated time and effort, especially in describing realistic expectations of evidence of achievement to students in the form of criteria sheets (or grading rubrics). It is also takes time out of delivering content to teach students how to judge their own work using criteria sheets. In 2007, to engage third year Microbiology students in using criteria sheets for the first time in their degree, we devised an innovative assessment tutorial supported by online resources. We were sceptical of much of the literature that reported ‘agreed’ characteristics of our predominantly gen Y cohort, because of the older ages of the majority of authors. These authors claim gen Y has a propensity for digital media, overconfidence in their own abilities and a collaborative orientation. We rejected this stereotype when developing the tutorial. Evaluations by students were positive and there was no dramatic change to grades for the unit. These results are similar to those in the literature for non gen Y cohorts. This lends support to our claim that giving students control over their own learning, irrespective of their generational label, is worth the time and effort.

Harnessing assessment and feedback in the first year to support learning success, engagement and retention

It has been argued that intentional first year curriculum design has a critical role to play in enhancing first year student engagement, success and retention (Kift, 2008). A fundamental first year curriculum objective should be to assist students to make the successful transition to assessment in higher education. Scott (2006) has identified that ‘relevant, consistent and integrated assessment … [with] prompt and constructive feedback’ are particularly relevant to student retention generally; while Nicol (2007) suggests that ‘lack of clarity regarding expectations in the first year, low levels of teacher feedback and poor motivation’ are key issues in the first year. At the very minimum, if we expect first year students to become independent and self-managing learners, they need to be supported in their early development and acquisition of tertiary assessment literacies (Orrell, 2005). Critical to this attainment is the necessity to alleviate early anxieties around assessment information, instructions, guidance, and performance. This includes, for example:  inducting students thoroughly into the academic languages and assessment genres they will encounter as the vehicles for evidencing learning success; and  making expectations about the quality of this evidence clear. Most importantly, students should receive regular formative feedback of their work early in their program of study to aid their learning and to provide information to both students and teachers on progress and achievement. Leveraging research conducted under an ALTC Senior Fellowship that has sought to articulate a research-based 'transition pedagogy' (Kift & Nelson, 2005) – a guiding philosophy for intentional first year curriculum design and support that carefully scaffolds and mediates the first year learning experience for contemporary heterogeneous cohorts – this paper will discuss theoretical and practical strategies and examples that should be of assistance in implementing good assessment and feedback practices across a range of disciplines in the first year.

Does the summative assessment of real world learning using criterion-referenced assessment need to be discipline specific?

This paper synthesises the existing literature on the contemporary conception of ‘real world’ and compares it with similar notions such as ‘authentic’ and ‘work integrated learning’. While the term ‘real world’ may be partly dependent on the discipline, it does not necessarily follow that the criterion-referenced assessment of ‘real world’ assessment must involve criteria and performance descriptors that are discipline specific. Two examples of summative assessment (court report and trial process exercise) from a final year core subject at the Queensland University of Technology, LWB432 Evidence, emphasise real world learning, are authentic, innovative and better prepare students for the transition into the workplace than more generic forms of assessment such as tutorial participation or oral presentations. The court report requires students to attend a criminal trial in a Queensland Court and complete a two page report on what they saw in practice compared with what they learned in the classroom. The trial process exercise is a 50 minute written closed book activity conducted in tutorials, where students plan questions that they would ask their witness in examination-in-chief, plan questions that they would ask their opponent’s witness in cross-examination, plan questions that they would ask in reexamination given what their opponent asked in cross-examination, and prepare written objections to their opponent’s questions. The trial process exercise simulates the real world, whereas the court report involves observing the real world, and both assessment items are important to the role of counsel. The design of the criterion-referenced assessment rubrics for the court report and trial process exercise is justified by the literature. Notably, the criteria and performance descriptors are not necessarily law specific and this paper highlights the parts that may be easily transferred to other disciplines.

An adaptable framework for the teaching and assessment of software development across year levels

Few frameworks exist for the teaching and assessment of programming subjects that are coherent and logical. Nor are they sufficiently generic and adaptable to be used outside the particular tertiary institutions in which they were developed. This paper presents the Teaching and Assessment of Software Development (TASD) frame-work. We describe its development and implementation at an Australian university and demonstrate, with examples, how it has been used, with supporting data. Extracts of criteria sheets (grading rubrics) for a variety of assessment tasks are included. The numerous advantages of this new framework are discussed with comparisons made to those reported in the published literature.

How do the learning outcomes of students compare in units with different levels of practical work? A comparative study about the relation between Problem-­Based Learning and assessment approaches

The literature supporting the notion that active, student-centered learning is superior to passive, teacher-centered instruction is encyclopedic (Bonwell & Eison, 1991; Bruning, Schraw, & Ronning, 1999; Haile, 1997a, 1997b, 1998; Johnson, Johnson, & Smith, 1999). Previous action research demonstrated that introducing a learning activity in class improved the learning outcomes of students (Mejias, 2010). People acquire knowledge and skills through practice and reflection, not by watching and listening to others telling them how to do something. In this context, this project aims to find more insights about the level of interactivity in the curriculum a class should have and its alignment with assessment so the intended learning outcomes (ILOs) are achieved. In this project, interactivity is implemented in the form of problem- based learning (PBL). I present the argument that a more continuous formative feedback when implemented with the correct amount of PBL stimulates student engagement bringing enormous benefits to student learning. Different levels of practical work (PBL) were implemented together with two different assessment approaches in two subjects. The outcomes were measured using qualitative and quantitative data to evaluate the levels of student engagement and satisfaction in the terms of ILOs.

Development and implementation of the Australian universities radiation therapy student clinical assessment form

Purpose: Prior to 2009, one of the problems faced by radiation therapists who supervised and assessed students on placement in Australian clinical centres, was that each of the six Australian universities where Radiation Therapy (RT) programmes were conducted used different clinical assessment and reporting criteria. This paper describes the development of a unified national clinical assessment and reporting form that was implemented nationally by all six universities in 2009. Methods: A four phase methodology was used to develop the new assessment form and user guide. Phase 1 included university consensus around domains of student practice and assessment, and alignment with national competency standards; Phase 2 was a national consensus workshop attended by radiation therapists involved in student supervision and assessment; Phase 3 was an action research re-iterative Delphi technique involving two rounds of a mail-out to gain further expert consensus; and stage 4 was national piloting of the developed assessment form. Results: The new assessment form includes five main domains of practice and 19 sub-domain criteria which students are assessed against during placement. Feedback from the pilot centre participants was positive, with the new form being assessed to be comprehensive and complemented by the accompanying user guide. Conclusion: The new assessment form has improved both the formative and summative assessment of students on placement, as well as enhancing the quality of feedback to students and the universities. The new national form has high acceptance from the Australian universities and has been subject to wide review by the profession.

Towards fairer assessment

Drawing on the largest Australian collection and analysis of empirical data on multiple facets of Aboriginal and Torres Strait Islander education in state schools to date, this article critically analyses the systemic push for standardized testing and improved scores, and argues for a greater balance of assessment types by providing alternative, inclusive, participatory approaches to student assessment. The evidence for this article derives from a major evaluation of the Stronger Smarter Learning Communities. The first large-scale picture of what is occurring in classroom assessment and pedagogy for Indigenous students is reported in this evaluation yet the focus in this article remains on the issue of fairness in student assessment. The argument presented calls for “a good balance between formative and summative assessment” (OECD, Synergies for Better Learning An International Perspective on Evaluation and Assessment, Pointers for Policy Development, 2013) at a time of unrelenting high-stakes, standardized testing in Australia with a dominance of secondary as opposed to primary uses of NAPLAN data by systems, schools and principals. A case for more “intelligent accountability in education” (O’Neill, Oxford Review of Education 39(1):4–16, 2013) together with a framework for analyzing efforts toward social justice in education (Cazden, International Journal of Educational Psychology 1(3):178–198, 2012) and fairer assessment make the case for more alternative assessment practices in recognition of the need for teachers’ pedagogic practice to cater for increased diversity.

OSCE online : making assessment of clinical competencies transparent, accountable & authentic

Creating an authentic assessment which at once assesses competencies, scene management, communication and overall patient care is challenging in the competitive tertiary education market. Increasing student numbers and the cost of evaluating scenario based competencies serve to ensure the need for consistent objectivity and need for timely feedback to students on their performance. Objective structured clinical examination (OSCE) is currently the most flexible approach to competency based formative and summative assessment and widely used within paramedic degree programs. Students are understandably compelled to perform well and can be frustrated by not receiving timely and appropriate feedback. Increasingly a number of products aimed at providing a more efficient and paperless approach have begun to enter the market. These products, it is suggested are aimed at medicine programs and not at allied health professions and limited to one operating system and therefore ignore issues surrounding equity and accessibility. OSCE Online aims to address this gap in the market and is tailored to these disciplines. The application will provide a service that can be both tailored and standardised from a pre-written bank, depending upon requirement to fit around the needs of clinical competency assessment. Delivering authentic assessments to address student milestones in their training to become paramedics is the cornerstone of OSCE Online. By not being restricted to a specific device it will address issues of functionality, adaptability, accessibility, authenticity and importantly: transparency and accountability by producing contemporaneous data allowing issues to be easily identified and rectified.

Teachers using annotations to engage students in assessment conversations : recontextualising knowledge

Assessment for Learning practices with students such as feedback, and self- and peer assessment are opportunities for teachers and students to develop a shared understanding of how to create quality learning performances. Quality is often represented through achievement standards. This paper explores how primary school teachers in Australia used the process of annotating work samples to develop shared understanding of achievement standards during their curriculum planning phase, and how this understanding informed their teaching so that their students also developed this understanding. Bernstein's concept of the pedagogic device is used to identify the ways teachers recontextualised their assessment knowledge into their pedagogic practices. Two researchers worked alongside seven primary school teachers in two schools over a year, gathering qualitative data through focus groups and interviews. Three general recontextualising approaches were identified in the case studies; recontextualising standards by reinterpreting the role of rubrics, recontextualising by replicating the annotation process with the students and recontextualising by reinterpreting practices with students. While each approach had strengths and limitations, all of the teachers concluded that annotating conversations in the planning phase enhanced their understanding, and informed their practices in helping students to understand expectations for quality.