Closing the loop 21st century style : providing feedback on written assessment via MP3 recordings

Feedback on student performance, whether in the classroom or on written assignments, enables them to reflect on their understandings and restructure their thinking in order to develop more powerful ideas and capabilities. Research has identified a number of broad principles of good feedback practice. These include the provision of feedback that facilitates the development of reflection in learning; helps clarify what good performance is in terms of goals, criteria and expected standards; provides opportunities to close the gap between current and desired performance; delivers high quality information to students about their learning; and encourages positive motivational beliefs and self-esteem. However, high staff–student ratios and time pressures often result in a gulf between this ideal and reality. Whilst greater use of criteria referenced assessment has enabled an improvement in the extent of feedback being provided to students, this measure alone does not go far enough to satisfy the requirements of good feedback practice. Technology offers an effective and efficient means by which personalised feedback may be provided to students. This paper presents the findings of a trial of the use of the freely available Audacity program to provide individual feedback via MP3 recordings to final year Media Law students at the Queensland University of Technology on their written assignments. The trial has yielded wide acclaim by students as an effective means of explaining the exact reasons why they received the marks they were awarded, the things they did well and the areas needing improvement. It also showed that good feedback practice can be achieved without the burden of an increase in staff workload.

Teachers using annotations to engage students in assessment conversations : recontextualising knowledge

Assessment for Learning practices with students such as feedback, and self- and peer assessment are opportunities for teachers and students to develop a shared understanding of how to create quality learning performances. Quality is often represented through achievement standards. This paper explores how primary school teachers in Australia used the process of annotating work samples to develop shared understanding of achievement standards during their curriculum planning phase, and how this understanding informed their teaching so that their students also developed this understanding. Bernstein's concept of the pedagogic device is used to identify the ways teachers recontextualised their assessment knowledge into their pedagogic practices. Two researchers worked alongside seven primary school teachers in two schools over a year, gathering qualitative data through focus groups and interviews. Three general recontextualising approaches were identified in the case studies; recontextualising standards by reinterpreting the role of rubrics, recontextualising by replicating the annotation process with the students and recontextualising by reinterpreting practices with students. While each approach had strengths and limitations, all of the teachers concluded that annotating conversations in the planning phase enhanced their understanding, and informed their practices in helping students to understand expectations for quality.

Integrating ‘role play’ in assessment to strengthen professional conduct and accountability of students : a pilot study

BACKGROUND In a process engineering setting, graduates are frequently allocated reviews of existing operations or required to scope new production processes by their supervisors with a view to improving or expanding on operations and overall productivity. These tasks may be carried out in teams and in consultation with the process engineer’s immediate line manager or a more experienced engineer, such as the Production or Maintenance Manager; ultimately reporting to senior management, which is frequently a non-engineer. Although professional skills development is part of engineering curricula, ‘professional conduct’ and ‘accountability’ required for dealing with peers and superiors in industry is not very well addressed at university. Consequently, upon graduation, many students are, in terms of knowledge and experience in this area, underprepared to work effectively in industry settings. PURPOSE The purpose of this study was to develop and implement a role-play scenario within a core 2nd year process engineering unit, so that students could gain knowledge, skills and experience in different aspects (and nuances) of professional conduct and accountability. DESIGN/METHOD In the role-play scenario, students worked in ‘engineering production teams’ to design a process for an iconic Queensland fruitcake and to present their solution and recommendations (culminating in a poster presentation) to an assessment panel consisting of staff, role-playing as, ‘production and plant managers’. Students were assessed on several areas, including professionalism using a criteria referenced assessment guide by a 3-member cross-disciplinary staff panel consisting of a Business Faculty lecturer, an engineer from industry and the lecturer of the Process Engineering unit. Professional conduct and accountability was gauged through direct questioning by the panel. Feedback was also sought from students on various aspects through a survey questionnaire after the role play activity at the end of semester. RESULTS Overall, the role play was very well performed with students achieving an average score of 79.3/100 (distinction grade). Professional conduct as assessed by panel was on average better than scores given for professional accountability (4.0 compared with 3.6 out of 5). Feedback from students indicated that the learning activities had contributed to their overall understanding of the content and the role of process engineers. Industry involvement was rated very highly as contributing to their learning at 4.8 (on Likert scale from 1 – 5) and the poster presentation was rated at 3.6. CONCLUSIONS This pilot study was successful in implementing a new assessment task for modelling professional conduct and accountability within a 2nd year core unit. This task incorporated a role-play activity and there was evidence to suggest that this and associated learning tasks were successful in broadening students’ understanding and skills in this area required for engineering practice. Following feedback given by students and staff, improvements will be made to the nature of the problem, how it is defined, its assessment, and the approach taken in the role-play scenario when the unit is offered in 2014.

An investigation into online moderation

The purpose of this paper is to understand how teachers’ identity as an assessor in a standards-referenced assessment system may be developed through their participation in online social moderation meetings. In these meetings teachers negotiate and share their understandings of assessment standards and judgement decisions. In particular, the paper focuses on the relationship between the technology, the moderation processes and teachers’ development in this assessment system. This paper draws on sociocultural theories of learning to analyse the qualitative data collected through observations of eleven online moderation meetings and interviews of the teachers involved in these meetings. The results provide insights into the mediating role of the technology with regard to teachers’ development of shared meanings and common practices within a standards-referenced assessment system.

Cracking the code : a checklist to complement CRAs for first year Justice students

It is well recognised in the literature on first year higher education that there is a need for Universities to provide further support and development in student learning skills and engagement. Assessment and feedback is an area with differing expectations and understandings among academics and students (e.g. AUSSE, CEQ). Consistency and explicitness in academic feedback is fundamental in assisting students in their transition to university education and learning. This poster captures the progress of an 18 month funded by the Faculty of Law Teaching and Learning Grant scheme (QUT). The project sought to develop and trial an assessment checklist/diagnostic tool to accompany Criteria Referenced Assessment sheets for students within the School of Justice, Law Faculty, Queensland University of Technology (QUT).The checklist was trialled across four units in the School of Justice (Law faculty) amongst an estimated cohort of over 600 students undertaking single and dual degrees.

Using annotations to inform an understanding of achievement standards

The implementation of a new national curriculum and standards-referenced assessment in Australia has been an opportunity and a challenge for teacher assessment practices. In this case study of teachers in two Queensland schools, we explore how annotating student or exemplar assessment tasks could support teacher assessment practice. Three learning conversations between the researchers and the teacher teams are interpreted through the lens of Bernstein’s (1999) horizontal and vertical discourses to understand the complexities of coming to know an assessment standard. The study contributes to the literature on the use of annotations by exploring how teachers negotiated the purposes and processes of annotation, how annotating student work or exemplars before teaching commenced supported teachers to experience greater clarity about assessment standards and, finally, some of the tensions experienced by the teachers as they considered this practice within the practicalities of their daily work.

Towards a theoretical framework for online professional discussions

Online interactions are becoming commonplace for a multitude of educational purposes. Each context presents a unique and dynamic mix of variables that combine to shape the practice and the identities of those involved. In this article, sociocultural theories of learning and sociocultural theories of technology are explored as a way to view and to map the complex interactions that can occur. The case of synchronous online moderation meetings are used as an example of the combination of variables that can impact on the development of shared understandings of a practice. Online moderation can involve teachers from geographically diverse areas discussing and negotiating their judgement decisions. These discussions represent an intersection of a national curriculum, standards-referenced assessment, moderation protocols, site-specific practices and understandings, and individual teachers’ knowledges and histories. It is suggested that the proposed theoretical combination addresses some of the limitations of each of the theories when investigating such a dynamic context. As higher education moves into increasing use of online modes of communication and a higher level of accountability the relevance of this discussion to higher education is evident.

Cracking the Code: a Checklist to Complement CRAs for First Year Justice Students

Project overview, promotional poster and how to access and use the checklist (student guide)

Contemplating Whitehead's freedom and discipline

This paper extends the previous application of Alfred Whitehead's educational ideas to the domain of enterprise education. In doing so, a unique approach to enterprise education is illustrated that links students to their reality whilst also connecting the curriculum to contemporary entrepreneurship theory. The paper reports upon past cycles of reflective practice related to the developing hic et nunc teaching and learning framework. Two specific findings of note have emerged. First, that student' learning outcomes are enhanced through the oscillating influence of freedom and discipline. However, in the absence of either factor, sub-optimal outcomes are seen to occur. That is, an imbalance between freedom and discipline has resulted in sub-optimal outcomes from either a lack of student interest or an inability to adequately apply acquired knowledge. Where gains have been made, the most obvious process has been through consultation with students. Second, that the students also play an important role in shaping the nature of the learning environments within which they interact. Both findings are of significant importance to all academics charged with the responsibility of developing enterprise education curriculum. The main implication of the paper is that in the absence of sound pedagogical practises, it is possible that enterprise programs may develop a tendency to reinforce past practises. The processes of constructive alignment and criterion-based assessment are argued to offer avenues through which students can influence the educational process. They also provide the educator with a reflective pathway through which continual improvements are constantly possible. This paper provides other academics with a window through which to view the ongoing development of a process that has been recognised nationally for teaching excellence and influenced many fine young entrepreneurs. The paper also draws attention to a set of core educational philosophies that have transferable value to any academic setting. It is noted that the task of developing a learner-centred curriculum for enterprise education has been an entrepreneurial endeavour in itself. Many mistakes have been made and many memorable achievements have been celebrated.

Sustainable urban development : a quadruple bottom line assessment framework

Sustainable development has long been promoted as the best answer to the world's environmental problems. This term has generated mass appeal as it implies that the development of the built environment and its associated resource consumption can both be achieved without jeopardising the natural environment. In the urban context, sustainability issues have been reflected in the pomotion of sustainable urbanisation in a manner that allows future generations to repeat this process. This paper attempts to highlight an increasing urgency in formulating a suitable model for assessing sustainability at urban level, because this is where the bulk of a nation's population reside, and where sustainability problems mostly occur. It will also point out to the increasing importance of governance in facilitating urban sustainability research. This assessment involves the use of physical, social, environmental and goverance aspects in assessing the extent to which development of an urban settlement is sustainable. Specifically, this assessment model is carried out to determine whether or not sustainable urban development pratice is implemented in the provision of residential development, and in particular whether the development of master-planned residential communities have more desireable outcomes compared to traditional residential subdivision.

Developing speaking assessment tasks to reflect the 'social turn' in language testing

Interactional competence has emerged as a focal point for language testing researchers in recent years. In spoken communication involving two or more interlocutors, the co-construction of discourse is central to successful interaction. The acknowledgement of co-construction has led to concern over the impact of the interlocutor and the separability of performances in speaking tests involving interaction. The purpose of this article is to review recent studies of direct relevance to the construct of interactional competence and its operationalisation by raters in the context of second language speaking tests. The review begins by tracing the emergence of interaction as a criterion in speaking tests from a theoretical perspective, and then focuses on research salient to interactional effectiveness that has been carried out in the context of language testing interviews and group and paired speaking tests.

Sustainability and mandatory disclosure in Queensland : an assessment of the impact on home buyer patterns

Sustainability Declarations were introduced by the Queensland State Government on 1 January 2010 as a compulsory measure for all dwelling sales. The purpose of this policy decision was to improve the relevance of sustainability in the home ownership decision making process. This paper assesses the initial impact of this initiative over its first year in operation. In partnership with the Real Estate Institute of Queensland, real estate agents and salespeople in Queensland were surveyed to determine what impact the Sustainability Declaration has had on home buyer decision making. The level of compliance by the real estate industry was also reviewed. These preliminary findings indicate a high level of compliance from the real estate industry, however results confirm that sustainability is yet to become a criterion of relevance to the majority of home buyers in Queensland. The Sustainability Declarations are a first step in raising awareness in home owners of the importance of sustainability in housing. Further monitoring of this impact will be carried out over time.

An assessment system for evaluation of driving competencies

Automobiles have deeply impacted the way in which we travel but they have also contributed to many deaths and injury due to crashes. A number of reasons for these crashes have been pointed out by researchers. Inexperience has been identified as a contributing factor to road crashes. Driver’s driving abilities also play a vital role in judging the road environment and reacting in-time to avoid any possible collision. Therefore driver’s perceptual and motor skills remain the key factors impacting on road safety. Our failure to understand what is really important for learners, in terms of competent driving, is one of the many challenges for building better training programs. Driver training is one of the interventions aimed at decreasing the number of crashes that involve young drivers. Currently, there is a need to develop comprehensive driver evaluation system that benefits from the advances in Driver Assistance Systems. A multidisciplinary approach is necessary to explain how driving abilities evolves with on-road driving experience. To our knowledge, driver assistance systems have never been comprehensively used in a driver training context to assess the safety aspect of driving. The aim and novelty of this thesis is to develop and evaluate an Intelligent Driver Training System (IDTS) as an automated assessment tool that will help drivers and their trainers to comprehensively view complex driving manoeuvres and potentially provide effective feedback by post processing the data recorded during driving. This system is designed to help driver trainers to accurately evaluate driver performance and has the potential to provide valuable feedback to the drivers. Since driving is dependent on fuzzy inputs from the driver (i.e. approximate distance calculation from the other vehicles, approximate assumption of the other vehicle speed), it is necessary that the evaluation system is based on criteria and rules that handles uncertain and fuzzy characteristics of the driving tasks. Therefore, the proposed IDTS utilizes fuzzy set theory for the assessment of driver performance. The proposed research program focuses on integrating the multi-sensory information acquired from the vehicle, driver and environment to assess driving competencies. After information acquisition, the current research focuses on automated segmentation of the selected manoeuvres from the driving scenario. This leads to the creation of a model that determines a “competency” criterion through the driving performance protocol used by driver trainers (i.e. expert knowledge) to assess drivers. This is achieved by comprehensively evaluating and assessing the data stream acquired from multiple in-vehicle sensors using fuzzy rules and classifying the driving manoeuvres (i.e. overtake, lane change, T-crossing and turn) between low and high competency. The fuzzy rules use parameters such as following distance, gaze depth and scan area, distance with respect to lanes and excessive acceleration or braking during the manoeuvres to assess competency. These rules that identify driving competency were initially designed with the help of expert’s knowledge (i.e. driver trainers). In-order to fine tune these rules and the parameters that define these rules, a driving experiment was conducted to identify the empirical differences between novice and experienced drivers. The results from the driving experiment indicated that significant differences existed between novice and experienced driver, in terms of their gaze pattern and duration, speed, stop time at the T-crossing, lane keeping and the time spent in lanes while performing the selected manoeuvres. These differences were used to refine the fuzzy membership functions and rules that govern the assessments of the driving tasks. Next, this research focused on providing an integrated visual assessment interface to both driver trainers and their trainees. By providing a rich set of interactive graphical interfaces, displaying information about the driving tasks, Intelligent Driver Training System (IDTS) visualisation module has the potential to give empirical feedback to its users. Lastly, the validation of the IDTS system’s assessment was conducted by comparing IDTS objective assessments, for the driving experiment, with the subjective assessments of the driver trainers for particular manoeuvres. Results show that not only IDTS was able to match the subjective assessments made by driver trainers during the driving experiment but also identified some additional driving manoeuvres performed in low competency that were not identified by the driver trainers due to increased mental workload of trainers when assessing multiple variables that constitute driving. The validation of IDTS emphasized the need for an automated assessment tool that can segment the manoeuvres from the driving scenario, further investigate the variables within that manoeuvre to determine the manoeuvre’s competency and provide integrated visualisation regarding the manoeuvre to its users (i.e. trainers and trainees). Through analysis and validation it was shown that IDTS is a useful assistance tool for driver trainers to empirically assess and potentially provide feedback regarding the manoeuvres undertaken by the drivers.

Assessment reform and educational change in Australia

This chapter addresses the changing climate of assessment policy and practice in Australia in response to global trends in education and the mounting accountability demands of standards-driven reform. Queensland, a State of Australia, has a tradition of respecting and trusting teacher judgment through the practice of, and policy commitment to, externally moderated school-based assessment. This chapter outlines the global trends in curriculum and assessment reform, and then analyzes the impact of international comparisons on national policy. The creation of the Australian Curriculum, Assessment and Reporting Authority (ACARA) together with the intent of establishing a standards-referenced framework raises tensions and challenges for teachers’ practice. The argument for sustaining confidence in teacher-based assessment is developed with reference to research evidence pertaining to the use of more authentic assessments and moderation practices for the purposes of improving learning, equity and accountability. Evidence is drawn from local studies of teacher judgment practice and used to demonstrate these developments and in so doing illuminate the complex issues of engaging the demands of policy while sustaining confidence in teacher assessment.