The effect of feedback from pupil to teacher on assessment for leaning and visible learning: An ethnographic case study in a community school in England and the outcome in a state high school in Queensland, Australia

This study shows that there is positive regulatory effect of feedback from pupils to teachers on Assessment for Learning (AfL), classroom proactiveness, and on visible and progressive learning but not on behaviour. This research finding further articulates feedback from pupil to teacher as a paradigm shift from the classical paradigm of feedback from teacher to pupil. Here, the emphasis is geared towards pupils understanding of objectives built from previous knowledge. These are then feedback onto the teachers by the pupils in the form of discrete loops of cues and questions, where they are with their learning. This therefore enables them to move to the next level of understanding, and thus acquired independence, which in turn is reflected by their success in both formative and summative assessments. This study therefore shows that when feedback from pupil to teacher is used in combination with teacher to pupil feedback, AfL is ameliorated and hence, visible and accelerated learning occurs in a gender, nor subject non-dependent manner.

Enhanced student learning in accounting utilising web-based technology, peer-review feedback and reflective practices: A learning community approach to assessment

Higher education is becoming a major driver of economic competitiveness in an increasingly knowledge-driven global economy. Maintaining the competitive edge has seen an increase in public accountability of higher education institutions through the mechanism of ranking universities based on the quality of their teaching and learning outcomes. As a result, assessment processes are under scrutiny, creating tensions between standardisation and measurability and the development of creative and reflective learners. These tensions are further highlighted in the context of large undergraduate subjects, learner diversity and time-poor academics and students. Research suggests that high level and complex learning is best developed when assessment, combined with effective feedback practices, involves students as partners in these processes. This article reports on a four-phase, cross-institution and cross-discipline project designed to embed peer-review processes as part of the assessment in two large, undergraduate accounting classes. Using a social constructivist view of learning, which emphasises the role of both teacher and learner in the development of complex cognitive understandings, we undertook an iterative process of peer review. Successive phases built upon students’ feedback and achievements and input from language/learning and curriculum experts to improve the teaching and learning outcomes.

Engaging Students in the Implementation of Criterion Referenced Assessment in First Year Law

The Queensland University of Technology (QUT) University Academic Board approved a new QUT Assessment Policy in September 2003, which requires a criterion-referenced approach as opposed to a norm-referenced approach to assessment across the university(QUT,MOPP,2003). In 2004, the QUT Law School embarked upon a process of awareness raising about criterion-referenced assessment amongst staff and from 2004 – 2005 staggered the implementation of criterion-referenced assessment in all first year core undergraduate law units. This paper will briefly discuss the benefits and potential pitfalls of criterion referenced assessment and the context for implementing it in the first year law program, report on student’s feedback on the introduction of criterion referenced assessment and the strategies adopted in 2005 to engage students more fully in criterion referenced assessment processes to enhance their learning outcomes.

Developing a participatory impact assessment approach and action research culture within a communication for social change organisation in Nepal

Developing an effective impact evaluation framework, managing and conducting rigorous impact evaluations, and developing a strong research and evaluation culture within development communication organisations presents many challenges. This is especially so when both the community and organisational context is continually changing and the outcomes of programs are complex and difficult to clearly identify.----- This paper presents a case study from a research project being conducted from 2007-2010 that aims to address these challenges and issues, entitled Assessing Communication for Social Change: A New Agenda in Impact Assessment. Building on previous development communication projects which used ethnographic action research, this project is developing, trailing and rigorously evaluating a participatory impact assessment methodology for assessing the social change impacts of community radio programs in Nepal. This project is a collaboration between Equal Access – Nepal (EAN), Equal Access – International, local stakeholders and listeners, a network of trained community researchers, and a research team from two Australian universities. A key element of the project is the establishment of an organisational culture within EAN that values and supports the impact assessment process being developed, which is based on continuous action learning and improvement. The paper describes the situation related to monitoring and evaluation (M&E) and impact assessment before the project began, in which EAN was often reliant on time-bound studies and ‘success stories’ derived from listener letters and feedback. We then outline the various strategies used in an effort to develop stronger and more effective impact assessment and M&E systems, and the gradual changes that have occurred to date. These changes include a greater understanding of the value of adopting a participatory, holistic, evidence-based approach to impact assessment. We also critically review the many challenges experienced in this process, including:----- • Tension between the pressure from donors to ‘prove’ impacts and the adoption of a bottom-up, participatory approach based on ‘improving’ programs in ways that meet community needs and aspirations.----- • Resistance from the content teams to changing their existing M&E practices and to the perceived complexity of the approach.----- • Lack of meaningful connection between the M&E and content teams.----- • Human resource problems and lack of capacity in analysing qualitative data and reporting results.----- • The contextual challenges, including extreme poverty, wide cultural and linguistic diversity, poor transport and communications infrastructure, and political instability.----- • A general lack of acceptance of the importance of evaluation within Nepal due to accepting everything as fate or ‘natural’ rather than requiring investigation into a problem.

Dancing between diversity and consistency : refining assessment in postgraduate degrees in dance

This publication is the culmination of a 2 year Australian Learning and Teaching Council's Project Priority Programs Research Grant which investigates key issues and challenges in developing flexible guidelines lines for best practice in Australian Doctoral and Masters by Research Examination, encompassing the two modes of investigation, written and multi-modal (practice-led/based) theses, their distinctiveness and their potential interplay. The aims of the project were to address issues of assessment legitimacy raised by the entry of practice-orientated dance studies into Australian higher degrees; examine literal embodiment and presence, as opposed to cultural studies about states of embodiment; foreground the validity of questions around subjectivity and corporeal intelligence/s and the reliability of artistic/aesthetic communications, and finally to celebrate ‘performance mastery’(Melrose 2003) as a rigorous and legitimate mode of higher research. The project began with questions which centred around: the functions of higher degree dance research; concepts of 'master-ness’ and ‘doctorateness’; the kinds of languages, structures and processes which may guide candidates, supervisors, examiners and research personnel; the purpose of evaluation/examination; addressing positive and negative attributes of examination. Finally the study examined ways in which academic/professional, writing/dancing, tradition/creation and diversity/consistency relationships might be fostered to embrace change. Over two years, the authors undertook a qualitative national study encompassing a triangulation of semi-structured face to face interviews and industry forums to gather views from the profession, together with an analysis of existing guidelines, and recent literature in the field. The most significant primary data emerged from 74 qualitative interviews with supervisors, examiners, research deans and administrators, and candidates in dance and more broadly across the creative arts. Qualitative data gathered from the two primary sources, was coded and analysed using the NVivo software program. Further perspectives were drawn from international consultant and dance researcher Susan Melrose, as well as publications in the field, and initial feedback from a draft document circulated at the World Dance Alliance Global Summit in July 2008 in Brisbane. Refinement of data occurred in a continual sifting process until the final publication was produced. This process resulted in a set of guidelines in the form of a complex dynamic system for both product and process oriented outcomes of multi-modal theses, along with short position papers on issues which arose from the research such as contested definitions, embodiment and ephemerality, ‘liveness’ in performance research higher degrees, dissolving theory/practice binaries, the relationship between academe and industry, documenting practices and a re-consideration of the viva voce.

Implementation of an integrated, technology-based, discovery mode assessment item involving an incubation period to enhance learning outcomes for engineering maths students

In this paper we discuss our current efforts to develop and implement an exploratory, discovery mode assessment item into the total learning and assessment profile for a target group of about 100 second level engineering mathematics students. The assessment item under development is composed of 2 parts, namely, a set of "pre-lab" homework problems (which focus on relevant prior mathematical knowledge, concepts and skills), and complementary computing laboratory exercises which are undertaken within a fixed (1 hour) time frame. In particular, the computing exercises exploit the algebraic manipulation and visualisation capabilities of the symbolic algebra package MAPLE, with the aim of promoting understanding of certain mathematical concepts and skills via visual and intuitive reasoning, rather than a formal or rigorous approach. The assessment task we are developing is aimed at providing students with a significant learning experience, in addition to providing feedback on their individual knowledge and skills. To this end, a noteworthy feature of the scheme is that marks awarded for the laboratory work are primarily based on the extent to which reflective, critical thinking is demonstrated, rather than the amount of CBE-style tasks completed by the student within the allowed time. With regard to student learning outcomes, a novel and potentially critical feature of our scheme is that the assessment task is designed to be intimately linked to the overall course content, in that it aims to introduce important concepts and skills (via individual student exploration) which will be revisited somewhat later in the pedagogically more restrictive formal lecture component of the course (typically a large group plenary format). Furthermore, the time delay involved, or "incubation period", is also a deliberate design feature: it is intended to allow students the opportunity to undergo potentially important internal re-adjustments in their understanding, before being exposed to lectures on related course content which are invariably delivered in a more condensed, formal and mathematically rigorous manner. In our presentation, we will discuss in more detail our motivation and rationale for trailing such a scheme for the targeted student group. Some of the advantages and disadvantages of our approach (as we perceived them at the initial stages) will also be enumerated. In a companion paper, the theoretical framework for our approach will be more fully elaborated, and measures of student learning outcomes (as obtained from eg. student provided feedback) will be discussed.

Assessment of oral proficiency in EAP programs : a case for pair interaction

Reliability and validity in the testing of spoken language are essential in order to assess learners' English language proficiency as evidence of their readiness to begin courses in tertiary institutions. Research has indicated that the task chosen to elicit language samples can have a marked effect on both the nature of the interaction, including the power differential, and assessment, raising the issue of ethics. This exploratory studey, with a group of 32 students from the Peoples's Republic of China preparing for tertiary study in Singapore, compares test-takers' reactions to the use of an oral proficiency interview and a pair interaction.

Assessment for learning as a participative pedagogy

AfL practices observed in case studies in a North Queensland school were analysed from a sociocultural theoretical perspective. AfL practices of feedback, dialogue and peer assessment were viewed as an opportunity for students to learn the social expectations about being an autonomous learner, or central participant within the classroom community of practice. This process of becoming more expert and belonging within the community of practice involved students negotiating identities of participation that included knowing both academic skills and social expectations within the classroom. This paper argues that when AfL practices are viewed as ways of enhancing participation, there is potential for learners to negotiate identities as autonomous learners. AfL practices within the daily classroom interactions and pedagogy that enabled students to develop a shared repertoire, joint enterprise and mutual engagement in the classroom communities of practice are described. The challenges for teachers in shifting their gaze to patterns of participation are also briefly discussed.

A virtual test assessment tool for second year signal processing students

The development and use of a virtual assessment tool for a signal processing unit is described. It allows students to take a test from anywhere using a web browser to connect to the university server that hosts the test. While student responses are of the multiple choice type, they have to work out problems to arrive at the answer to be entered. CGI programming is used to verify student identification information and record their scores as well as provide immediate feedback after the test is complete. The tool has been used at QUT for the past 3 years and student feedback is discussed. The virtual assessment tool is an efficient alternative to marking written assignment reports that can often take more hours than actual lecture hall contact from a lecturer or tutor. It is especially attractive for very large classes that are now the norm at many universities in the first two years.

Motivating domestic energy conservation through comparative, community-based feedback in mobile and social media

The progress of technology has led to the increased adoption of energy monitors among household energy consumers. While the monitors available on the market deliver real-time energy usage feedback to the consumer, the format of this data is usually unengaging and mundane. Moreover, it fails to address consumers with different motivations and needs to save and compare energy. This paper presents a study that seeks to provide initial indications for motivation-specific design of energy-related feedback. We focus on comparative feedback supported by a community of energy consumers. In particular, we examine eco-visualisations, temporal self-comparison, norm comparison, one-on-one comparison and ranking, whereby the last three allow us to explore the potential of socialising energy-related feedback. These feedback types were integrated in EnergyWiz – a mobile application that enables users to compare with their past performance, neighbours, contacts from social networking sites and other EnergyWiz users. The application was evaluated in personal, semi-structured interviews, which provided first insights on how to design motivation-related comparative feedback.

An assessment system for evaluation of driving competencies

Automobiles have deeply impacted the way in which we travel but they have also contributed to many deaths and injury due to crashes. A number of reasons for these crashes have been pointed out by researchers. Inexperience has been identified as a contributing factor to road crashes. Driver’s driving abilities also play a vital role in judging the road environment and reacting in-time to avoid any possible collision. Therefore driver’s perceptual and motor skills remain the key factors impacting on road safety. Our failure to understand what is really important for learners, in terms of competent driving, is one of the many challenges for building better training programs. Driver training is one of the interventions aimed at decreasing the number of crashes that involve young drivers. Currently, there is a need to develop comprehensive driver evaluation system that benefits from the advances in Driver Assistance Systems. A multidisciplinary approach is necessary to explain how driving abilities evolves with on-road driving experience. To our knowledge, driver assistance systems have never been comprehensively used in a driver training context to assess the safety aspect of driving. The aim and novelty of this thesis is to develop and evaluate an Intelligent Driver Training System (IDTS) as an automated assessment tool that will help drivers and their trainers to comprehensively view complex driving manoeuvres and potentially provide effective feedback by post processing the data recorded during driving. This system is designed to help driver trainers to accurately evaluate driver performance and has the potential to provide valuable feedback to the drivers. Since driving is dependent on fuzzy inputs from the driver (i.e. approximate distance calculation from the other vehicles, approximate assumption of the other vehicle speed), it is necessary that the evaluation system is based on criteria and rules that handles uncertain and fuzzy characteristics of the driving tasks. Therefore, the proposed IDTS utilizes fuzzy set theory for the assessment of driver performance. The proposed research program focuses on integrating the multi-sensory information acquired from the vehicle, driver and environment to assess driving competencies. After information acquisition, the current research focuses on automated segmentation of the selected manoeuvres from the driving scenario. This leads to the creation of a model that determines a “competency” criterion through the driving performance protocol used by driver trainers (i.e. expert knowledge) to assess drivers. This is achieved by comprehensively evaluating and assessing the data stream acquired from multiple in-vehicle sensors using fuzzy rules and classifying the driving manoeuvres (i.e. overtake, lane change, T-crossing and turn) between low and high competency. The fuzzy rules use parameters such as following distance, gaze depth and scan area, distance with respect to lanes and excessive acceleration or braking during the manoeuvres to assess competency. These rules that identify driving competency were initially designed with the help of expert’s knowledge (i.e. driver trainers). In-order to fine tune these rules and the parameters that define these rules, a driving experiment was conducted to identify the empirical differences between novice and experienced drivers. The results from the driving experiment indicated that significant differences existed between novice and experienced driver, in terms of their gaze pattern and duration, speed, stop time at the T-crossing, lane keeping and the time spent in lanes while performing the selected manoeuvres. These differences were used to refine the fuzzy membership functions and rules that govern the assessments of the driving tasks. Next, this research focused on providing an integrated visual assessment interface to both driver trainers and their trainees. By providing a rich set of interactive graphical interfaces, displaying information about the driving tasks, Intelligent Driver Training System (IDTS) visualisation module has the potential to give empirical feedback to its users. Lastly, the validation of the IDTS system’s assessment was conducted by comparing IDTS objective assessments, for the driving experiment, with the subjective assessments of the driver trainers for particular manoeuvres. Results show that not only IDTS was able to match the subjective assessments made by driver trainers during the driving experiment but also identified some additional driving manoeuvres performed in low competency that were not identified by the driver trainers due to increased mental workload of trainers when assessing multiple variables that constitute driving. The validation of IDTS emphasized the need for an automated assessment tool that can segment the manoeuvres from the driving scenario, further investigate the variables within that manoeuvre to determine the manoeuvre’s competency and provide integrated visualisation regarding the manoeuvre to its users (i.e. trainers and trainees). Through analysis and validation it was shown that IDTS is a useful assistance tool for driver trainers to empirically assess and potentially provide feedback regarding the manoeuvres undertaken by the drivers.

Towards a classification of criterion referenced assessment models in mathematics courses : student and academic perspectives

We present the findings of a study into the implementation of explicitly criterion- referenced assessment in undergraduate courses in mathematics. We discuss students' concepts of criterion referencing and also the various interpretations that this concept has among mathematics educators. Our primary goal was to move towards a classification of criterion referencing models in quantitative courses. A secondary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. The data and feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, it did not alter the way the actually approached the assessment activity.

Communicating assessment criteria is not sufficient for influencing students' approaches to assessment tasks – perspectives from a differential equations class

This report presents the findings of an exploratory study into the perceptions held by students regarding the use of criterion-referenced assessment in an undergraduate differential equations class. Students in the class were largely unaware of the concept of criterion referencing and of the various interpretations that this concept has among mathematics educators. Our primary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. Quantitative data and qualitative feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, the manner in which they actually approached the assessment activity was not altered as a result of the use of explicitly communicated grading criteria.