Portfolio assessment

At a time when global uncertainty is paramount and when a new form or re-form of curriculum is emerging – with content displaced by skills and knowledge acquisition by learning - assessment, too, begins to take on a new from or re-form. The focus for assessment has shifted to that which engages and promotes learning as s process rather than an assessment that focuses solely on measuring and reporting learning as product or score. The use of the portfolio for assessment offers the potential for the process and progress – integral to learning - to be included.

Academic rigour in science assessment tasks

The critical problem of student disengagement and underachievement in the middle years of schooling (Years 4 . 9) has focussed attention on the quality of educational programs in schools, in Australia and elsewhere. The loss of enthusiasm for science in the middle years is particularly problematic given the growing demand for science professionals. Reshaping middle years programs has included an emphasis on integrating Information and Communication Technologies (ICTs) and improving assessment practices to engage students in higher cognitive processes and enhance academic rigour. Understanding the nature of academic rigour and how to embed it in students. science assessment tasks that incorporate the use of ICTs could enable teachers to optimise the quality of the learning environment. However, academic rigour is not clearly described or defined in the literature and there is little empirical evidence upon which researchers and teachers could draw to enhance understandings. This study used a collective case study design to explore teachers' understandings of academic rigour within science assessment tasks. The research design is based on a conceptual framework that is underpinned by socio-cultural theory. Three methods were used to collect data from six middle years teachers and their students. These methods were a survey, focus group discussion with teachers and a group of students and individual semi-structured interviews with teachers. Findings of the case study revealed six criteria of academic rigour, namely, higher order thinking, alignment, building on prior knowledge, scaffolding, knowledge construction and creativity. Results showed that the middle years teachers held rich understandings of academic rigour that led to effective utilisation of ICTs in science assessment tasks. Findings also indicated that teachers could further enhance their understandings of academic rigour in some aspects of each of the criteria. In particular, this study found that academic rigour could have been further optimised by: promoting more thoughtful discourse and interaction to foster higher order thinking; increasing alignment between curriculum, pedagogy, and assessment, and students. prior knowledge; placing greater emphasis on identifying, activating and building on prior knowledge; better differentiating the level of scaffolding provided and applying it more judiciously; fostering creativity throughout tasks; enhancing teachers‟ content knowledge and pedagogical content knowledge, and providing more in-depth coverage of fewer topics to support knowledge construction. Key contributions of this study are a definition and a model which clarify the nature of academic rigour.

Culture-fair assessment : challenging Indigenous students through effortful mathematics teaching

This paper reports on a mathematics education research project centred on teachers’ pedagogical practices and capacity to assess Indigenous Australian students in a culture-fair manner. The project has been funded by the Australian Research Council Linkage program and is being conducted in seven Catholic and Independent primary schools in north Queensland. Our Industry Partners are Catholic Education and the Association of Independent Schools, Queensland. The study aims to provide greater understanding about how to build more equitable assessment practices to address the issue of underperforming Aboriginal and Torres Strait Islander (ATSI) students in regional and remote Australia. The goal is to identify ways forward by attending to culture-fair assessment practice. The research is exploring the attitudes, beliefs and responses of Indigenous students to assessment in the context of mathematics learning with particular focus on teacher knowledge in these educational settings in relation to the design of assessment tasks that are authentic and engaging for these students in an accountability context. This approach highlights how teachers need to distinguish the ‘funds of knowledge’ (González, Moll, Floyd Tenery, Rivera, Rendón, Gonzales & Amanti, 2008) that Indigenous students draw on and how teachers need to be culturally responsive in their pedagogy to open up curriculum and assessment practice to allow for different ways of knowing and being

Strategies to engage engineering students in group project work

Project focused group work is significant in developing social and personal skills as well as extending the ability to identify, formulate and solve engineering problems. As a result of increasing undergraduate class sizes, along with the requirement for many students to work part-time, group projects, peer and collaborative learning are seen as a fundamental part of engineering education. Group formation, connection to learning objectives and fairness of assessment has been widely reported as major issues that leave students dissatisfied with group project based units. Several strategies were trialled including a study of formation of groups by different methods across two engineering disciplines over the past 2 years. Other strategies involved a more structured approach to assessment practices of civil and electrical engineering disciplines design units. A confidential online teamwork management tool was used to collect and collate student self and peer assessment ratings and used for both formative feedback as well as assessment purposes. Student satisfaction and overall academic results in these subjects have improved since the introduction of these interventions. Both student and staff feedback highlight this approach as enhancing student engagement and satisfaction, improved student understanding of group roles, reducing number of dysfunctional groups whilst requiring less commitment of academic resources.

Talking the talk : oracy demands in first year university assessment tasks

With more constructivist approaches to learning in higher education and more value on teamwork skills, students’ oracy (speaking and listening) features more prominently in curriculum, pedagogy and assessment. The paper reports on a study of two first-year Australian university courses in disciplines with explicit industry orientations and high proportions of international students. Drawing on classroom observations and interviews with the lecturers, this paper investigates their pedagogical designs on oracy and the oracy demands of their assessment tasks. The study found that talk-based assessment tasks (a group project and a group oral presentation) featured in both courses but the two courses treated students’ oracy differently: as product or process. The contrast between the two assessment designs explicates issues around EAL student needs, authentic links to industry, the provenance of criteria used to assess performance, perceptions about the relevance of talk and the ‘hidden assessment’ of oracy.

An assessment system for evaluation of driving competencies

Automobiles have deeply impacted the way in which we travel but they have also contributed to many deaths and injury due to crashes. A number of reasons for these crashes have been pointed out by researchers. Inexperience has been identified as a contributing factor to road crashes. Driver’s driving abilities also play a vital role in judging the road environment and reacting in-time to avoid any possible collision. Therefore driver’s perceptual and motor skills remain the key factors impacting on road safety. Our failure to understand what is really important for learners, in terms of competent driving, is one of the many challenges for building better training programs. Driver training is one of the interventions aimed at decreasing the number of crashes that involve young drivers. Currently, there is a need to develop comprehensive driver evaluation system that benefits from the advances in Driver Assistance Systems. A multidisciplinary approach is necessary to explain how driving abilities evolves with on-road driving experience. To our knowledge, driver assistance systems have never been comprehensively used in a driver training context to assess the safety aspect of driving. The aim and novelty of this thesis is to develop and evaluate an Intelligent Driver Training System (IDTS) as an automated assessment tool that will help drivers and their trainers to comprehensively view complex driving manoeuvres and potentially provide effective feedback by post processing the data recorded during driving. This system is designed to help driver trainers to accurately evaluate driver performance and has the potential to provide valuable feedback to the drivers. Since driving is dependent on fuzzy inputs from the driver (i.e. approximate distance calculation from the other vehicles, approximate assumption of the other vehicle speed), it is necessary that the evaluation system is based on criteria and rules that handles uncertain and fuzzy characteristics of the driving tasks. Therefore, the proposed IDTS utilizes fuzzy set theory for the assessment of driver performance. The proposed research program focuses on integrating the multi-sensory information acquired from the vehicle, driver and environment to assess driving competencies. After information acquisition, the current research focuses on automated segmentation of the selected manoeuvres from the driving scenario. This leads to the creation of a model that determines a “competency” criterion through the driving performance protocol used by driver trainers (i.e. expert knowledge) to assess drivers. This is achieved by comprehensively evaluating and assessing the data stream acquired from multiple in-vehicle sensors using fuzzy rules and classifying the driving manoeuvres (i.e. overtake, lane change, T-crossing and turn) between low and high competency. The fuzzy rules use parameters such as following distance, gaze depth and scan area, distance with respect to lanes and excessive acceleration or braking during the manoeuvres to assess competency. These rules that identify driving competency were initially designed with the help of expert’s knowledge (i.e. driver trainers). In-order to fine tune these rules and the parameters that define these rules, a driving experiment was conducted to identify the empirical differences between novice and experienced drivers. The results from the driving experiment indicated that significant differences existed between novice and experienced driver, in terms of their gaze pattern and duration, speed, stop time at the T-crossing, lane keeping and the time spent in lanes while performing the selected manoeuvres. These differences were used to refine the fuzzy membership functions and rules that govern the assessments of the driving tasks. Next, this research focused on providing an integrated visual assessment interface to both driver trainers and their trainees. By providing a rich set of interactive graphical interfaces, displaying information about the driving tasks, Intelligent Driver Training System (IDTS) visualisation module has the potential to give empirical feedback to its users. Lastly, the validation of the IDTS system’s assessment was conducted by comparing IDTS objective assessments, for the driving experiment, with the subjective assessments of the driver trainers for particular manoeuvres. Results show that not only IDTS was able to match the subjective assessments made by driver trainers during the driving experiment but also identified some additional driving manoeuvres performed in low competency that were not identified by the driver trainers due to increased mental workload of trainers when assessing multiple variables that constitute driving. The validation of IDTS emphasized the need for an automated assessment tool that can segment the manoeuvres from the driving scenario, further investigate the variables within that manoeuvre to determine the manoeuvre’s competency and provide integrated visualisation regarding the manoeuvre to its users (i.e. trainers and trainees). Through analysis and validation it was shown that IDTS is a useful assistance tool for driver trainers to empirically assess and potentially provide feedback regarding the manoeuvres undertaken by the drivers.

Communicating assessment criteria is not sufficient for influencing students' approaches to assessment tasks – perspectives from a differential equations class

This report presents the findings of an exploratory study into the perceptions held by students regarding the use of criterion-referenced assessment in an undergraduate differential equations class. Students in the class were largely unaware of the concept of criterion referencing and of the various interpretations that this concept has among mathematics educators. Our primary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. Quantitative data and qualitative feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, the manner in which they actually approached the assessment activity was not altered as a result of the use of explicitly communicated grading criteria.

The link between age, career goals, and adaptive development for work-related learning among local government employees

More recently, lifespan development psychology models of adaptive development have been applied to the workforce to investigate ageing worker and lifespan issues. The current study uses the Learning and Development Survey (LDS) to investigate employee selection and engagement of learning and development goals and opportunities and constraints for learning at work in relation to demographics and career goals. It was found that mature age was associated with perceptions of preferential treatment of younger workers with respect to learning and development. Age was also correlated with several career goals. Findings suggest that younger workers’ learning and development options are better catered for in the workplace. Mature aged workers may compensate for unequal learning opportunities at work by studying for an educational qualification or seeking alternate job opportunities. The desire for a higher level job within the organization or educational qualification was linked to engagement in learning and development goals at work. It is suggested that an understanding of employee perceptions in the workplace in relation to goals and activities may be important in designing strategies to retain workers.

Raising the stakes : the challenges for teacher assessment

These are changing times for teachers and their students in Australia with the introduction of a national curriculum and standards driven reform. While countries in Europe such as England, and in Asia such as Singapore, are changing policy to make more use of assessment to support and improve learning it appears that we in Australia are moving towards creating policy that will raise the assessment stakes for the alleged purposes of transparency, accountability and fairness. What can be learnt from countries that have had years of high stakes testing? How can Australia avoid the mistakes of past curriculum and assessment reform efforts? And how can Australian teachers build their capacity to maximise their use of the learning power of assessment? These are key questions that will be addressed in this presentation with reference to innovative research from global networks that have maintained the assessment focus on learning.

Learning and teaching from the edge to centre stage : Critical factors in embedding sustainable university-wide engagement in external awards and grants funding initiatives.

Teaching awards, grants and fellowships are strategies used to recognise outstanding contributions to learning and teaching, encourage innovation, and to shift learning and teaching from the edge to centre stage. Examples range from school, faculty and institutional award and grant schemes to national schemes such as those offered by the Australian Learning and Teaching Council (ALTC), the Carnegie Foundation for the Advancement of Teaching in the United States, and the Fund for the Development of Teaching and Learning in higher education in the United Kingdom. The Queensland University of Technology (QUT) has experienced outstanding success in all areas of the ALTC funding since the inception of the Carrick Institute for Learning and Teaching in 2004. This paper reports on a study of the critical factors that have enabled sustainable and resilient institutional engagement with ALTC programs. As a lens for examining the QUT environment and practices, the study draws upon the five conditions of the framework for effective dissemination of innovation developed by Southwell, Gannaway, Orrell, Chalmers and Abraham (2005, 2010): 1. Effective, multi-level leadership and management 2. Climate of readiness for change 3. Availability of resources 4. Comprehensive systems in institutions and funding bodies 5. Funding design The discussion on the critical factors and practical and strategic lessons learnt for successful university-wide engagement offer insights for university leaders and staff who are responsible for learning and teaching award, grant and associated internal and external funding schemes.

Raising the stakes: the challenges for teacher assessment

These are changing times in Australia for teachers and their students, with the introduction of a national curriculum and standards driven reform. While countries in Europe such as England, and in Asia such as Singapore, are changing policy to use assessment in the support of and improvement of learning it appears that we in Australia are moving towards creating policy that will raise the assessment stakes for the putative purposes of transparency, accountability and fairness. What can be learnt from countries that have had years of high stakes testing? How can Australia avoid the mistakes of past curriculum and assessment reform efforts? And how can Australian teachers build their capacity to maximise their use of the learning power of assessment? These are the questions that are addressed in this article, with reference to innovative research from global networks that have maintained the assessment focus on learning rather than accountability practices.

Sustaining teacher professionalism in the context of standards referenced assessment reform

Standards referenced reform, tied to reporting, engages directly with assessment issues related to accountability. Assessment is the key to good education and is inseparable from curriculum. In an accountability context, standards are used as a lever to improve the reliability and consistency of teacher judgement; and classroom evidence is used by education systems for reporting and tracking achievement over time. Assessment is thus a powerful driver for change and is at the heart of the teaching-learning dynamic. The relationship between the learner, learning and assessment needs to be kept central and the idea of teacher empowerment is fundamental. This chapter is a call to honour and sustain teacher professionalism through educative forms of school-based and teacher-led evaluation, assessment and communities of judgement practice. It supports the argument for a central place for classroom assessment in the role of assessment in educational accountability...

The assessment of mental health in children with intellectual disability

Aim: Individuals with intellectual disability (ID) have higher rates of mental health problems than the general population. Assessment tends to rely heavily on self-report, but persons with ID often have difficulties in identifying and describing their own thoughts and feelings. Measures that are psychometrically sound with typically developing populations may not be as robust in samples with ID. The aim of the current study was to examine a range of self-report measures for assessing the mental health of children with ID, and to consider the appropriateness of minor modifications to those instruments. Method: The participants were 58 children with ID (mean 11.7 years) attending Year 6 in mainstream primary schools. At the first time point they completed four established measures of depression, anxiety and mood. Minor modifications were made to wording and format at re-administration six months later. Results: Internal consistency varied considerably across measures. Modifications resulted in small or no improvements, but the results were relatively consistent over time and across similar measures. Some gender differences were evident. Conclusions: The findings confirm the difficulties that children with ID may have when responding to self-report measures of mental health, and suggest that care should be taken in choice of instruments. While modifications can produce small improvements, it is clear that more robust measures of mental health are needed for persons with ID.

A survey to evaluate the implementation of a national clinical assessment form

Purpose: The Australian Universities Radiation Therapy Student Clinical Assessment Form (AURTSCAF) was designed to assess the clinical skills of radiation therapy (RT) students from the six universities that offer entry level RT programs. Given the AURTSCAF has now been in use for over two years, the Radiation Therapy Program Coordinators (RTPC) group initiated a post implementation evaluation survey. This formed the final phase of the AURTSCAF project and was funded by the Radiation Oncology Division of the Department of Health and Ageing. Methods: A cross-sectional designed survey using purposive sampling was distributed via email to all RT clinical sites. The survey asked questions about the requirements of a pass grade for students at different stages of their program, and the addition of a new category of assessment related to fitness to practise. Response types included both forced choice closed ended responses and open ended responses. There was also a section for open comments about the AURTSCAF. Results: There were 100 responses (55%) from clinicians who had utilised the assessment form over the previous 12 month period. Responses highlighted several positives with regard to the utility and implementation of the form. Comments regarding areas for improvement with the standardisation of the grading of students and consensus for the addition of a new domain in fitness for practise have informed the recommended changes proposed for 2012. Conclusion: This evaluation has provided a representative sample of the views of clinicians involved in assessing students on clinical placement. Recommendations include the addition of the sixth domain of assessment: Fitness for practise, the addition of descriptors and prompts for this domain in the user guide, the addition of a consensus statement about the use of the rating scale and dissemination of the proposed changes nationally.