Communicating assessment criteria is not sufficient for influencing students' approaches to assessment tasks – perspectives from a differential equations class

This report presents the findings of an exploratory study into the perceptions held by students regarding the use of criterion-referenced assessment in an undergraduate differential equations class. Students in the class were largely unaware of the concept of criterion referencing and of the various interpretations that this concept has among mathematics educators. Our primary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. Quantitative data and qualitative feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, the manner in which they actually approached the assessment activity was not altered as a result of the use of explicitly communicated grading criteria.

Improving criteria and feedback in student assessment in law

Spurred on by both the 1987 Pearce Report1 and the general changes to higher education spawned by the “Dawkins revolution” from 1988, there has been much critical self-evaluation leading to profound improvements to the quality of teaching in Australian law schools.2 Despite the changes there are still areas of general law teaching practice which have lagged behind recent developments in our understanding of what constitutes high quality teaching. One such area is assessment criteria and feedback. The project Improving Feedback in Student Assessment in Law is an attempt to remedy this. It aims to produce a manual containing key principles for the design of assessment and the provision of feedback, with practical yet flexible ideas and illustrations which law teachers may adopt or modify. Most of the examples have been developed by teachers at the University of Melbourne Law School. The project was supported in 1996 by a Committee for the Advancement of University Teaching grant and the manual will be published late in 1997.3 This note summarises the core principles which are elaborated further in the manual.

International review of environmental assessment tools and databases

Existing widely known environmental assessment models, primarily those for Life Cycle Assessment of manufactured products and buildings, were reviewed to grasp their characteristics, since the past several years have seen a significant increase in interest and research activity in the development of building environmental assessment methods. Each method or tool was assessed under the headings of description, data requirement, end-use, assessment criteria (scale of assessment and scoring/ weighting system)and present status

The refractive folio : exegesis and assessment in the open studio

Folio submission is universally regarded as the most appropriate means for measuring a student’s performance in the studio. However, developing meaningful and defensible assessment criteria is persistent challenge for all tertiary art educators. In discipline-based studios, the parameters provided by medium and technique provide useful points of reference for assessing creative performance. But how can student performance be evaluated when there is no discipline-based framework to act as a point of reference? The ‘open’ studio approach to undergraduate teaching presents these and other pedagogical challenges. This paper discusses the innovative approaches to studio-based teaching and assessment at QUT. Vital to the QUT open studio model is the studio rationale – an exegetical document that establishes an individualised theoretical framework through which a student’s understandings can be, in part, evaluated. This paper argues that the exegetical folio effectively reconciles the frequently divergent imperatives of creative, professional and academic skills, while retaining the centrality of the studio as a site for the production of new material, processual and conceptual understandings.

Towards a classification of criterion referenced assessment models in mathematics courses : student and academic perspectives

We present the findings of a study into the implementation of explicitly criterion- referenced assessment in undergraduate courses in mathematics. We discuss students' concepts of criterion referencing and also the various interpretations that this concept has among mathematics educators. Our primary goal was to move towards a classification of criterion referencing models in quantitative courses. A secondary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. The data and feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, it did not alter the way the actually approached the assessment activity.

Explicit, latent and meta-criteria : types of criteria at play in professional judgement practice

This paper engages with debates about whether comprehensive prior specification of criteria and standards is sufficient for informed professional judgement. A preoccupation has emerged with the specificity and explication of criteria intended to regulate judgement. This has resulted in criteria-compliance in the use of defined standards to validate judgements and improve reliability and consistency. Compliance has become a priority, the consequence being the prominence of explicit criteria, to the lack of acknowledgement of the operation of latent and meta-criteria within judgement practice. This paper examines judgement as a process involving three categories of assessment criteria in the context of standards-referenced systems: explicit, latent and meta-criteria. These are understood to be wholly interrelated and interdependent. A conceptualisation of judgement involving the interplay of the three criteria types is presented with an exploration of how they function to focus or alter assessments of quality in judgements of achievement in English and Mathematics.

Designing online assessment tools for disengaged youth

This article reports on the development of online assessment tools for disengaged youth in flexible learning environments. Sociocultural theories of learning and assessment and Bourdieu’s sociological concepts of capital and exchange were used to design a purpose-built content management system. This design experiment engaged participants in assessment that led to the exchange of self, peer and teacher judgements for credentialing. This collaborative approach required students and teachers to adapt and amend social networking practices for students to submit and judge their own and others’ work using comments, ratings, keywords and tags. Students and teachers refined their evaluative expertise across contexts, and negotiated meanings and values of digital works, which gave rise to revised versions and emergent assessment criteria. By combining social networking tools with sociological models of capital, assessment activities related to students’ digital productions were understood as valuations and judgements within an emergent, negotiable social field of exchange.

A micro-level indexing model for the assessment of sustainable urban ecosystems

During the last several decades, the quality of natural resources and their services have been exposed to significant degradation from increased urban populations combined with the sprawl of settlements, development of transportation networks and industrial activities (Dorsey, 2003; Pauleit et al., 2005). As a result of this environmental degradation, a sustainable framework for urban development is required to provide the resilience of natural resources and ecosystems. Sustainable urban development refers to the management of cities with adequate infrastructure to support the needs of its population for the present and future generations as well as maintain the sustainability of its ecosystems (UNEP/IETC, 2002; Yigitcanlar, 2010). One of the important strategic approaches for planning sustainable cities is „ecological planning‟. Ecological planning is a multi-dimensional concept that aims to preserve biodiversity richness and ecosystem productivity through the sustainable management of natural resources (Barnes et al., 2005). As stated by Baldwin (1985, p.4), ecological planning is the initiation and operation of activities to direct and control the acquisition, transformation, disruption and disposal of resources in a manner capable of sustaining human activities with a minimum disruption of ecosystem processes. Therefore, ecological planning is a powerful method for creating sustainable urban ecosystems. In order to explore the city as an ecosystem and investigate the interaction between the urban ecosystem and human activities, a holistic urban ecosystem sustainability assessment approach is required. Urban ecosystem sustainability assessment serves as a tool that helps policy and decision-makers in improving their actions towards sustainable urban development. There are several methods used in urban ecosystem sustainability assessment among which sustainability indicators and composite indices are the most commonly used tools for assessing the progress towards sustainable land use and urban management. Currently, a variety of composite indices are available to measure the sustainability at the local, national and international levels. However, the main conclusion drawn from the literature review is that they are too broad to be applied to assess local and micro level sustainability and no benchmark value for most of the indicators exists due to limited data availability and non-comparable data across countries. Mayer (2008, p. 280) advocates that by stating "as different as the indices may seem, many of them incorporate the same underlying data because of the small number of available sustainability datasets". Mori and Christodoulou (2011) also argue that this relative evaluation and comparison brings along biased assessments, as data only exists for some entities, which also means excluding many nations from evaluation and comparison. Thus, there is a need for developing an accurate and comprehensive micro-level urban ecosystem sustainability assessment method. In order to develop such a model, it is practical to adopt an approach that uses a method to utilise indicators for collecting data, designate certain threshold values or ranges, perform a comparative sustainability assessment via indices at the micro-level, and aggregate these assessment findings to the local level. Hereby, through this approach and model, it is possible to produce sufficient and reliable data to enable comparison at the local level, and provide useful results to inform the local planning, conservation and development decision-making process to secure sustainable ecosystems and urban futures. To advance research in this area, this study investigated the environmental impacts of an existing urban context by using a composite index with an aim to identify the interaction between urban ecosystems and human activities in the context of environmental sustainability. In this respect, this study developed a new comprehensive urban ecosystem sustainability assessment tool entitled the „Micro-level Urban-ecosystem Sustainability IndeX‟ (MUSIX). The MUSIX model is an indicator-based indexing model that investigates the factors affecting urban sustainability in a local context. The model outputs provide local and micro-level sustainability reporting guidance to help policy-making concerning environmental issues. A multi-method research approach, which is based on both quantitative analysis and qualitative analysis, was employed in the construction of the MUSIX model. First, a qualitative research was conducted through an interpretive and critical literature review in developing a theoretical framework and indicator selection. Afterwards, a quantitative research was conducted through statistical and spatial analyses in data collection, processing and model application. The MUSIX model was tested in four pilot study sites selected from the Gold Coast City, Queensland, Australia. The model results detected the sustainability performance of current urban settings referring to six main issues of urban development: (1) hydrology, (2) ecology, (3) pollution, (4) location, (5) design, and; (6) efficiency. For each category, a set of core indicators was assigned which are intended to: (1) benchmark the current situation, strengths and weaknesses, (2) evaluate the efficiency of implemented plans, and; (3) measure the progress towards sustainable development. While the indicator set of the model provided specific information about the environmental impacts in the area at the parcel scale, the composite index score provided general information about the sustainability of the area at the neighbourhood scale. Finally, in light of the model findings, integrated ecological planning strategies were developed to guide the preparation and assessment of development and local area plans in conjunction with the Gold Coast Planning Scheme, which establishes regulatory provisions to achieve ecological sustainability through the formulation of place codes, development codes, constraint codes and other assessment criteria that provide guidance for best practice development solutions. These relevant strategies can be summarised as follows: • Establishing hydrological conservation through sustainable stormwater management in order to preserve the Earth’s water cycle and aquatic ecosystems; • Providing ecological conservation through sustainable ecosystem management in order to protect biological diversity and maintain the integrity of natural ecosystems; • Improving environmental quality through developing pollution prevention regulations and policies in order to promote high quality water resources, clean air and enhanced ecosystem health; • Creating sustainable mobility and accessibility through designing better local services and walkable neighbourhoods in order to promote safe environments and healthy communities; • Sustainable design of urban environment through climate responsive design in order to increase the efficient use of solar energy to provide thermal comfort, and; • Use of renewable resources through creating efficient communities in order to provide long-term management of natural resources for the sustainability of future generations.

Elements of Better Assessment for the Improvement of Learning

Building a community of shared practice at the classroom level calls for clarity about the important assessment capabilities and dispositions of teachers, especially when teachers are expected to take a direct focus on learning. In this chapter, we present new ways of thinking about teachers’ assessment literacies, offering a formulation of better assessment for the improvement of learning, including three elements, namely (i) assessment criteria and standards; (ii) the teacher’s professional judgment; and (iii) social moderation. The potential of the first element lies in teachers’ classroom practices that deliberately embed assessment criteria and standards in pedagogy in productive ways. The second element involves the engagement of teachers and students in judgment practice, that develops the understanding that judgment involves more than the application of explicit or stated criteria. More fundamental is the matter of how teachers bring to bear stated features of quality and other intellectual and experiential resources in arriving at judgment. That is to say, they range across and orient to explicit (stated), tacit (unstated) and meta-criteria in judgment making. These insights have direct relevance to teachers’ efforts to develop students’ own evaluative experience, especially as this involves students working with stated features of quality for self-assessment and peer-assessment purposes. Further, practices for social moderation are discussed, giving examples of good practice in moderation, how teachers experience moderation and the potential benefits of various types.

A criteria based approach for evaluating innovation commercialisation

This study attempts to provide a criteria-based approach that can be used to evaluate the potential for technology transfer and commercialisation of a new technology from university research. More specifically, this study offers the critical factors for assessing marketability and feasibility of an innovation for the commercialisation and technology transfer process. The Delphi technique has been used to refine and categorise assessment criteria identified from various models and frameworks that emerged from literature. Proposed categories of criteria that are found to be important in the evaluation and assessment of a new technology for the commercialisation purpose include: Technological Readiness; Legal and Regulatory; Social Benefits and Impact; Economic and Market Factors.

The automation of radiotherapy treatment planning assessment

Aim The assessment of treatment plans is an important component in the education of radiation therapists. The establishment of a grade for a plan is currently based on subjective assessment of a range of criteria. The automation of assessment could provide a number of advantages including faster feedback, reduced chance of human error, and simpler aggregation of past results. Method A collection of treatments planned by a cohort of 27 second year radiation therapy students were selected for quantitative evaluation. Treatment sites included the bladder, cervix, larynx, parotid and prostate, although only the larynx plans had been assessed in detail. The plans were designed with the Pinnacle system and exported using the DICOM framework. Assessment criteria included beam arrangement optimisation, volume contouring, target dose coverage and homogeneity, and organ-at-risk sparing. The in-house Treatment and Dose Assessor (TADA) software1 was evaluated for suitability in assisting with the quantitative assessment of these plans. Dose volume data were exported in per-student and per-structure data tables, along with beam complexity metrics, dose volume histograms, and reports on naming conventions. Results The treatment plans were exported and processed using TADA, with the processing of all 27 plans for each treatment site taking less than two minutes. Naming conventions were successfully checked against a reference protocol. Significant variations between student plans were found. Correlation with assessment feedback was established for the larynx plans. Conclusion The data generated could be used to inform the selection of future assessment criteria, monitor student development, and provide useful feedback to the students. The provision of objective, quantitative evaluations of plan quality would be a valuable addition to not only radiotherapy education programmes but also for staff development and potentially credentialing methods. New functionality within TADA developed for this work could be applied clinically to, for example, evaluate protocol compliance.

Finding order in chaos : a problem based collaborative assignment for 1300 students across 19 disciplines

The use of collaborative assignments for assessment is a risky undertaking for students and course designers. Yet the benefits, in terms of core learning outcomes, competencies, collaborative sense making and student involvement, suggest that the effort is worthwhile. Formal descriptions and rules do little to ameliorate the perception of risk and increased anxiety by students. (Ryan, 2007). BEB100 Introducing Professional Learning is a faculty-wide foundation unit with over 1300 students from 19 disciplines across the Faculty of the Built Environment and Engineering (“BEE”) at the Queensland University of Technology (“QUT”), Brisbane, Australia. Finding order in chaos outlines the approach and justification, assessment criteria, learning resources, teamwork tools, tutorial management, communication strategies, 2007-09 Student Learning Experience Survey results, annual improvements, findings and outcomes.

Capital Works Procurement: The Selection of a Building Procurement Method

The determination of the most appropriate procurement method for capital works projects is a challenging task for the Department of Housing and Works (DHW) and other Western Australian State Government Agencies because of the array of assessment criteria that are considered and the procurement methods that are available. A number of different procurement systems can be used to deliver capital works projects such a traditional, design and construct and management. Sub-classifications of these systems have proliferated and continue to emerge in response to market demands. The selection of an inappropriate procurement method may lead to undesirable project outcomes. To facilitate DHW in selecting an appropriate procurement method for its capital works projects, a six step procurement method selection process is presented. The characteristics of the most common forms of procurement method used in Australia are presented. Case studies where procurement methods have been used for specific types of capital works in Western Australia are offered to provide a reference point and learning opportunity for procurement method selection.

Decision - Support Software Tool Handbook

The road and transport industry in Australia and overseas has come a long way to understanding the impact of road traffic noise on the urban environment. Most road authorities now have guidelines to help assess and manage the impact of road traffic noise on noise-sensitive areas and development. While several economic studies across Australia and overseas have tried to value the impact of noise on property prices, decision-makers investing in road traffic noise management strategies have relatively limited historic data and case studies to go on. The perceived success of a noise management strategy currently relies largely on community expectations at a given time, and is not necessarily based on the analysis of the costs and benefits, or the long-term viability and value to the community of the proposed treatment options. With changing trends in urban design, it is essential that the 'whole-of-life' costs and benefits of noise ameliorative treatment options and strategies be identified and made available for decisionmakers in future investment considerations. For this reason, CRC for Construction Innovation Australia funded a research project, Noise Management in Urban Environments to help decision-makers with future road traffic noise management investment decisions. RMIT University and the Queensland Department of Main Roads (QDMR) have conducted the research work, in collaboration with the Queensland Department of Public Works, ARUP Pty Ltd, and the Queensland University of Technology. The research has formed the basis for the development of a decision-support software tool, and helped collate technical and costing data for known noise amelioration treatment options. We intend that the decision support software tool (DST) should help an investment decision-maker to be better informed of suitable noise ameliorative treatment options on a project-by-project basis and identify likely costs and benefits associated with each of those options. This handbook has been prepared as a procedural guide for conducting a comparative assessment of noise ameliorative options. The handbook outlines the methodology and assumptions adopted in the decision-support framework for the investment decision-maker and user of the DST. The DST has been developed to provide an integrated user-friendly interface between road traffic noise modelling software, the relevant assessment criteria and the options analysis process. A user guide for the DST is incorporated in this handbook.