Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.

Continuous and pedagogically sound assessment : A new economy of power relations in Australian Law schools

In recent decades, assessment practices within Australian law schools have moved from the overwhelming use of end-of-year closed-book examinations to an increase in the use of a wider range of techniques. This shift is often characterised as providing a ‘better’ learning environment for students, contributing more positively to their own ‘personal development’ within higher education, or, considered along the lines of critical legal thought, as ‘liberating’ them from the ‘conservatising’ and ‘indoctrinating’ effects of the power relations that operate in law schools. This paper seeks to render problematic such liberal-progressive narratives about these changes to law school assessment practices. It will do so by utilising the work of French historian and philosopher Michel Foucault on power, arguing that the current range of assessment techniques demonstrates a shift in the ‘economy’ of power relations within the law school. Rather than ‘liberating’ students from relations of power, these practices actually extend the power relations through which students are governed. This analysis is intended to inform legal education research and assessment practice by providing a far more nuanced conceptual framework than one that seeks to ‘free’ law students from these ‘repressive’ practices, or hopes to ‘objectively’ contribute to their ‘personal development’.

Intensive Case Management: A Critical Re-Appraisal of the Scientific Evidence for Effectiveness

Intensive Case Management (ICM) is widely claimed to be an evidence-based and cost effective program for people with high levels of disability as a result of mental illness. However, the findings of recent randomized controlled trials comparing ICM with ‘usual services’ suggest that both clinical and cost effectiveness of ICM may be weakening. Possible reasons for this, including fidelity of implementation, researcher allegiance effects and changes in the wider service environment within which ICM is provided, are considered. The implications for service delivery and research are discussed.

Hunters & gatherers : strategies for curriculum mapping and data collection for assurance of learning

Assurance of learning is a predominant feature in both quality enhancement and assurance in higher education. Assurance of learning is a process that articulates explicit program outcomes and standards, and systematically gathers evidence to determine the extent to which performance matches expectations. Benefits accrue to the institution through the systematic assessment of whole of program goals. Data may be used for continuous improvement, program development, and to inform external accreditation and evaluation bodies. Recent developments, including the introduction of the Tertiary Education and Quality Standards Agency (TEQSA) will require universities to review the methods they use to assure learning outcomes. This project investigates two critical elements of assurance of learning: 1. the mapping of graduate attributes throughout a program; and 2. the collection of assurance of learning data. An audit was conducted with 25 of the 39 Business Schools in Australian universities to identify current methods of mapping graduate attributes and for collecting assurance of learning data across degree programs, as well as a review of the key challenges faced in these areas. Our findings indicate that external drivers like professional body accreditation (for example: Association to Advance Collegiate Schools of Business (AACSB)) and TEQSA are important motivators for assuring learning, and those who were undertaking AACSB accreditation had more robust assurance of learning systems in place. It was reassuring to see that the majority of institutions (96%) had adopted an embedding approach to assuring learning rather than opting for independent standardised testing. The main challenges that were evident were the development of sustainable processes that were not considered a burden to academic staff, and obtainment of academic buy in to the benefits of assuring learning per se rather than assurance of learning being seen as a tick box exercise. This cultural change is the real challenge in assurance of learning practice.

Emergency health services : demand and service delivery models. Monograph 1: literature review and activity trends

Emergency Health Services (EHS), encompassing hospital-based Emergency Departments (ED) and pre-hospital ambulance services, are a significant and high profile component of Australia’s health care system and congestion of these, evidenced by physical overcrowding and prolonged waiting times, is causing considerable community and professional concern. This concern relates not only to Australia’s capacity to manage daily health emergencies but also the ability to respond to major incidents and disasters. EHS congestion is a result of the combined effects of increased demand for emergency care, increased complexity of acute health care, and blocked access to ongoing care (e.g. inpatient beds). Despite this conceptual understanding there is a lack of robust evidence to explain the factors driving increased demand, or how demand contributes to congestion, and therefore public policy responses have relied upon limited or unsound information. The Emergency Health Services Queensland (EHSQ) research program proposes to determine the factors influencing the growing demand for emergency health care and to establish options for alternative service provision that may safely meet patient’s needs. The EHSQ study is funded by the Australian Research Council (ARC) through its Linkage Program and is supported financially by the Queensland Ambulance Service (QAS). This monograph is part of a suite of publications based on the research findings that examines the existing literature, and current operational context. Literature was sourced using standard search approaches and a range of databases as well as a selection of articles cited in the reviewed literature. Public sources including the Australian Institute of Health and Welfare (AIHW), the Council of Ambulance Authorities (CAA) Annual Reports, Australian Bureau of Statistics (ABS) and Department of Health and Ageing (DoHA) were examined for trend data across Australia.

Quantitative fit assessment of a precontoured fracture fixation plate : its automation and an investigation on the borderline cases

Virtual methods to assess the fitting of a fracture fixation plate were proposed recently, however with limitations such as simplified fit criteria or manual data processing. This study aims to automate a fit analysis procedure using clinical-based criteria, and then to analyse the results further for borderline fit cases. Three dimensional (3D) models of 45 bones and of a precontoured distal tibial plate were utilized to assess the fitting of the plate automatically. A Matlab program was developed to automatically measure the shortest distance between the bone and the plate at three regions of interest and a plate-bone angle. The measured values including the fit assessment results were recorded in a spreadsheet as part of the batch-process routine. An automated fit analysis procedure will enable the processing of larger bone datasets in a significantly shorter time, which will provide more representative data of the target population for plate shape design and validation. As a result, better fitting plates can be manufactured and made available to surgeons, thereby reducing the risk and cost associated with complications or corrective procedures. This in turn, is expected to translate into improving patients' quality of life.

The development of a program to assist international nursing students to be prepared for clinical practice in the Australian health care context

Health education in Western Countries has grown considerably in the last decade and this has occurred for a number of reasons. Specifically Universities actively recruit International students as the health workforce becomes global; also it is much easier for students to move and study globally. Internationally there is a health workforce shortage and if students gain a degree in a reputable university their ability to work globally is improved significantly. However, when studying to practice in the health care field the student must undertake clinical practice in an acute or aged care setting. This can be a significant problem for students who are culturally and linguistically diverse in an English speaking country such as Australia. The issues that can arise stem from the language differences where communication, interpretation understanding and reading the cultural norms of the health care setting are major challenges for International students. To assist international students to be successful in their clinical education, an extra curriculum workshop program was developed to provide additional support. The program which runs twice each year includes on-campus interactive workshops that are complemented by targeted support provided for students and clinical staff who are supervising students’ practice experience in the workplace. As this is an English speaking country the workshop is based on practicing reading, writing, listening and speaking, as well as exploring basic health care concepts and cultural differences. This enables students to gain knowledge of and practice interpretation of cultural norms and expectations in a safe environment. This innovative series of interactive workshops in a highly student-centred learning environment combine education with role play and discussion with peers who are supported by culturally aware and competent Educators. Over the years it has been running, the program has been undertaken by an increasing number of students. In 2011, more than 100 students are expected to participate. Student evaluation of the program has confirmed that it has assisted the majority of them to be successful in their clinical studies. Effectiveness of the project is measured throughout the program and in follow up sessions. This ongoing information allows for continuous development of the program that serves to meet individual needs of the International student, the University and Service providers such as the hospitals. This feedback from students regarding their increased comprehension of the Australian colloquial Language, healthcare terminology, critical thinking and clinical skill development and a cultural awareness also enables them to maintain their feelings of self confidence and self esteem.

Integration of CENA practice standards for postgraduate emergency nursing assessment

The use of professional competency standards to assess postgraduate nursing student’s clinical performance has been in place since 2004, at the Queensland University of Technology, School of Nursing & Midwifery (SONAM) when the Graduate Certificate in Emergency Nursing degree commenced. Emergency nursing students were assessed in their workplace, using a Clinical Performance Appraisal Tool or CPAT which was based on the Australian College of Critical Care Nurses (ACCCN) Competency Standards. With the subsequent formation of a separate Emergency Nursing Course advisory group in 2007, there was a review of clinical assessment course component. The release of the 2008 CENA revised Practice Standards for the Emergency Nursing Specialist’s, led to the emergency nursing course advisory committee supporting the integration of the CENA practice standards for assessment of emergency nurses in preference to the less relevant ACCCN competency standards. The SONAM emergency nursing study area team commenced the phasing in and progression of the CENA practice standards across the two Graduate Certificate units, and Graduate Diploma and Master of Nursing (emergency) clinical major options in 2009. As some units undertaken in the degree are available to nurses in other disciplines a separate CPAT was devised for the clinical assessments according to speciality context. The team has had to carefully consider how the professional standards are integrated into the teaching and assessment of the unit and not just applied instead of the ACCCN competency standards. Professional standards for the emergency context has also helped tailor course content and learning outcomes to be relevant across a number of emergency nursing contexts in Australia. The assessment of the CPAT is undertaken at the workplace by QUT appointed clinical lecturers. Clinical lecturers need to apply and have suitable postgraduate qualification to undertake the position. The clinical lecturer support role is well established at QUT. The integration of the new CENA practice standards has necessitated a review of the postgraduate assessment of emergency nurses. A clinical lecturer workshop has been organised to review role, scope and how to utilise the new look CENA based CPAT, clinical assessment format.

Retrospective evaluation and prospective value-add : a review of R&D investment in Australia

This paper describes a lead project currently underway through Australia’s Sustainable Built Environment National Research Centre evaluating diffusion mechanisms and impacts of R&D investment in the Australian built environment. Through a retrospective analysis of R&D investment trends and industry outcomes, and a prospective assessment of industry futures using strategic foresighting, a future-focussed industry R&D roadmap and pursuant policy guidelines will be developed. This research aims to build new understandings and knowledge relevant to R&D funding strategies, research team formation and management, dissemination of outcomes and industry uptake. Each of these issues are critical due to: the disaggregated nature of the built environment industry; intense competition; limited R&D investment; and new challenges (e.g. IT, increased environmental expectations). This paper details the context within which this project is being undertaken and the research design. Findings of the retrospective analysis of past R&D investment in Australia will be presented at this conference.

The Biodex Isokinetic Dynamometer for knee strength assessment in children : advantages and limitations

Objective: To critically appraise the Biodex System 4 isokinetic dynamometer for strength assessment of children. Methods: Appraisal was based on experiences from two independent laboratories involving testing of 213 children. Issues were recorded and the manufacturer was consulted regarding appropriate solutions. Results: The dynamometer had insufficient height adjustment for alignment of the knee for some children, requiring the construction of padding to better fit the child within the dynamometer. Potential for entrapment of the non-testing leg was evident in the passive and eccentric modes and a leg bracket restraint was constructed. Automated gravity correction did not operate when protocols were linked or data was exported to an external device. Conclusions: Limitations were noted, some of which were applicable to knee strength testing in general and others which were specific to use with children. However, most of these obstacles could be overcome, making the Biodex System 4 suitable for assessment of knee strength in children.

Data flow analysis of embedded program expressions

Data flow analysis techniques can be used to help assess threats to data confidentiality and integrity in security critical program code. However, a fundamental weakness of static analysis techniques is that they overestimate the ways in which data may propagate at run time. Discounting large numbers of these false-positive data flow paths wastes an information security evaluator's time and effort. Here we show how to automatically eliminate some false-positive data flow paths by precisely modelling how classified data is blocked by certain expressions in embedded C code. We present a library of detailed data flow models of individual expression elements and an algorithm for introducing these components into conventional data flow graphs. The resulting models can be used to accurately trace byte-level or even bit-level data flow through expressions that are normally treated as atomic. This allows us to identify expressions that safely downgrade their classified inputs and thereby eliminate false-positive data flow paths from the security evaluation process. To validate the approach we have implemented and tested it in an existing data flow analysis toolkit.

An evaluation of the pattern and structure mathematics awareness program in the early school years

This paper reports a 2-year longitudinal study on the effectiveness of the Pattern and Structure Mathematical Awareness Program (PASMAP) on students’ mathematical development. The study involved 316 Kindergarten students in 17 classes from four schools in Sydney and Brisbane. The development of the PASA assessment interview and scale are presented. The intervention program provided explicit instruction in mathematical pattern and structure that enhanced the development of students’ spatial structuring, multiplicative reasoning, and emergent generalisations. This paper presents the initial findings of the impact of the PASMAP and illustrates students’ structural development.

Developing the Pattern and Structure Assessment (PASA) interview to inform early mathematics learning

The Pattern and Structure Mathematical Awareness Program(PASMAP) stems from a 2-year longitudinal study on students’ early mathematical development. The paper outlines the interview assessment the Pattern and Structure Assessment(PASA) designed to describe students’ awareness of mathematical pattern and structure across a range of concepts. An overview of students’ performance across items and descriptions of their structural development are described.

Closing the loop 21st century style : providing feedback on written assessment via MP3 recordings

Feedback on student performance, whether in the classroom or on written assignments, enables them to reflect on their understandings and restructure their thinking in order to develop more powerful ideas and capabilities. Research has identified a number of broad principles of good feedback practice. These include the provision of feedback that facilitates the development of reflection in learning; helps clarify what good performance is in terms of goals, criteria and expected standards; provides opportunities to close the gap between current and desired performance; delivers high quality information to students about their learning; and encourages positive motivational beliefs and self-esteem. However, high staff–student ratios and time pressures often result in a gulf between this ideal and reality. Whilst greater use of criteria referenced assessment has enabled an improvement in the extent of feedback being provided to students, this measure alone does not go far enough to satisfy the requirements of good feedback practice. Technology offers an effective and efficient means by which personalised feedback may be provided to students. This paper presents the findings of a trial of the use of the freely available Audacity program to provide individual feedback via MP3 recordings to final year Media Law students at the Queensland University of Technology on their written assignments. The trial has yielded wide acclaim by students as an effective means of explaining the exact reasons why they received the marks they were awarded, the things they did well and the areas needing improvement. It also showed that good feedback practice can be achieved without the burden of an increase in staff workload.

Transferring knowledge through peer-reviewed assessment : the creation of a community of practice and the threats to its survival

While it is generally accepted in the learning and teaching literature that assessment is the single biggest influence on how students approach their learning, assessment methods within higher education are generally conservative and inflexible. Constrained by policy and accreditation requirements and the need for the explicit articulation of assessment standards for public accountability purposes, assessment tasks can fail to engage students or reflect the tasks students will face in the world of practice. Innovative assessment design can simultaneously deliver program objectives and active learning through a knowledge transfer process which increases student participation. This social constructivist view highlights that acquiring an understanding of assessment processes, criteria and standards needs active student participation. Within this context, a peer-assessed, weekly, assessment task was introduced in the first “serious” accounting subject offered as part of an undergraduate degree. The positive outcomes of this assessment innovation was that student failure rates declined 15%, tutorial participation increased fourfold, tutorial engagement increased six-fold and there was a 100% approval rating for the retention of the assessment task. In contributing to the core conference theme of “seismic” shifts within higher education, in stark contrast to the positive student response, staff-related issues of assessment conservatism and the necessity of meeting increasing research commitments, threatened the assessment task’s survival. These opposing forces to change have the potential to weaken the ability of higher education assessment arrangements to adequately serve either a new generation of students or the sector's community stakeholders.