In-training assessment - Its potential in enhancing clinical teaching

Context In-training assessment (ITA) has established its place alongside formative and summative assessment at both the undergraduate and postgraduate level. In this paper the authors aimed to identify those characteristics of ITA that could enhance clinical teaching. Methods A literature review and discussions by an expert working group at the Ninth Cambridge Conference identified the aspects of ITA that could enhance clinical teaching. Results The features of ITA identified included defining the specific benefits to the learner, teacher and institution, and highlighting the patient as the context for ITA and clinical teaching. The ‘mapping’ of a learner’s progress towards the clinical teaching objectives by using multiple assessments over time, by multiple observers in both a systematic and opportunistic way correlates with the incremental nature of reaching clinical competence. Conclusions The importance of ITA based on both direct and indirect evidence of what the learner actually does in the real clinical setting is emphasized. Particular attention is given to addressing concerns in the more controversial areas of assessor training, ratings and documentation for ITA. Areas for future research are also identified.

Facilitating unlearning during implementation of new technology

Purpose – One of the critical issues for change management, particularly in relation to the implementation of new technologies, is the existence of prior knowledge and established mental models which may hinder change efforts. Understanding unlearning and how it might assist during organizational change is a way to address this resistance. The purpose of this paper is to present research designed to identify specific factors that facilitate unlearning. Design/methodology/approach – Drawing together issues identified as potential influencers of unlearning, a survey questionnaire was developed and administered in an Australian corporation undergoing large-scale change due to the implementation of an enterprise information system. The results were analyzed to identify specific factors that impact on unlearning. Findings – Findings from this paper identify factors that hinder or help the unlearning process during times of change including understanding the need for change, the level of organizational support and training, assessment of the change, positive experience and informal support, the organization's history of change, individual's prior outlooks, and individuals' feelings and expectations. Research limitations/implications – The use of only one organization does not allow for comparisons between organizations of different sizes, cultures or industries and therefore extension of this research is recommended. Practical implications – For practitioners, this paper provides specific elements at both the level of individuals and the organization that need to be considered for optimal unlearning during times of change. Originality/value – Previous literature on unlearning has been predominantly conceptual and theoretical. These empirical findings serve to further an earlier model based on qualitative research into potential influencers of unlearning.

Intern underperformance is detected more frequently in emergency medicine rotations

Objective: To determine the frequency and nature of intern underperformance as documented on in-training assessment forms. Methods: A retrospective review of intern assessment forms from a 2 year period (2009–2010) was conducted at a tertiary referral hospital in Brisbane, Queensland. The frequency of interns assessed as ‘requiring substantial assistance’ and/or ‘requires further development’ on mid- or end-of-term assessment forms was determined. Forms were analysed by the clinical rotation, time of year and domain(s) of clinical practice in which underperformance was documented. Results: During 2009 and 2010 the overall documented incidence of intern underperformance was 2.4% (95% CI 1.5–3.9%). Clinical rotation in emergency medicine detected significantly more underperformance compared with other rotations (P < 0.01). Interns predominantly had difficulty with ‘clinical judgment and decision-making skills’, ‘time management skills’ and ‘teamwork and colleagues’ (62.5%, 55% and 32.5% of underperforming assessments, respectively). Time of the year did not affect frequency of underperformance. A proportion of 13.4% (95% CI 9.2–19.0%) of interns working at the institution over the study period received at least one assessment in which underperformance was documented. Seventy-six per cent of those interns who had underperformance identified by mid-term assessment successfully completed the term following remediation. Conclusion: The prevalence of underperformance among interns is low, although higher than previously suggested. Emergency medicine detects relatively more interns in difficulty than other rotations.

How should a vocational education and training course be evaluated?

The course evaluation process used by a large VET provider was evaluated using guidelines suggested by the course evaluation literature and feedback obtained from multiple stakeholders. A modified model is presented as an exemplar for course evaluation in the VET sector.

Authentic and conventional assessment in Singapore schools : an empirical study of teacher assignments and student work

This is an empirical examination of the quality of teacher assignments and student work in Singapore schools. Using a theoretical framework based on principles of authentic assessment and intellectual quality, two sets of criteria and scoring rubrics were developed for the training of expert teachers to judge the quality of assignments and student work. Following rigorous training, the inter-rater reliability of expert teacher scoring was high. Samples of teacher assignments and student work were collected in English, social studies, mathematics, and science subject areas from a random stratified sample of 30 elementary schools and 29 high schools. For both grade levels, there were significant differences for the authentic intellectual quality of teachers’ assignments by subject area. Likewise, the differences of authentic intellectual quality for student work were significant and varied by subject area. Subject area effect was large. The correlations between the quality of teachers’ assignment tasks and student work were strong and significant at both grade levels. Where teachers set more intellectually demanding tasks, students were more likely to generate work or artefacts judged to be of higher quality. The findings suggest that teacher professional development in authentic intellectual assessment task design can contribute to the improvement of student learning and performance. It is argued that this will be a key requisite of educational systems like Singapore that are seeking to expand pedagogy and student outcomes beyond a focus on factual and rote knowledge.

Competence in Intelligence Testing : A Training Model for Postgraduate Psychology Students

The assessment of intellectual ability is a core competency in psychology. The results of intelligence tests have many potential implications and are used frequently as the basis for decisions about educational placements, eligibility for various services, and admission to specific groups. Given the importance of intelligence test scores, accurate test administration and scoring are essential; yet there is evidence of unacceptably high rates of examiner error. This paper discusses competency and postgraduate training in intelligence testing and presents a training model for postgraduate psychology students. The model aims to achieve high levels of competency in intelligence testing through a structured method of training, practice and feedback that incorporates peer support, self-reflection and multiple methods for evaluating competency.

An assessment of the potential of family day care as a nutrition promotion setting in South Australia

Articles > Journals > Health journals > Nutrition & Dietetics: The Journal of the Dieticians Association of Australia articles > March 2003 Article: An assessment of the potential of Family Day Care as a nutrition promotion setting in South Australia. (Original Research). Article from:Nutrition & Dietetics: The Journal of the Dieticians Association of Australia Article date:March 1, 2003 Author:Daniels, Lynne A.; Franco, Bunny; McWhinnie, Julie-Anne CopyrightCOPYRIGHT 2006 Dietitians Association of Australia. This material is published under license from the publisher through the Gale Group, Farmington Hills, Michigan. All inquiries regarding rights or concerns about this content should be directed to customer service. (Hide copyright information) Related articles Ads by Google TAFE Child Care Courses Government accredited courses. Study anytime, anywhere. www.seeklearning.com.au Get Work in Child Care Certificate III Children's Services 4 Day Course + Take Home Assessment HBAconsult.com.au Abstract Objective: To assess the potential role of Family Day Care in nutrition promotion for preschool children. Design and setting: A questionnaire to examine nutrition-related issues and practices was mailed to care providers registered in the southern region of Adelaide, South Australia. Care providers also supplied a descriptive, qualitative recall of the food provided by parents or themselves to each child less than five years of age in their care on the day closest to completion of the questionnaire. Subjects: 255 care providers. The response rate was 63% and covered 643 preschool children, mean 4.6 (SD 2.8) children per carer. Results: There was clear agreement that nutrition promotion was a relevant issue for Family Day Care providers. Nutrition and food hygiene knowledge was good but only 54% of respondents felt confident to address food quality issues with parents. Sixty-five percent of respondents reported non-neutral approaches to food refusal and dawdling (reward, punishment, cajoling) that overrode the child's control of the amount eaten. The food recalls indicated that most children (> 75%) were offered fruit at least once. Depending on the hours in care, (0 to 4, 5 to 8, greater than 8 hours), 20%, 32% and 55%, respectively, of children were offered milk and 65%, 82% and 87%, respectively, of children were offered high fat and sugar foods. Conclusions: Questionnaire responses suggest that many care providers are committed to and proactive in a range of nutrition promotion activities. There is scope for strengthening skills in the management of common problems, such as food refusal and dawdling, consistent with the current evidence for approaches to early feeding management that promote the development of healthy food preferences and eating patterns. Legitimising and empowering care providers in their nutrition promotion role requires clear policies, guide lines, adequate pre- and in-service training, suitable parent materials, and monitoring.

Developing risk assessment tools for fleet settings : where to from here?

The costs of work-related crashes In Australia and overseas, fleet safety or work-related road safety is an issue gaining increased attention from researchers, organisations, road safety practitioners and the general community. This attention is primarily in response to the substantial physical, emotional and economic costs associated with work-related road crashes. The increased risk factors and subsequent costs of work-related driving are also now well documented in the literature. For example, it is noteworthy that research has demonstrated that work-related drivers on average report a higher level of crash involvement compared to personal car drivers (Downs et al., 1999; Kweon and Kockelman, 2003) and in particular within Australia, road crashes are the most common form of work-related fatalities (Haworth et al., 2000).

European advanced driver training programs : reasons for optimism

Post license advanced driver training programs in the US and early programs in Europe have often failed to accomplish their stated objectives because, it is suspected, that drivers gain self perceived driving skills that exceed their true skills—leading to increased post training crashes. The consensus from the evaluation of countless advanced driver training programs is that these programs are a detriment to safety, especially for novice, young, male drivers. Some European countries including Sweden, Finland, Austria, Luxembourg, and Norway, have continued to refine these programs, with an entirely new training philosophy emerging around 1990. These ‘post-renewal’ programs have shown considerable promise, despite various data quality and availability concerns. These programs share in common a focus on teaching drivers about self assessment and anticipation of risk, as opposed to teaching drivers how to master driving at the limits of tire adhesion. The programs focus on factors such as self actualization and driving discipline, rather than low level mastery of skills. Drivers are meant to depart these renewed programs with a more realistic assessment of their driving abilities. These renewed programs require considerable specialized and costly infrastructure including dedicated driver training facilities with driving modules engineered specifically for advanced driver training and highly structured curricula. They are conspicuously missing from both the US road safety toolbox and academic literature. Given the considerable road safety concerns associated with US novice male drivers in particular, these programs warrant further attention. This paper reviews the predominant features and empirical evidence surrounding post licensing advanced driver training programs focused on novice drivers. A clear articulation of differences between the renewed and current US advanced driver training programs is provided. While the individual quantitative evaluations range from marginally to significantly effective in reducing novice driver crash risk, they have been criticized for evaluation deficiencies ranging from small sample sizes to confounding variables to lack of exposure metrics. Collectively, however, the programs sited in the paper suggest at least a marginally positive effect that needs to be validated with further studies. If additional well controlled studies can validate these programs, a pilot program in the US should be considered.

An assessment system for evaluation of driving competencies

Automobiles have deeply impacted the way in which we travel but they have also contributed to many deaths and injury due to crashes. A number of reasons for these crashes have been pointed out by researchers. Inexperience has been identified as a contributing factor to road crashes. Driver’s driving abilities also play a vital role in judging the road environment and reacting in-time to avoid any possible collision. Therefore driver’s perceptual and motor skills remain the key factors impacting on road safety. Our failure to understand what is really important for learners, in terms of competent driving, is one of the many challenges for building better training programs. Driver training is one of the interventions aimed at decreasing the number of crashes that involve young drivers. Currently, there is a need to develop comprehensive driver evaluation system that benefits from the advances in Driver Assistance Systems. A multidisciplinary approach is necessary to explain how driving abilities evolves with on-road driving experience. To our knowledge, driver assistance systems have never been comprehensively used in a driver training context to assess the safety aspect of driving. The aim and novelty of this thesis is to develop and evaluate an Intelligent Driver Training System (IDTS) as an automated assessment tool that will help drivers and their trainers to comprehensively view complex driving manoeuvres and potentially provide effective feedback by post processing the data recorded during driving. This system is designed to help driver trainers to accurately evaluate driver performance and has the potential to provide valuable feedback to the drivers. Since driving is dependent on fuzzy inputs from the driver (i.e. approximate distance calculation from the other vehicles, approximate assumption of the other vehicle speed), it is necessary that the evaluation system is based on criteria and rules that handles uncertain and fuzzy characteristics of the driving tasks. Therefore, the proposed IDTS utilizes fuzzy set theory for the assessment of driver performance. The proposed research program focuses on integrating the multi-sensory information acquired from the vehicle, driver and environment to assess driving competencies. After information acquisition, the current research focuses on automated segmentation of the selected manoeuvres from the driving scenario. This leads to the creation of a model that determines a “competency” criterion through the driving performance protocol used by driver trainers (i.e. expert knowledge) to assess drivers. This is achieved by comprehensively evaluating and assessing the data stream acquired from multiple in-vehicle sensors using fuzzy rules and classifying the driving manoeuvres (i.e. overtake, lane change, T-crossing and turn) between low and high competency. The fuzzy rules use parameters such as following distance, gaze depth and scan area, distance with respect to lanes and excessive acceleration or braking during the manoeuvres to assess competency. These rules that identify driving competency were initially designed with the help of expert’s knowledge (i.e. driver trainers). In-order to fine tune these rules and the parameters that define these rules, a driving experiment was conducted to identify the empirical differences between novice and experienced drivers. The results from the driving experiment indicated that significant differences existed between novice and experienced driver, in terms of their gaze pattern and duration, speed, stop time at the T-crossing, lane keeping and the time spent in lanes while performing the selected manoeuvres. These differences were used to refine the fuzzy membership functions and rules that govern the assessments of the driving tasks. Next, this research focused on providing an integrated visual assessment interface to both driver trainers and their trainees. By providing a rich set of interactive graphical interfaces, displaying information about the driving tasks, Intelligent Driver Training System (IDTS) visualisation module has the potential to give empirical feedback to its users. Lastly, the validation of the IDTS system’s assessment was conducted by comparing IDTS objective assessments, for the driving experiment, with the subjective assessments of the driver trainers for particular manoeuvres. Results show that not only IDTS was able to match the subjective assessments made by driver trainers during the driving experiment but also identified some additional driving manoeuvres performed in low competency that were not identified by the driver trainers due to increased mental workload of trainers when assessing multiple variables that constitute driving. The validation of IDTS emphasized the need for an automated assessment tool that can segment the manoeuvres from the driving scenario, further investigate the variables within that manoeuvre to determine the manoeuvre’s competency and provide integrated visualisation regarding the manoeuvre to its users (i.e. trainers and trainees). Through analysis and validation it was shown that IDTS is a useful assistance tool for driver trainers to empirically assess and potentially provide feedback regarding the manoeuvres undertaken by the drivers.

iRAP Malaysia training course : decade of action for road safety

The International Road Assessment Program (iRAP) is a not-for-profit organisation that works in partnership with governments and non-government organisations in all parts of the world to make roads safe. The iRAP Malaysia pilot study on 3700km of road identified the potential to prevent 31,800 deaths and serious injuries over the next 20 years from proven engineering improvements. To help ensure the iRAP data and results are available to planners and engineers, iRAP, together with staff from the Centre for Accident Research and Road Safety – Queensland (CARRS-Q) and the Malaysian Institute of Road Safety Research (MIROS), developed a five-day iRAP training course that covers the background, theory and practical application of iRAP protocols, with a special focus on Malaysian case studies. Funding was provided by a competitive grant from the Australia-Malaysia Institute.

Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.