Evaluation of assessment in the context of work-based learning: Qualitative perspectives of new graduates

Aim Evaluation or assessment of competence is an important step to ensure the safety and efficacy of health professionals, including dietitians. Most competency-based assessment studies are focussed on valid and reliable methods of assessment for the preparation of entry-level dietitians, few papers have explored student dietitians’ perceptions of these evaluations. This study aimed to explore the perceptions of recent graduates from accredited nutrition and dietetics training programs in Australia. It also aimed to establish the relevance of competency-based assessment to adequately prepare them for entry-level work roles. Methods A purposive sample of newly-graduated dietitians with a range of assessment experiences and varied employment areas was recruited. A qualitative approach, using in-depth interviews with 13 graduates, with differing assessment experiences was undertaken. Graduates were asked to reflect upon their competency-based assessment experiences whilst a student. Data was thematically analysed by multiple authors. Results Four themes emerged from the data analysis: (i) Transparency and consistency are critical elements of work-based competency assessment. (ii) Students are willing to take greater responsibility in their assessment process. (iii) Work-based competency assessment prepares students for employment. (iv) The relationship between students and their assessors can impact on the student experience and their assessment performance. Conclusions Understanding this unique perspective of students can improve evaluation of future health professionals and assist in designing valid competency-based assessment approaches.

Implementation of sustainable evidence-based practice for the assessment and management of pain in residential aged care facilities

Pain is common in residential aged care facilities (RACFs). In 2005, the Australian Pain Society developed 27 recommendations for good practice in the identification, assessment, and management of pain in these settings. This study aimed to address implementation of the standards and evaluate outcomes. Five facilities in Australia participated in a comprehensive evaluation of RACF pain practice and outcomes. Pre-existing pain management practices were compared with the 27 recommendations, before an evidence-based pain management program was introduced that included training and education for staff and revised in-house pain-management procedures. Post-implementation audits evaluated the program's success. Aged care staff teams also were assessed on their reports of self-efficacy in pain management. The results show that before the implementation program, the RACFs demonstrated full compliance on 6 to 12 standards. By the project's completion, RACFs demonstrated full compliance with 10 to 23 standards and major improvements toward compliance in the remaining standards. After implementation, the staff also reported better understanding of the standards (p < .001) or of facility pain management guidelines (p < .001), increased confidence in therapies for pain management (p < .001), and increased confidence in their training to assess pain (p < .001) and recognize pain in residents with dementia who are nonverbal (p = .003). The results show that improved evidence-based practice in RACFs can be achieved with appropriate training and education. Investing resources in the aged care workforce via this implementation program has shown improvements in staff self-efficacy and practice.

Supporting residents’ expression of sexuality : the initial construction of a sexuality assessment tool for residential aged care facilities

Background: Sexuality is a key component of quality of life and well-being and a need to express one’s sexuality continues into old age. Staff and families in residential aged care facilities often find expressions of sexuality by residents, particularly those living with dementia, challenging and facilities often struggle to address individuals’ needs in this area. This paper describes the development of an assessment tool which enables residential aged care facilities to identify how supportive their organisation is of all residents’ expression of their sexuality, and thereby improve where required. Methods: Multi-phase design using qualitative methods and a Delphi technique. Tool items were derived from the literature and verified by qualitative interviews with aged care facility staff, residents and families. The final item pool was confirmed via a reactive Delphi process. Results: A final item pool of sixty-nine items grouped into seven key areas allows facilities to score their compliance with the areas identified as being supportive of older people’s expression of their sexuality in a residential aged care environment. Conclusions: The sexuality assessment tool (SexAT) guides practice to support the normalization of sexuality in aged care homes and assists facilities to identify where enhancements to the environment, policies, procedures and practices, information and education/training are required. The tool also enables facilities to monitor initiatives in these areas over time.

Regressive measures won’t help youth into work or training

The 2014 federal budget implemented a so-called crackdown on what Minister for Social Services Kevin Andrews calls young people who are content to “sit on the couch at home and pick up a welfare cheque”. The crackdown will change access to income support for people under 30 years of age. From January 1 2015, all young people seeking Newstart Allowance and Youth Allowance for the first time will be required to “demonstrate appropriate job search and participation in employment services support for six months before receiving payments”. Upon qualifying, recipients must then spend 25 hours per week in Work for the Dole in order to receive income support for a six-month period. What happens beyond this six months is unclear. What is clear is that these policy changes, together with the Minister’s accompanying statements, are informed by a deficit view of disadvantaged youth. It is a view that demonstrates how little politicians know or understand about these young peoples’ past circumstances.

How do experts think about statistics? Hints for improving undergraduate and postgraduate training

Experts are increasingly being called upon to quantify their knowledge, particularly in situations where data is not yet available or of limited relevance. In many cases this involves asking experts to estimate probabilities. For example experts, in ecology or related fields, might be called upon to estimate probabilities of incidence or abundance of species, and how they relate to environmental factors. Although many ecologists undergo some training in statistics at undergraduate and postgraduate levels, this does not necessarily focus on interpretations of probabilities. More accurate elicitation can be obtained by training experts prior to elicitation, and if necessary tailoring elicitation to address the expert’s strengths and weaknesses. Here we address the first step of diagnosing conceptual understanding of probabilities. We refer to the psychological literature which identifies several common biases or fallacies that arise during elicitation. These form the basis for developing a diagnostic questionnaire, as a tool for supporting accurate elicitation, particularly when several experts or elicitors are involved. We report on a qualitative assessment of results from a pilot of this questionnaire. These results raise several implications for training experts, not only prior to elicitation, but more strategically by targeting them whilst still undergraduate or postgraduate students.

Centroid training to achieve effective text classification

Traditional text classification technology based on machine learning and data mining techniques has made a big progress. However, it is still a big problem on how to draw an exact decision boundary between relevant and irrelevant objects in binary classification due to much uncertainty produced in the process of the traditional algorithms. The proposed model CTTC (Centroid Training for Text Classification) aims to build an uncertainty boundary to absorb as many indeterminate objects as possible so as to elevate the certainty of the relevant and irrelevant groups through the centroid clustering and training process. The clustering starts from the two training subsets labelled as relevant or irrelevant respectively to create two principal centroid vectors by which all the training samples are further separated into three groups: POS, NEG and BND, with all the indeterminate objects absorbed into the uncertain decision boundary BND. Two pairs of centroid vectors are proposed to be trained and optimized through the subsequent iterative multi-learning process, all of which are proposed to collaboratively help predict the polarities of the incoming objects thereafter. For the assessment of the proposed model, F1 and Accuracy have been chosen as the key evaluation measures. We stress the F1 measure because it can display the overall performance improvement of the final classifier better than Accuracy. A large number of experiments have been completed using the proposed model on the Reuters Corpus Volume 1 (RCV1) which is important standard dataset in the field. The experiment results show that the proposed model has significantly improved the binary text classification performance in both F1 and Accuracy compared with three other influential baseline models.

Trainee-therapists are not all equal: Examination of therapeutic efficiency, effectiveness and early client dropout after 12 months of clinical training

Objective Contemporary research demonstrates the feasibility of assessing therapeutic performance of trainee-therapists through the use of objective measures of client treatment outcome. Further, significant variation between individual therapists based on their client treatment outcomes has been demonstrated. This study sets out to determine whether a reliable composite measure of therapeutic efficiency, effectiveness and early dropout can be developed and used to objectively compare trainee-therapists against each other. Design and methods Treatment outcomes of 611 clients receiving treatment from 58 trainee-therapists enrolled in a professional training programme were tracked with the OQ-45.2 over a 6-year period to assess therapeutic efficiency, therapeutic effectiveness and early client dropout. Results Significant variation between trainee-therapists was observed for each index. Findings of a moderately strong correlation between therapeutic efficiency and effectiveness enabled the ranking of trainee-therapists based upon a composite measure of these indexes. A non-significant correlation was found between early client dropout and measures of therapeutic effectiveness and efficiency. Conclusions The findings stress the importance of utilizing objective measures to track the treatment outcomes. Despite all trainee-therapists being enrolled in the same training programme, significant variation between trainee-therapists' therapeutic efficiency and effectiveness was found to exist. Practitioner points Developing of potential benchmarking tools that enable trainee-therapists, supervisors and educational institutions to quickly assess therapeutic performance can become part of a holistic assessment of a trainee-therapist's clinical development. Despite an inherent optimistic belief that therapists do not cause harm, there appears to be a small and significant proportion of trainee-therapists who consistently evidence little therapeutic change. Considerable variability in trainee-therapists' therapeutic efficiency and effectiveness can exist in the one training programme. Early client dropout may not be associated with therapists' therapeutic effectiveness and efficiency.

A virtual radiation therapy workflow training simulation

Aim Simulation forms an increasingly vital component of clinical skills development in a wide range of professional disciplines. Simulation of clinical techniques and equipment is designed to better prepare students for placement by providing an opportunity to learn technical skills in a “safe” academic environment. In radiotherapy training over the last decade or so this has predominantly comprised treatment planning software and small ancillary equipment such as mould room apparatus. Recent virtual reality developments have dramatically changed this approach. Innovative new simulation applications and file processing and interrogation software have helped to fill in the gaps to provide a streamlined virtual workflow solution. This paper outlines the innovations that have enabled this, along with an evaluation of the impact on students and educators. Method Virtual reality software and workflow applications have been developed to enable the following steps of radiation therapy to be simulated in an academic environment: CT scanning using a 3D virtual CT scanner simulation; batch CT duplication; treatment planning; 3D plan evaluation using a virtual linear accelerator; quantitative plan assessment, patient setup with lasers; and image guided radiotherapy software. Results Evaluation of the impact of the virtual reality workflow system highlighted substantial time saving for academic staff as well as positive feedback from students relating to preparation for clinical placements. Students valued practice in the “safe” environment and the opportunity to understand the clinical workflow ahead of clinical department experience. Conclusion Simulation of most of the radiation therapy workflow and tasks is feasible using a raft of virtual reality simulation applications and supporting software. Benefits of this approach include time-saving, embedding of a case-study based approach, increased student confidence, and optimal use of the clinical environment. Ongoing work seeks to determine the impact of simulation on clinical skills.

A comparison of training methods to increase neck muscle strength

Objective To compare two neck strength training modalities. Background Neck injury in pilots flying high performance aircraft is a concern in aviation medicine. Strength training may be an effective means to strengthen the neck and decrease injury risk. Methods The cohort consisted of 32 age-height-weight matched participants, divided into two experimental groups; the Multi-Cervical Unit (MCU) and Thera-Band tubing groups (THER), and a control (CTRL) group. Ten weeks of training were undertaken and pre-and post isometric strength testing for all groups was performed on the MCU. Comparisons between the three groups were made using a Kruskal-Wallis test and effect sizes between the MCU and the THER groups and the THER and CTRL groups were also calculated. Results The MCU group displayed the greatest increase in isometric strength (flexion 64.4%, extension 62.9%, left lateral flexion 53.3%, right lateral flexion 49.1%) and differences were only statistically significant (p<0.05) when compared to the CTRL group. Increases in neck strength for the THER group were lower than that shown in the MCU group (flexion 42.0%, extension 29.9%, left lateral flexion 26.7%, right lateral flexion 24.1%). Moderate to large effect sizes were found between the MCU and THER as well as the THER and CTRL groups. Conclusions This study demonstrated that the MCU was the most effective training modality to increase isometric cervical muscle strength. Thera-Band tubing did however, produce moderate gains in isometric neck strength

A global perspective on psycho-educational assessment

For psychologists in less developed countries, psycho-educational assessment is often challenging due to a lack of specialist training and a scarcity of appropriate, psychometrically robust instruments. This paper focuses on school psychology and psycho-educational assessment in three countries: Bangladesh, China and Iran. Despite differences in demographic and cultural features, these countries share similar issues that restrict the practice of psycho-educational assessment. We conclude that it is important for psychologists in western countries to support professional training and testing practices in less developed countries.

Mixed pixel analysis and assessment for flood mapping

The most difficult operation in flood inundation mapping using optical flood images is to map the ‘wet’ areas where trees and houses are partly covered by water. This can be referred to as a typical problem of the presence of mixed pixels in the images. A number of automatic information extracting image classification algorithms have been developed over the years for flood mapping using optical remote sensing images, with most labelling a pixel as a particular class. However, they often fail to generate reliable flood inundation mapping because of the presence of mixed pixels in the images. To solve this problem, spectral unmixing methods have been developed. In this thesis, methods for selecting endmembers and the method to model the primary classes for unmixing, the two most important issues in spectral unmixing, are investigated. We conduct comparative studies of three typical spectral unmixing algorithms, Partial Constrained Linear Spectral unmixing, Multiple Endmember Selection Mixture Analysis and spectral unmixing using the Extended Support Vector Machine method. They are analysed and assessed by error analysis in flood mapping using MODIS, Landsat and World View-2 images. The Conventional Root Mean Square Error Assessment is applied to obtain errors for estimated fractions of each primary class. Moreover, a newly developed Fuzzy Error Matrix is used to obtain a clear picture of error distributions at the pixel level. This thesis shows that the Extended Support Vector Machine method is able to provide a more reliable estimation of fractional abundances and allows the use of a complete set of training samples to model a defined pure class. Furthermore, it can be applied to analysis of both pure and mixed pixels to provide integrated hard-soft classification results. Our research also identifies and explores a serious drawback in relation to endmember selections in current spectral unmixing methods which apply fixed sets of endmember classes or pure classes for mixture analysis of every pixel in an entire image. However, as it is not accurate to assume that every pixel in an image must contain all endmember classes, these methods usually cause an over-estimation of the fractional abundances in a particular pixel. In this thesis, a subset of adaptive endmembers in every pixel is derived using the proposed methods to form an endmember index matrix. The experimental results show that using the pixel-dependent endmembers in unmixing significantly improves performance.

The Scored Patient-Generated Subjective Global Assessment (PG-SGA) and its Association with Quality of Life in Ambulatory Patients Receiving Radiotherapy

OBJECTIVE: To evaluate the scored Patient-generated Subjective Global Assessment (PG-SGA) tool as an outcome measure in clinical nutrition practice and determine its association with quality of life (QoL). DESIGN: A prospective 4 week study assessing the nutritional status and QoL of ambulatory patients receiving radiation therapy to the head, neck, rectal or abdominal area. SETTING: Australian radiation oncology facilities. SUBJECTS: Sixty cancer patients aged 24-85 y. INTERVENTION: Scored PG-SGA questionnaire, subjective global assessment (SGA), QoL (EORTC QLQ-C30 version 3). RESULTS: According to SGA, 65.0% (39) of subjects were well-nourished, 28.3% (17) moderately or suspected of being malnourished and 6.7% (4) severely malnourished. PG-SGA score and global QoL were correlated (r=-0.66, P<0.001) at baseline. There was a decrease in nutritional status according to PG-SGA score (P<0.001) and SGA (P<0.001); and a decrease in global QoL (P<0.001) after 4 weeks of radiotherapy. There was a linear trend for change in PG-SGA score (P<0.001) and change in global QoL (P=0.003) between those patients who improved (5%) maintained (56.7%) or deteriorated (33.3%) in nutritional status according to SGA. There was a correlation between change in PG-SGA score and change in QoL after 4 weeks of radiotherapy (r=-0.55, P<0.001). Regression analysis determined that 26% of the variation of change in QoL was explained by change in PG-SGA (P=0.001). CONCLUSION: The scored PG-SGA is a nutrition assessment tool that identifies malnutrition in ambulatory oncology patients receiving radiotherapy and can be used to predict the magnitude of change in QoL.