Overcoming inconsistencies in placement assessment : the case for developmental assessment centers

Placements are integral to many university courses and to increasing student employability skills. Nevertheless, several complications, such as the assessment of placement experiences which often go against the principles of procedural justice, may limit placement effectiveness. For example, procedures are not applied uniformly across students; and evaluations of intangible qualities are susceptible to biases. As a result, effort and learning can be compromised. This paper advocates the use of developmental assessment centers to help solve these shortcomings. Developmental assessment centers are often used in organizations to evaluate capabilities of individuals and to facilitate development. Participants complete a series of work related and standardized tasks. Multiple raters then utilize a systematic approach to evaluate participants on a range of competencies, and consequently present constructive feedback to facilitate learning. Therefore, developmental assessment center principles match the key determinants of procedural justice and thus overcome many problems with traditional placement assessments.

Development and implementation of the Australian universities radiation therapy student clinical assessment form

Purpose: Prior to 2009, one of the problems faced by radiation therapists who supervised and assessed students on placement in Australian clinical centres, was that each of the six Australian universities where Radiation Therapy (RT) programmes were conducted used different clinical assessment and reporting criteria. This paper describes the development of a unified national clinical assessment and reporting form that was implemented nationally by all six universities in 2009. Methods: A four phase methodology was used to develop the new assessment form and user guide. Phase 1 included university consensus around domains of student practice and assessment, and alignment with national competency standards; Phase 2 was a national consensus workshop attended by radiation therapists involved in student supervision and assessment; Phase 3 was an action research re-iterative Delphi technique involving two rounds of a mail-out to gain further expert consensus; and stage 4 was national piloting of the developed assessment form. Results: The new assessment form includes five main domains of practice and 19 sub-domain criteria which students are assessed against during placement. Feedback from the pilot centre participants was positive, with the new form being assessed to be comprehensive and complemented by the accompanying user guide. Conclusion: The new assessment form has improved both the formative and summative assessment of students on placement, as well as enhancing the quality of feedback to students and the universities. The new national form has high acceptance from the Australian universities and has been subject to wide review by the profession.

Are dietetics educators' attitudes to assessment a barrier to expanding placement opportunities? Results of a Delphi study

Aim Assessment of entry-level health professionals is complex, especially in the work-based setting, placing additional pressures on these learning environments. The present study aims to gain understanding and ideally consensus regarding the setting for assessment of all elements of competence for entry-level dietitians across Australia. Methods Seventy-five experienced academic and practitioner assessors were invited to participate in an online Delphi survey. The 166 entry-level performance criteria of the competency standards for dietitians formed the basis of the questions in the survey, with rating on which ones could be assessed in the practice setting, those which could be assessed in a classroom/university setting and which could be assessed in either setting. Forty-three of 75 invited assessors responded to the first round of the Delphi. A second modified survey was sent to the 43 participants with 34 responding. Results Consensus was achieved for the assessment setting for 86 (52%) of the performance criteria after two rounds of surveying. The majority of these performance criteria achieved consensus at round one (n = 44) and were deemed to be best assessed in the practice setting (n = 55). This study highlighted the perspectives of assessors and their preference for the work-based setting for assessment. Conclusions To reduce the focus on work-based settings as the only place for competence-based assessment of health professionals, there is a need to support individual and organisational change through challenging existing norms around assessment.

Potential utilisation of assessment centre methodology to enhance student placement outcomes, experiences and employability

Background: Placements are of particular importance due to the richness of learning associated with placement experiences and the wider links they represent to industry and the community. Students often refer to placements as the most significant, productive and memorable component of their training. Importantly, placements also provide the opportunity for students to become work ready, i.e., integrate their knowledge into a new set of employable skills and personal qualities. There is now increased emphasis on employability by employers and universities alike; in fact, it now represents a critical performance measure for Australian universities. Despite these key points, there appears to be inconsistencies in approaches to maximising work placements across and within Universities. Assessment centre methodology may represent a useful approach to standardise and optimise work placements for all stakeholders. Assessment centre methodology has been used successfully for selection purposes in industry for the past 50 years. However, their use as a developmental tool is less prominent. Furthermore, their application in the higher education setting, particularly in the context of placements and student development appears under researched.
Overview of issue: Only one published study was found that reported the use of a developmental assessment centre with a post-graduate sample. That study took place over 10 years ago when the concept of employability was still gaining recognition and work placements were less common. The current paper reviews this unique concept in the context of the existing literature and the current needs of Universities, employers and students.
Discussion: It is argued that the reconsideration of assessment centre methodology for development represents an innovative approach to consistently maximising work placement outcomes, experiences and employability.
Conclusions: Given the importance and increased use of placements, the application of assessment centre methodology within the placement curriculum warrants further research. This methodology represents a standardised approach for implementation within a range of placement programs to enhance student development, placement outcomes and employability.

Evaluation of the use of Assessment Centre methodology to enhance development planning, work placement outcomes and work readiness for postgraduate students : a pilot

Background: Placements as a form of Work Integrated Learning are widely recognised for the positive impact they have on improving student employability and work readiness. Students can maximise strengths, improve areas of weakness, and develop a strong understanding of the requirements of their chosen field within the confines of a well monitored and rich learning environment. Assessment Centres (ACs) are commonly used in corporate settings for recruitment, selection and more recently to provide developmental feedback to participants. Based on a recent literature review, the present the present project evaluates the application of AC methodology as a developmental tool within the placement milieu. The review, which is also included the current conference proceedings details the benefits of utilising the AC process forming the impetus for the present pilot (Sturre; von Treuer & Keele 2010).
Aims: The primary aim of the paper was to evaluate the application of AC methodology as a tool for measuring and subsequently enhancing professional competencies in a sample of postgraduate students in organisational psychology (n=15).
Method: A longitudinal design was utilised with numerous evaluation points from placement stakeholders. This paper presents the first wave of findings. Students undertook a range of activities, including an in-tray exercise, role play, written report, leaderless group discussion and a personality assessment. Comprehensive feedback was provided by organisational psychologists who also fulfil the role of placement co-ordinators. With the assistance of Placement Co-ordinators, students prepared development plans relating to the competencies identified as requiring development. These plans were to be addressed and progress monitored during consecutive placements.
Results: Initial perceptions gathered from students regarding the AC process were very encouraging. Performance
evaluations collected to date, as measured by behaviourally based ratings scales completed by the students themselves and their workplace supervisors illustrate the positive effect of this methodology. The rigour and comprehensive techniques offered by the methodology enabled students to focus on and improve areas identified for development.
Conclusions: It is important to note that the present design formed a pilot study and as mentioned was undertaken with a limited sample. Future implementation is planned with larger samples, enabling a more comprehensive analysis of the methodology. Nevertheless, the methodology appears to provide a much needed strategy for the assessment and ongoing development of students prior to and during work placements. The application provides early intervention enabling students to address development needs with input from both university and organisational stakeholders based on an established, standardised process.

Assessment of a WIN quality training demonstration project : phase II report : training and early placement experiences /

Prepared under: grant no. 51-11-78-03 from the Employment and Training Administration, U.S. Department of Labor.

Initial placement methods for Haitian adult ESOL students : the NYS place test plus profile data compared to subjective assessment

Few valid and reliable placement procedures are available to assess the English language proficiency of adults who enroll in English for Speakers of Other Languages (ESOL) programs. Whereas placement material exists for children and university ESOL students, the needs of students in adult community education programs have not been adequately addressed. Furthermore, the research suggests that a number of variables, such as, native language, age, prior schooling, length of residence, and employment are related to second language acquisition. Numerous studies contribute to our understanding of the relationship of these factors to second language acquisition of Spanish-speaking students. Again, there is a void in the research investigating the factors affecting second language acquisition and consequently, appropriate placement of Haitian Creole-speaking students. This study compared a standardized instrument, the NYS Place Test, used alone and in combination with a writing sample in English, to subjective judgement of a department coordinator for initial placement of Haitian adult ESOL students in a community education program. The study also investigated whether or not consideration of student profile data improved the accuracy of the test. Finally, the study sought to determine if a relationship existed between student profile data and those who withdrew from the program or did not enter a class after registering. Analysis of the data by crosstabulation and chi-square revealed that the standardized NYS Place Test was at least as accurate as subjective department coordinator placement and that one procedure could be substituted for li other. Although the writing sample in English improved accuracy of placement by the NYS test, the results were not significant. Of the profile variables, only length of residence was found to be significantly related to accuracy of placement using the NYS Place Test. The number of incorrect placements was higher for those students who lived in the host country from twenty-five to one hundred ten months. A post hoc analysis of NYS test scores according to level showed that those learners who placed in level three also had a significantly higher incidence of incorrect placements. No significant relationship was observed between the profile variables and those who withdrew from the program or registered but did not enter a class.

A collaborative model for learning and assessment of legal placements

This paper will report on the evaluation of a new undergraduate legal workplace unit, LWB421 Learning in Professional Practice. LWB421 was developed in response to the QUT’s strategic planning and a growing view that work experience is essential to developing the skills that law graduates need in order to be effective legal practitioners (Stuckey, 2007). Work integrated learning provides a context for students to develop their skills, to see the link between theory and practice and support students in making the transition from university to practice (Shirley, 2006). The literature in Australian legal education has given little consideration to the design of legal internship subjects (as distinct from legal clinic programs). Accordingly the design of placement subjects needs to be carefully considered to ensure alignment of learning objectives, learning tasks and assessment. Legal placements offer students the opportunity to develop their professional skills in practice, reflect on their own learning and job performance and take responsibility for their career development and planning. This paper will examine the literature relating to the design of placement subjects, particularly in a legal context. It will propose a collaborative model to facilitate learning and assessment of legal work placement subjects. The basis of the model is a negotiated learning contract between the student, workplace supervisor and academic supervisor. Finally the paper will evaluate the model in the context of LWB421. The evaluation will be based on data from surveys of students and supervisors and focus group sessions.

The development and evaluation of an innovative nursing practice model to improve undergraduate nursing students' competence in paediatric physical assessment

Introduction The purpose of this study was to develop, implement and evaluate the impact of an educational intervention, comprising an innovative model of clinical decisionmaking and educational delivery strategy for facilitating nursing students‘ learning and development of competence in paediatric physical assessment practices. Background of the study Nursing students have an undergraduate education that aims to produce graduates of a generalist nature who demonstrate entry level competence for providing nursing care in a variety of health settings. Consistent with population morbidity and health care roles, paediatric nursing concepts typically form a comparatively small part of undergraduate curricula and students‘ exposure to paediatric physical assessment concepts and principles are brief. However, the nursing shortage has changed traditional nursing employment patterns and new graduates form the majority of the recruitment pool for paediatric nursing speciality staff. Paediatric nursing is a popular career choice for graduates and anecdotal evidence suggests that nursing students who select a clinical placement in their final year intend to seek employment in paediatrics upon graduation. Although concepts of paediatric nursing are included within undergraduate curriculum, students‘ ability to develop the required habits of mind to practice in what is still regarded as a speciality area of practice is somewhat limited. One of the areas of practice where this particularly impacts is in paediatric nursing physical assessment. Physical assessment is a fundamental component of nursing practice and competence in this area of practice is central to nursing students‘ development of clinical capability for practice as a registered nurse. Timely recognition of physiologic deterioration of patients is a key outcome of nurses‘ competent use of physical assessment strategies, regardless of the practice context. In paediatric nursing contexts children‘s physical assessment practices must specifically accommodate the child‘s different physiological composition, function and pattern of clinical deterioration (Hockenberry & Barrera, 2007). Thus, to effectively manage physical assessment of patients within the paediatric practice setting nursing students need to integrate paediatric nursing theory into their practice. This requires significant information processing and it is in this process where students are frequently challenged. The provision of rules or models can guide practice and assist novice-level nurses to develop their capabilities (Benner, 1984; Benner, Hooper-Kyriakidis & Stannard, 1999). Nursing practice models are cognitive tools that represent simplified patterns of expert analysis employing concepts that suit the limited reasoning of the inexperienced, and can represent the =rules‘ referred to by Benner (1984). Without a practice model of physical assessment students are likely to be uncertain about how to proceed with data collection, the interpretation of paediatric clinical findings and the appraisal of findings. These circumstances can result in ad hoc and unreliable nursing physical assessment that forms a poor basis for nursing decisions. The educational intervention developed as part of this study sought to resolve this problem and support nursing students‘ development of competence in paediatric physical assessment. Methods This study utilised the Context Input Process Product (CIPP) Model by Stufflebeam (2004) as the theoretical framework that underpinned the research design and evaluation methodology. Each of the four elements in the CIPP model were utilised to guide discrete stages of this study. The Context element informed design of the clinical decision-making process, the Paediatric Nursing Physical Assessment model. The Input element was utilised in appraising relevant literature, identifying an appropriate instructional methodology to facilitate learning and educational intervention delivery to undergraduate nursing students, and development of program content (the CD-ROM kit). Study One employed the Process element and used expert panel approaches to review and refine instructional methods, identifying potential barriers to obtaining an effective evaluation outcome. The Product element guided design and implementation of Study Two, which was conducted in two phases. Phase One employed a quasiexperimental between-subjects methodology to evaluate the impact of the educational intervention on nursing students‘ clinical performance and selfappraisal of practices in paediatric physical assessment. Phase Two employed a thematic analysis and explored the experiences and perspectives of a sample subgroup of nursing students who used the PNPA CD-ROM kit as preparation for paediatric clinical placement. Results Results from the Process review in Study One indicated that the prototype CDROM kit containing the PNPA model met the predetermined benchmarks for face validity and the impact evaluation instrumentation had adequate content validity in comparison with predetermined benchmarks. In the first phase of Study Two the educational intervention did not result in statistically significant differences in measures of student performance or self-appraisal of practice. However, in Phase Two qualitative commentary from students, and from the expert panel who reviewed the prototype CD-ROM kit (Study One, Phase One), strongly endorsed the quality of the intervention and its potential for supporting learning. This raises questions regarding transfer of learning and it is likely that, within this study, several factors have influenced students‘ transfer of learning from the educational intervention to the clinical practice environment, where outcomes were measured. Conclusion In summary, the educational intervention employed in this study provides insights into the potential e-learning approaches offer for delivering authentic learning experiences to undergraduate nursing students. Findings in this study raise important questions regarding possible pedagogical influences on learning outcomes, issues within the transfer of theory to practice and factors that may have influenced findings within the context of this study. This study makes a unique contribution to nursing education, specifically with respect to progressing an understanding of the challenges faced in employing instructive methods to impact upon nursing students‘ development of competence. The important contribution transfer of learning processes make to students‘ transition into the professional practice context and to their development of competence within the context of speciality practice is also highlighted. This study contributes to a greater awareness of the complexity of translating theoretical learning at undergraduate level into clinical practice, particularly within speciality contexts.

The use of high dynamic range luminance mapping in the assessment, understanding and defining of visual issues in post occupancy building assessment

The international focus on embracing daylighting for energy efficient lighting purposes and the corporate sector’s indulgence in the perception of workplace and work practice “transparency” has spurned an increase in highly glazed commercial buildings. This in turn has renewed issues of visual comfort and daylight-derived glare for occupants. In order to ascertain evidence, or predict risk, of these events; appraisals of these complex visual environments require detailed information on the luminances present in an occupant’s field of view. Conventional luminance meters are an expensive and time consuming method of achieving these results. To create a luminance map of an occupant’s visual field using such a meter requires too many individual measurements to be a practical measurement technique. The application of digital cameras as luminance measurement devices has solved this problem. With high dynamic range imaging, a single digital image can be created to provide luminances on a pixel-by-pixel level within the broad field of view afforded by a fish-eye lens: virtually replicating an occupant’s visual field and providing rapid yet detailed luminance information for the entire scene. With proper calibration, relatively inexpensive digital cameras can be successfully applied to the task of luminance measurements, placing them in the realm of tools that any lighting professional should own. This paper discusses how a digital camera can become a luminance measurement device and then presents an analysis of results obtained from post occupancy measurements from building assessments conducted by the Mobile Architecture Built Environment Laboratory (MABEL) project. This discussion leads to the important realisation that the placement of such tools in the hands of lighting professionals internationally will provide new opportunities for the lighting community in terms of research on critical issues in lighting such as daylight glare and visual quality and comfort.

Assessment of online discussion forums for law students

This paper goes beyond the existing literature and explores the innovative topic of designing criterion-referenced assessment for online discussion forums. There are several benefits of embedding online discussion forums into subjects including engaging students in collaborative learning, and encouraging deeper analysis, critical thinking and reflection. Using the assessment principles of validity, reliability and transparency, this paper offers a range of practical strategies to tutors who plan to develop criterion-referenced assessment as opposed to norm-referenced assessment for online discussion forums, applies the assessment principles in the context of an undergraduate law subject, and exemplars a rubric for an online discussion forum in a work placement subject.

Bioimpedance in the assessment of unilateral lymphedema of a limb: the optimal frequency

Background: Bioimpedance techniques provide a reliable method of assessing unilateral lymphedema in a clinical setting. Bioimpedance devices are traditionally used to assess body composition at a current frequency of 50 kHz. However, these devices are not transferable to the assessment of lymphedema, as the sensitivity of measuring the impedance of extracellular fluid is frequency dependent. It has previously been shown that the best frequency to detect extracellular fluid is 0 kHz (or DC). However, measurement at this frequency is not possible in practice due to the high skin impedance at DC, and an estimate is usually determined from low frequency measurements. This study investigated the efficacy of various low frequency ranges for the detection of lymphedema. Methods and Results: Limb impedance was measured at 256 frequencies between 3 kHz and 1000 kHz for a sample control population, arm lymphedema population, and leg lymphedema population. Limb impedance was measured using the ImpediMed SFB7 and ImpediMed L-Dex® U400 with equipotential electrode placement on the wrists and ankles. The contralateral limb impedance ratio for arms and legs was used to calculate a lymphedema index (L-Dex) at each measurement frequency. The standard deviation of the limb impedance ratio in a healthy control population has been shown to increase with frequency for both the arm and leg. Box and whisker plots of the spread of the control and lymphedema populations show that there exists good differentiation between the arm and leg L-Dex measured for lymphedema subjects and the arm and leg L-Dex measured for control subjects up to a frequency of about 30 kHz. Conclusions: It can be concluded that impedance measurements above a frequency of 30 kHz decrease sensitivity to extracellular fluid and are not reliable for early detection of lymphedema.