Objectifying the Subjective : Assessment and Feedback in Creative Arts Studio Learning and Teaching

This paper examines the Assessment and Feedback aspects of Studio Teaching as Creative Arts pedagogy. Prompted by USQ’s newly offered Bachelor of Creative Arts (BCA), the author has developed an Assessment Matrix specifically designed to satisfy a number of imperatives, including: • ‘objectifying’ the subjective aspects of creative practice as assessable coursework/research • providing the means by which accurate, detailed, personalised and confidential feedback may be provided to students individually • providing consistent, accurate, meaningful assessment records for student, lecturer, and institution • ensuring consistency, continuity, and transparency of assessment processes and records to satisfy quality audits • minimising marking and assessment time, whilst maximising assessment integrity and depth • requiring only basic level skills and knowledge of a computer application already in common use (Microsoft Excel) • adaptability to a range of creative courses ‐ across disciplines This Assessment Matrix has been in development (and trialled) since January 2009.

A shared reading intervention with parents to enhance young children’s early literacy skills

Parental reading to children from an early age has been shown to enhance children’s emergent literacy skills. A pragmatic randomized controlled trial (RCT) was used to investigate the effects of two forms of shared reading interventions on children’s language and literacy skills. Parents of 80 preparatory year children from outer suburban schools of an Australian metropolitan city were trained to use shared reading strategies in an eight-week home intervention. Families were assigned to one of three groups: Dialogic Reading (DR), Dialogic Reading with the addition of Print Referencing (DR + PR), or an attention-matched control group. The sample comprised 42 boys and 38 girls ranging in age from 4.9 years to 6.3 years (M = 5.5, SD = 0.3). Data were collected at pre, post, and at three months follow-up. Measures assessed children’s oral language (receptive and expressive vocabulary), phonological awareness (rhyme, word completion), alphabet knowledge, and concepts about print. Analyses of change from pre to post showed significant effects for the DR and DR + PR groups compared to the control group on three of the six measures: expressive language, rhyme, and concepts about print. At 3-month follow-up assessment, the two intervention groups maintained significantly better performance on the measure of concepts of print only. At both time points, there were no group differences between the DR and DR+PR conditions. These findings illustrate the potential of a brief home focused intervention on promoting children’s emergent literacy.

Assessment Understood as Enabling

This chapter outlines a perspective of educational assessment as enabling, whereby the learner is central and assessment is focused on supporting the knowledge, skills and dispositions necessary for lifelong learning. It argues that better education for young people is achievable when educational policy and practice give priority to learning improvement, thereby making assessment for accountability a related, though secondary, concern. The chapter describes how this work of internationally recognized scholars brings together diverse perspectives and theoretical frameworks and, in so doing, provides readers with a range of ways to consider their pathway through the book. A ‘map’ and summaries of chapters suggest a reading according to a thematic approach, geographical setting, author/s profile or content purposes depending on the reader’s own priorities. A section on assessment past, present, and futures calls for a rebalancing of improvement and accountability goals, and for countries to be careful to avoid privileging large-scale testing over other forms of data about learning and achievement.

Retooling Chinese primary school teachers to use technology creatively to promote innovation and problem solving skills in science classrooms

This paper reports on the initial phase of a Professional Learning Program (PLP) undertaken by 100 primary school teachers in China that aimed to facilitate the development of adaptive expertise in using technology to facilitate innovative science teaching and learning such as that envisaged by the Chinese Ministry of Education’s (2010-2020) education reforms. Key principles derived from literature about professional learning and scaffolding of learning informed the design of the PLP. The analysis of data revealed that the participants had made substantial progress towards the development of adaptive expertise. This was manifested not only by advances in the participants’ repertoires of Subject Matter Knowledge and Pedagogical Content Knowledge but also in changes to their levels of confidence and identities as teachers. By the end of the initial phase of the PLP, the participants had coalesced into a professional learning community that readily engaged in the sharing, peer review, reuse and adaption, and collaborative design of innovative science learning and assessment activities. The findings from the study indicate that those engaged in the development of PLPs for teachers in China need to take cognizance of certain cultural factors and traditions idiosyncratic to the Chinese educational system. A set of revised principles is then presented to inform the future design and implementation of PLPs for teachers in China.

Measuring multidisciplinary team effectiveness in a ward-based healthcare setting : development of the Team Functioning Assessment Tool

Nontechnical skills relating to team functioning are vital to the effective delivery of patient care and safety. In this study, we develop a reliable behavioral marker tool for assessing nontechnical skills that are critical to the success of ward-based multidisciplinary healthcare teams. The Team Functioning Assessment Tool (TFAT) was developed and refined using a literature review, focus groups, card-sorting exercise, field observations, and final questionnaire evaluation and refinement process. Results demonstrated that Clinical Planning, Executive Tasks, and Team Relations are important facets of effective multidisciplinary healthcare team functioning. The TFAT was also shown to yield acceptable inter-rater agreement.

Measuring ward-based multidisciplinary healthcare team functioning : a validation study of the Team Functioning Assessment Tool (TFAT)

The team functioning assessment tool (TFAT) has been shown to be a reliable behavioral marker tool for assessing nontechnical skills that are critical to the success of ward-based healthcare teams. This paper aims to refine and shorten the length of the TFAT to improve usability, and establish its reliability and construct validity. Psychometric testing based on 110 multidisciplinary healthcare teams demonstrated that the TFAT is a reliable and valid tool for measuring team members’ nontechnical skills in regards to Clinical Planning, Executive Tasks, and Team Functioning. Providing support for concurrent validity, high TFAT ratings were predicted by low levels of organizational constraints and high levels of group potency. There was also partial support for the negative relationships between time pressure, leadership ambiguity, and TFAT ratings. The paper provides a discussion on the applicability of the tool for assessing multidisciplinary healthcare team functioning in the context of improving team effectiveness and patient safety for ward-based hospital teams.

Teaching and assessment of prescribing medicines : a multidisciplinary review

Background Prescribing is a complex task, requiring specific knowledge and skills, and the execution of effective, context-specific clinical reasoning. Prescribing errors can result in significant morbidity and mortality. For all professions with prescribing rights, a clear need exists to ensure students graduate with a well-defined set of prescribing skills, which will contribute to competent prescribing.

The role of radiotherapy patients in provision of student interpersonal skills feedback

Background At Queensland University of Technology, student radiation therapists receive regular feedback from clinical staff relating to clinical interpersonal skills. Although this is of great value, there is anecdotal evidence that students communicate differently with patients when under observation. Purpose The aim of this pilot was to counter this perceived observer effect by allowing patients to provide students with additional feedback. Materials and methods Radiotherapy patients from two departments were provided with anonymous feedback forms relating to aspects of student interpersonal skills. Clinical assessors, mentors and students were also provided with feedback forms, including questions about the role of patient feedback. Patient perceptions of student performance were correlated with staff feedback and assessment scores. Results Results indicated that the feedback was valued by both students and patients. Students reported that the additional dimension focused them on communication, set goals for development and increased motivation. These changes derived from both feedback and study participation, suggesting that the questionnaires could be a useful teaching tool. Patients scored more generously than mentors, although there was agreement in relative grading. Conclusions The anonymous questionnaire is a convenient and valuable method of gathering patient feedback on students. Future iterations will determine the optimum timing for this method of feedback.

Making sense of inconsistencies in psycho-educational assessment: A case report

The case is presented of an 8-year-old young boy who was referred for psycho-educational assessment because of difficulties with writing. The paper provides an example of the way in which a case unfolds as further assessment data become available, and describes a number of challenging aspects of the assessment process. In this case, dilemmas arose when test results were inconsistent at different time points, and when the results were inconsistent with clinical observations. The case report illustrates the ways in which practitioners can collaborate to make sense of such discrepancies, with each contributing a slightly different perspective or set of skills that, in combination, assist to better understand a child’s difficulties.

Integrating ‘role play’ in assessment to strengthen professional conduct and accountability of students : a pilot study

BACKGROUND In a process engineering setting, graduates are frequently allocated reviews of existing operations or required to scope new production processes by their supervisors with a view to improving or expanding on operations and overall productivity. These tasks may be carried out in teams and in consultation with the process engineer’s immediate line manager or a more experienced engineer, such as the Production or Maintenance Manager; ultimately reporting to senior management, which is frequently a non-engineer. Although professional skills development is part of engineering curricula, ‘professional conduct’ and ‘accountability’ required for dealing with peers and superiors in industry is not very well addressed at university. Consequently, upon graduation, many students are, in terms of knowledge and experience in this area, underprepared to work effectively in industry settings. PURPOSE The purpose of this study was to develop and implement a role-play scenario within a core 2nd year process engineering unit, so that students could gain knowledge, skills and experience in different aspects (and nuances) of professional conduct and accountability. DESIGN/METHOD In the role-play scenario, students worked in ‘engineering production teams’ to design a process for an iconic Queensland fruitcake and to present their solution and recommendations (culminating in a poster presentation) to an assessment panel consisting of staff, role-playing as, ‘production and plant managers’. Students were assessed on several areas, including professionalism using a criteria referenced assessment guide by a 3-member cross-disciplinary staff panel consisting of a Business Faculty lecturer, an engineer from industry and the lecturer of the Process Engineering unit. Professional conduct and accountability was gauged through direct questioning by the panel. Feedback was also sought from students on various aspects through a survey questionnaire after the role play activity at the end of semester. RESULTS Overall, the role play was very well performed with students achieving an average score of 79.3/100 (distinction grade). Professional conduct as assessed by panel was on average better than scores given for professional accountability (4.0 compared with 3.6 out of 5). Feedback from students indicated that the learning activities had contributed to their overall understanding of the content and the role of process engineers. Industry involvement was rated very highly as contributing to their learning at 4.8 (on Likert scale from 1 – 5) and the poster presentation was rated at 3.6. CONCLUSIONS This pilot study was successful in implementing a new assessment task for modelling professional conduct and accountability within a 2nd year core unit. This task incorporated a role-play activity and there was evidence to suggest that this and associated learning tasks were successful in broadening students’ understanding and skills in this area required for engineering practice. Following feedback given by students and staff, improvements will be made to the nature of the problem, how it is defined, its assessment, and the approach taken in the role-play scenario when the unit is offered in 2014.

Development of an online role-play assessment initiative for nursing students in their bioscience studies

Devising assessment tasks for large units that embrace academic goals of authenticity and assessment variety can be a challenge. We developed an online Role-Play Assessment Initiative for first year nursing students in bioscience. Students responded to a case study by preparing two role-play dialogues: as a nurse with the patient, and between two nurses. The aims were to assess whether the students could: 1) understand the underlying disease process (pathophysiology) and relate it to clinical practice; 2) use language appropriate for lay and medical conversation; and 3) apply information using active learning. We conducted a student survey using quantitative questions (Likert scale: 1=strongly disagree to 5=strongly agree), and qualitative questions. 65 completed surveys were received. 80% of respondents agreed (includes agree or strongly agree) that it was a useful way to learn and understand pathophysiology of the case study. 86% agreed that it was useful to apply pathophysiology from lectures to a clinical setting. Overall, students found it enjoyable, which is beneficial for enhanced student engagement, and agreed that it allowed them to work well in a group (74% and 85%, respectively). Most qualitative suggestions for improvement related to group work, despite the encouraging response to group work in quantitative questions. Most positive comments surrounded different communication with a nurse compared with a patient. These results demonstrate that students developed deeper understanding of pathophysiology through active learning and were able to expand their nursing career skills during the role-play. Learning using role-play to simulate the workforce has fostered active learning.

Teaching and assessment of prescribing medicines : how does pharmacy perform?

BACKGROUND Prescribing is a complex task, requiring specific knowledge and skills combined with effective, context-specific clinical reasoning. Prescribing errors can result in significant morbidity and mortality. For all professions with prescribing rights, a clear need exists to ensure students graduate with a well-defined set of prescribing skills, which will contribute to competent prescribing. AIM To describe the methods employed to teach and assess the principles of effective prescribing across five non-medical professions at Queensland University of Technology. METHOD The NPS National Prescribing Competencies Framework (PCF) was used as the prescribing standard. A curriculum mapping exercise was undertaken to determine how well the PCF was addressed across the disciplines of paramedic science, pharmacy, podiatry, nurse practitioner and optometry. Identified gaps in teaching and/or assessment were noted. RESULTS Prescribing skills and knowledge are taught and assessed using a range of methods across disciplines. A multi-modal approach is employed by all disciplines. The Pharmacy discipline uses more tutorial sessions to teach prescribing principles and relies less on case studies and clinical appraisal to assess prescribing when compared to other disciplines. Within the pharmacy discipline approximately 90% of the PCF competencies are taught and assessed. This compares favourably with the other disciplines. CONCLUSION Further work is required to establish a practical, effective approach to the assessment of prescribing competence especially between the university and clinical settings. Effective and reliable assessment of prescribing undertaken by students in diverse settings remains challenging.

The clinical utility of the Cornell Scale for Depression in Dementia as a routine assessment in nursing homes

Objective To examine the clinical utility of the Cornell Scale for Depression in Dementia (CSDD) in nursing homes. Setting 14 nursing homes in Sydney and Brisbane, Australia. Participants 92 residents with a mean age of 85 years. Measurements Consenting residents were assessed by care staff for depression using the CSDD as part of their routine assessment. Specialist clinicians conducted assessment of depression using the Semi-structured Clinical Diagnostic Interview for DSM-IV-TR Axis I Disorders for residents without dementia or the Provisional Diagnostic Criteria for Depression in Alzheimer Disease for residents with dementia to establish expert clinical diagnoses of depression. The diagnostic performance of the staff completed CSDD was analyzed against expert diagnosis using receiver operating characteristic (ROC) curves. Results The CSDD showed low diagnostic accuracy, with areas under the ROC curve being 0.69, 0.68 and 0.70 for the total sample, residents with dementia and residents without dementia, respectively. At the standard CSDD cutoff score, the sensitivity and specificity were 71% and 59% for the total sample, 69% and 57% for residents with dementia, and 75% and 61% for residents without dementia. The Youden index (for optimizing cut-points) suggested different depression cutoff scores for residents with and without dementia. Conclusion When administered by nursing home staff the clinical utility of the CSDD is highly questionable in identifying depression. The complexity of the scale, the time required for collecting relevant information, and staff skills and knowledge of assessing depression in older people must be considered when using the CSDD in nursing homes.