Feedback on student performance in the workplace: The role of workplace supervisors

This chapter highlights the im portance of feedback in work - integrated learning (WIL) , the key role of workplace supervisors, and the importance of continuous improvement in systems to support feedback processes. The paper proposes a definition of feedback and formative feedback, as well as approaches for providing industry feedback in W I L . It further reports on a case analysis based on workplace supervisors providing feedback to students in engineering and urban development , yielding certain insights into student pe rformance in the workplace, and m ore importantly, hi ghlighting the need to enhance the use of feedback processes. This is requir ed in a context where delivering feedback in WIL is generally acknowledge d to be complex, and where the role of the industry supervisor in appraising the performance of the student in the workplace needs to be very c learly defined in order for supervisor s ’ feedback to have optimal impact. F eedback in WIL i s set against the bac kdrop of recognizing the importance and complexity of stakeholder engagement in WIL in general, and the intricacy associated with the provision of feedback from industry supervisors in particular. Student self - assessment is briefly considered as a fu rther dimension of their participation in providing feedback on their own performance in the workplace . ( Asia - Pacific Journal of Cooperative Education, Special Issue, 2014, 15 (3 ), 241 - 25 2

Inertial sensor real-time feedback enhances the learning of cervical spine manipulation: A prospective study

Background Cervical Spinal Manipulation (CSM) is considered a high-level skill of the central nervous system because it requires bimanual coordinated rhythmical movements therefore necessitating training to achieve proficiency. The objective of the present study was to investigate the effect of real-time feedback on the performance of CSM. Methods Six postgraduate physiotherapy students attending a training workshop on Cervical Spine Manipulation Technique (CSMT) using inertial sensor derived real-time feedback participated in this study. The key variables were pre-manipulative position, angular displacement of the thrust and angular velocity of the thrust. Differences between variables before and after training were investigated using t-tests. Results There were no significant differences after training for the pre-manipulative position (rotation p = 0.549; side bending p = 0.312) or for thrust displacement (rotation p = 0.247; side bending p = 0.314). Thrust angular velocity demonstrated a significant difference following training for rotation (pre-training mean (sd) 48.9°/s (35.1); post-training mean (sd) 96.9°/s (53.9); p = 0.027) but not for side bending (p = 0.521). Conclusion Real-time feedback using an inertial sensor may be valuable in the development of specific manipulative skill. Future studies investigating manipulation could consider a randomized controlled trial using inertial sensor real time feedback compared to traditional training.

Profiles of illicit drug use during annual key holiday and control periods in Australia: Wastewater analysis in an urban, a semi-rural and a vacation area

AIMS: To examine changes in illicit drug consumption between peak holiday season (23 December-3 January) in Australia and a control period two months later in a coastal urban area, an inland semi-rural area and an island populated predominantly by vacationers during holidays. DESIGN: Analysis of representative daily composite wastewater samples collected from the inlet of the major wastewater treatment plant in each area. SETTING: Three wastewater treatment plants. PARTICIPANTS: Wastewater treatment plants serviced approximately 350, 000 persons in the urban area, 120,000 in the semi-rural area and 1100-2400 on the island. MEASUREMENTS: Drug residues were analysed using liquid chromatography coupled to a tandem mass spectrometer. Per capita drug consumption was estimated. Changes in drug use were quantified using Hedges' g. FINDINGS: During the holidays, cannabis consumption in the semi-rural area declined (g = -2.8) as did methamphetamine (-0.8), whereas cocaine (+1.5) and ecstasy (+1.6) use increased. In the urban area, consumption of all drugs increased during holidays (cannabis +1.6, cocaine +1.2, ecstasy +0.8 and methamphetamine +0.3). In the vacation area, methamphetamine (+0.7), ecstasy (+0.7) and cocaine (+1.1) use increased, but cannabis (-0.5) use decreased during holiday periods. CONCLUSIONS: While the peak holiday season in Australia is perceived as a period of increased drug use, this is not uniform across all drugs and areas. Substantial declines in drug use in the semi-rural area contrasted with substantial increases in urban and vacation areas. Per capita drug consumption in the vacation area was equivalent to that in the urban area, implying that these locations merit particular attention for drug use monitoring and harm minimisation measures.

Illicit drug use between an annual key holiday and a control period: Wastewater analysis in a semi-rural, an urban and a vacation area

Introduction and Aims: Holiday periods are potentially a time for increased substance use as social events and private parties are more common. Data on community illicit drug consumption during holiday periods are limited. Besides existing methods for determining drug use, such as population surveys, one emerging method is to measure illicit drugs and/or their metabolites in wastewater samples. This study examined the change in consumption of cannabis, methamphetamine, cocaine and 3,4- methylenedioxymethamphetamine in three different types of areas (an inland semi-rural area, a coastal urban area and a vacation island) with respect to holiday times. Design and Methods: Samples were collected at the inlet of the major wastewater treatment plant in each area during a key annual holiday (i.e. the summer holiday including Christmas and New Year) and control period. Illicit drug residues in the daily composited samples were measured by liquid chromatography coupled with tandem mass spectrometry. Results: Drug use varied substantially among the three areas within each monitoring period as well as between the holiday and control period within each area. Use consistently increased and peaked over New Year particularly for cocaine and 3,4-methylenedioxymethamphetamine whereas cannabis and methamphetamine were relatively less subjected to holiday times in all the areas. Discussion and Conclusions: Wastewater sampling and analysis provides higher spatio-temporal resolution than national surveys and supplements drug epidemiology studies originating primary in metropolitan locations. Such data is essential for policy makers to plan potential intervention strategies associated with these illicit substances in regional areas and other settings besides urban areas in the future.

Using quantitative wastewater analysis to measure daily usage of conventional and emerging illicit drugs at an annual music festival

Introduction and Aims Wastewater analysis provides a non-intrusive way of measuring drug use within a population. We used this approach to determine daily use of conventional illicit drugs [cannabis, cocaine, methamphetamine and 3,4-methylenedioxymethamphetamine (MDMA)] and emerging illicit psychostimulants (benzylpiperazine, mephedrone and methylone) in two consecutive years (2010 and 2011) at an annual music festival. Design and Methods Daily composite wastewater samples, representative of the festival, were collected from the on-site wastewater treatment plant and analysed for drug metabolites. Data over 2 years were compared using Wilcoxon matched-pair test. Data from 2010 festival were compared with data collected at the same time from a nearby urban community using equivalent methods. Results Conventional illicit drugs were detected in all samples whereas emerging illicit psychostimulants were found only on specific days. The estimated per capita consumption of MDMA, cocaine and cannabis was similar between the two festival years. Statistically significant (P < 0.05; Z = −2.0–2.2) decreases were observed in use of methamphetamine and one emerging illicit psychostimulant (benzyl piperazine). Only consumption of MDMA was elevated at the festival compared with the nearby urban community. Discussion and Conclusions Rates of substance use at this festival remained relatively consistent over two monitoring years. Compared with the urban community, drug use among festival goers was only elevated for MDMA, confirming its popularity in music settings. Our study demonstrated that wastewater analysis can objectively capture changes in substance use at a music setting without raising major ethical issues. It would potentially allow effective assessments of drug prevention strategies in such settings in the future.

Sediment concentration prediction and statistical evaluation for annual load estimation

We consider the development of statistical models for prediction of constituent concentration of riverine pollutants, which is a key step in load estimation from frequent flow rate data and less frequently collected concentration data. We consider how to capture the impacts of past flow patterns via the average discounted flow (ADF) which discounts the past flux based on the time lapsed - more recent fluxes are given more weight. However, the effectiveness of ADF depends critically on the choice of the discount factor which reflects the unknown environmental cumulating process of the concentration compounds. We propose to choose the discount factor by maximizing the adjusted R-2 values or the Nash-Sutcliffe model efficiency coefficient. The R2 values are also adjusted to take account of the number of parameters in the model fit. The resulting optimal discount factor can be interpreted as a measure of constituent exhaustion rate during flood events. To evaluate the performance of the proposed regression estimators, we examine two different sampling scenarios by resampling fortnightly and opportunistically from two real daily datasets, which come from two United States Geological Survey (USGS) gaging stations located in Des Plaines River and Illinois River basin. The generalized rating-curve approach produces biased estimates of the total sediment loads by -30% to 83%, whereas the new approaches produce relatively much lower biases, ranging from -24% to 35%. This substantial improvement in the estimates of the total load is due to the fact that predictability of concentration is greatly improved by the additional predictors.

Deriatives disclosure in corporate annual reports: bank analysts; perceptions of usefulness

Responding to mixed evidence on the decision-usefulness of annual report disclosures for derivative financial instruments to capital market participants, and concerns identified by practice, this paper examines usefulness in a direct study of user perceptions. Interviews with analysts from Australia’s four major banks reveal essential usefulness, limited by the disclosures’ failure to reflect companies’ actual use of derivatives throughout the period, and inability of users to understand companies’ off-balance sheet risk and risk management practices from information considered generic and boilerplate. The research complements and extends existing archival and survey research and provides new evidence suggesting low-cost ways for increasing usefulness. It supports the International Accounting Standards Board’s disclosure recommendations in its recent Discussion Paper: A Review of the Conceptual Framework for Financial Reporting, but, at the same time, highlights that for these proposed measures to be successful in relation to IFRS 7, they may need to address other issues. The research increases knowledge of the informational requirements of lenders, an important class of financial information user, and supports calls from practice for companies to improve their disclosure of material economic risks.

Laboratories transformation

BACKGROUND OR CONTEXT Laboratories provide the physical spaces for engineering students to connect with theory and have a personal hands-on learning experience. Learning space design and development is well established in many universities however laboratories are often not part of that movement. While active, collaborative and group learning pedagogies are all key words in relation to these new spaces the concepts have always been central to laboratory based learning. The opportunity to build on and strengthen good practice in laboratories is immense. In the 2001 review “Universities in Crisis” many references are made to the decline of laboratories. One such comment in the review was made by Professor Ian Chubb (AVCC), who in 2013, as Chief Scientist for Australia, identifies the national concern about STEM education and presents a strategic plan to address the challenges ahead. What has been achieved and changed in engineering teaching and research laboratories in this time? PURPOSE OR GOAL A large number of universities in Australia and New Zealand own laboratory and other infrastructure designed well for the era they were built but now showing signs of their age, unable to meet the needs of today’s students, limiting the effectiveness of learning outcomes and presenting very low utilisation rates. This paper will present a model for new learning space design that improves student experience and engagement, supporting academic aims and significantly raising the space utilisation rate. APPROACH A new approach in laboratory teaching and research including new management has been adopted by the engineering disciplines at QUT. Flexibility is an underpinning principle along with the modularisation of fixed teaching and learning equipment, high utilisation of spaces and dynamic pedagogical approaches. The revitalised laboratories and workshop facilities are used primarily for the engineering disciplines and increasingly for integrated use across many disciplines in the STEM context. The new approach was built upon a base of an integrated faculty structure from 2005 and realised in 2010 as an associated development with the new Science and Engineering Centre (SEC). Evaluation through student feedback surveys for practical activities, utilisation rate statistics and uptake by academic and technical staff indicate a very positive outcome. DISCUSSION Resulting from this implementation has been increased satisfaction by students, creation of social learning and connecting space and an environment that meets the needs and challenges of active, collaborative and group learning pedagogies. Academic staff are supported, technical operations are efficient and laboratories are effectively utilised. RECOMMENDATIONS/IMPLICATIONS/CONCLUSION Future opportunities for continuous improvement are evident in using the student feedback to rectify faults and improve equipment, environment and process. The model is easily articulated and visible to other interested parties to contribute to sector wide development of learning spaces.

Abstract concepts made real: A pilot study examining pedagogical approaches in thermodynamics tutorials

BACKGROUND OR CONTEXT Thermodynamics is a core concept for mechanical engineers yet notoriously difficult. Evidence suggests students struggle to understand and apply the core fundamental concepts of thermodynamics with analysis indicating a problem with student learning/engagement. A contributing factor is that thermodynamics is a ‘science involving concepts based on experiments’ (Mayhew 1990) with subject matter that cannot be completely defined a priori. To succeed, students must engage in a deep-holistic approach while taking ownership of their learning. The difficulty in achieving this often manifests itself in students ‘not getting’ the principles and declaring thermodynamics ‘hard’. PURPOSE OR GOAL Traditionally, students practice and “learn” the application of thermodynamics in their tutorials, however these do not consider prior conceptions (Holman & Pilling 2004). As ‘hands on’ learning is the desired outcome of tutorials it is pertinent to study methods of improving their efficacy. Within the Australian context, the format of thermodynamics tutorials has remained relatively unchanged over the decades, relying anecdotally on a primarily didactic pedagogical approach. Such approaches are not conducive to deep learning (Ramsden 2003) with students often disengaged from the learning process. Evidence suggests (Haglund & Jeppsson 2012), however, that a deeper level and ownership of learning can be achieved using a more constructivist approach for example through self generated analogies. This pilot study aimed to collect data to support the hypothesis that the ‘difficulty’ of thermodynamics is associated with the pedagogical approach of tutorials rather than actual difficulty in subject content or deficiency in students. APPROACH Successful application of thermodynamic principles requires solid knowledge of the core concepts. Typically, tutorial sessions guide students in this application. However, a lack of deep and comprehensive understanding can lead to student confusion in the applications resulting in the learning of the ‘process’ of application without understanding ‘why’. The aim of this study was to gain empirical data on student learning of both concepts and application, within thermodynamic tutorials. The approach taken for data collection and analysis was: - 1 Four concurrent tutorial streams were timetabled to examine student engagement/learning in traditional ‘didactic’ (3 weeks) and non-traditional (3 weeks). In each week, two of the selected four sessions were traditional and two non-traditional. This provided a control group for each week. - 2 The non-traditional tutorials involved activities designed to promote student-centered deep learning. Specific pedagogies employed were: self-generated analogies, constructivist, peer-to-peer learning, inquiry based learning, ownership of learning and active learning. - 3 After a three-week period, teaching styles of the selected groups was switched, to allow each group to experience both approaches with the same tutor. This also acted to mimimise any influence of tutor personality / style on the data. - 4 At the conclusion of the trial participants completed a ‘5 minute essay’ on how they liked the sessions, a small questionnaire, modelled on the modified (Christo & Hoang, 2013)SPQ designed by Biggs (1987) and a small formative quiz to gauge the level of learning achieved. DISCUSSION Preliminary results indicate that overall students respond positively to in class demonstrations (inquiry based learning), and active learning activities. Within the active learning exercises, the current data suggests students preferred individual rather than group or peer-to-peer activities. Preliminary results from the open-ended questions such as “What did you like most/least about this tutorial” and “do you have other comments on how this tutorial could better facilitate your learning”, however, indicated polarising views on the nontraditional tutorial. Some student’s responded that they really like the format and emphasis on understanding the concepts, while others were very vocal that that ‘hated’ the style and just wanted the solutions to be presented by the tutor. RECOMMENDATIONS/IMPLICATIONS/CONCLUSION Preliminary results indicated a mixed, but overall positive response by students with more collaborative tutorials employing tasks promoting inquiry based, peer-to-peer, active, and ownership of learning activities. Preliminary results from student feedback supports evidence that students learn differently, and running tutorials focusing on only one pedagogical approached (typically didactic) may not be beneficial to all students. Further, preliminary data suggests that the learning / teaching style of both students and tutor are important to promoting deep learning in students. Data collection is still ongoing and scheduled for completion at the end of First Semester (Australian academic calendar). The final paper will examine in more detail the results and analysis of this project.

Feedback and clinical practice improvement: A tool to assist workplace supervisors and students

In work integrated learning, students may report difficulties applying theory learned at university to clinical practice. One contributing factor may be students' inability to engage in meaningful reflection and self-correcting behaviours. This paper reports the evaluation of a tool, process and resources developed to assist students to reflect on feedback and engage in self-assessment. Students were assisted to develop self-assessment skills by reflecting on, and engaging with feedback from previous workplace experiences to develop goals, learning outcomes and strategies to improve performance with mostly positive results. A secondary aim was to identify common learning strategies or barriers that impacted on student outcomes. Four themes emerged from the qualitative data: 1) preparing for clinical learning; 2) relationships and engagement levels; 3) shared awareness, and; 4) developing clinical practice. Overall students felt the tool assisted them to narrow their attention on what needed to be improved. While supervisors believed the tool helped them to focus on specific needs of each student. Common barriers to clinical practice improvement related to a lack of opportunity in some settings, and lack of staff willingness to support students to achieve identified goals. Students and supervisors found the use of the tools beneficial and assisted students to demonstrate a greater understanding of how to apply feedback received to support their learning in the clinical environment.

Developing a moderation community of practice

This paper reports on a study to evaluate technology-based processes for assessment moderation. The aim was to evaluate standard features found in an institutional Learning Management System, and their compatibility with the values and practices of a large teaching team. The process used an online discussion board forum for tutors, the paring of more experienced tutors with those new to the process, and further meetings conducted in both face-to-face and web conferencing environments. Online rubrics were used for assessing student work and the provision of feedback. A focus group conducted after marking was concluded and the analysis of the discussion board forum demonstrated a strong community of practice with a shared understanding of assessment requirements.

Perceptions of audio feedback in higher education assessment

The aim of this paper is to present results of research investigating the effectiveness of audio feedback in a third year undergraduate unit. While there is a large and growing body of literature about providing assessment feedback, there is little focussing on the use of audio media. This study employs a mixed method approach, involving semi-structured interviews with academic staff and a survey of students. Analysis of the interview data suggests that there are a number of issues surrounding acceptance of using audio feedback by lecturers. The next stage of the study is to examine the extent to which lecturers change their perceptions as they use audio feedback and to analyse the perceptions of the students (n=120), including the perceived importance of feedback, the ways in which they used the audio feedback and the extent to which they believe they control events that affect them. Ultimately, this study seeks to provide recommendations appropriate to the implementation of audio feedback in higher education.

'Take away from the dry sixties style marking': Lecturer and student perceptions and experiences of audio feedback

Providing audio feedback to assessment is relatively uncommon in higher education. However, published research suggests that it is preferred over written feedback by students but lecturers were less convinced. The aim of this paper is to examine further these findings in the context of a third year business ethics unit. Data was collected from two sources. The first is a series of in-depth, semi-structured interviews conducted with three lecturers providing audio feeback for the first time in Semester One 2011. The second source of data was drawn from the university student evaluation system. A total of 363 responses were used providing 'before' and 'after' perspectives about the effectiveness of audio feedback versus written feedback. Between 2005 and 2009 the survey data provided information about student attitudes to written assessment feedback (n=261). From 2010 onwards the data relates to audio (mp3) feedback (n=102). The analysis of he interview data indicated that introducing audio feedback should be done with care. The perception of the participating lecturers was mixed, ranging from sceptism to outright enthusiasm, but over time the overall approach became positive. It was found that particular attention needs to be paid to small (but important) technical details, and lecturers need to be convinced of its effectieness, especially that it is not necessarily more time consuming than providing written feedback. For students, the analysis revealed a clear preference for audio feedback. It is concluded that there is cause for concern and reason for optimism. It is a cause for concern because there is a possibility that scepticism on the part of academic staff seems to be based on assumptions about what students prefer and a concern about using the technology. There is reason for optimism because the evidence points towards students preferring audio feedback and as academic staff become more familiar with the technology the scepticism tends to evaporate. While this study is limited in scope, questions are raised about tackling negative staff perceptions of audio feedback that are worthy of further research.

Web-based active learning and frequent feedback: Engaging first-year university students

Web-based technology is particularly well-suited to promoting active student involvement in the processes of learning. All students enrolled in a first-year educational psychology unit were required to complete ten weekly online quizzes, ten weekly student-generated questions and ten weekly student answers to those questions. Results of an online survey of participating students strongly support the viability and perceived benefits of such an instructional approach. Although students reported that the 30 assessments were useful and reasonable, the most common theme to emerge from the professional reflections of participating lecturers was that the marking of questions and answers was unmanageable.

Understanding video based parent training intervention for children with autism

This paper investigates the challenges of delivering parent training intervention for autism over video. We conducted a qualitative field study of an intervention, which is based on a well-established training program for parents of children with autism, called Hanen More Than Words. The study was conducted with a Hanen Certified speech pathologist who delivered video based training to two mothers, each with a son having autism. We conducted observations of 14 sessions of the intervention spanning 3 months along with 3 semi-structured interviews with each participant. We identified different activities that participants performed across different sessions and analysed them based upon their implications on technology. We found that all the participants welcomed video based training but they also faced several difficulties, particularly in establishing rapport with other participants, inviting equal participation, and in observing and providing feedback on parent-child interactions. Finally, we reflect on our findings and motivate further investigations by defining three design sensitivities of Adaptation, Group Participation, and Physical Setup.