Abstract concepts made real: A pilot study examining pedagogical approaches in thermodynamics tutorials

BACKGROUND OR CONTEXT Thermodynamics is a core concept for mechanical engineers yet notoriously difficult. Evidence suggests students struggle to understand and apply the core fundamental concepts of thermodynamics with analysis indicating a problem with student learning/engagement. A contributing factor is that thermodynamics is a ‘science involving concepts based on experiments’ (Mayhew 1990) with subject matter that cannot be completely defined a priori. To succeed, students must engage in a deep-holistic approach while taking ownership of their learning. The difficulty in achieving this often manifests itself in students ‘not getting’ the principles and declaring thermodynamics ‘hard’. PURPOSE OR GOAL Traditionally, students practice and “learn” the application of thermodynamics in their tutorials, however these do not consider prior conceptions (Holman & Pilling 2004). As ‘hands on’ learning is the desired outcome of tutorials it is pertinent to study methods of improving their efficacy. Within the Australian context, the format of thermodynamics tutorials has remained relatively unchanged over the decades, relying anecdotally on a primarily didactic pedagogical approach. Such approaches are not conducive to deep learning (Ramsden 2003) with students often disengaged from the learning process. Evidence suggests (Haglund & Jeppsson 2012), however, that a deeper level and ownership of learning can be achieved using a more constructivist approach for example through self generated analogies. This pilot study aimed to collect data to support the hypothesis that the ‘difficulty’ of thermodynamics is associated with the pedagogical approach of tutorials rather than actual difficulty in subject content or deficiency in students. APPROACH Successful application of thermodynamic principles requires solid knowledge of the core concepts. Typically, tutorial sessions guide students in this application. However, a lack of deep and comprehensive understanding can lead to student confusion in the applications resulting in the learning of the ‘process’ of application without understanding ‘why’. The aim of this study was to gain empirical data on student learning of both concepts and application, within thermodynamic tutorials. The approach taken for data collection and analysis was: - 1 Four concurrent tutorial streams were timetabled to examine student engagement/learning in traditional ‘didactic’ (3 weeks) and non-traditional (3 weeks). In each week, two of the selected four sessions were traditional and two non-traditional. This provided a control group for each week. - 2 The non-traditional tutorials involved activities designed to promote student-centered deep learning. Specific pedagogies employed were: self-generated analogies, constructivist, peer-to-peer learning, inquiry based learning, ownership of learning and active learning. - 3 After a three-week period, teaching styles of the selected groups was switched, to allow each group to experience both approaches with the same tutor. This also acted to mimimise any influence of tutor personality / style on the data. - 4 At the conclusion of the trial participants completed a ‘5 minute essay’ on how they liked the sessions, a small questionnaire, modelled on the modified (Christo & Hoang, 2013)SPQ designed by Biggs (1987) and a small formative quiz to gauge the level of learning achieved. DISCUSSION Preliminary results indicate that overall students respond positively to in class demonstrations (inquiry based learning), and active learning activities. Within the active learning exercises, the current data suggests students preferred individual rather than group or peer-to-peer activities. Preliminary results from the open-ended questions such as “What did you like most/least about this tutorial” and “do you have other comments on how this tutorial could better facilitate your learning”, however, indicated polarising views on the nontraditional tutorial. Some student’s responded that they really like the format and emphasis on understanding the concepts, while others were very vocal that that ‘hated’ the style and just wanted the solutions to be presented by the tutor. RECOMMENDATIONS/IMPLICATIONS/CONCLUSION Preliminary results indicated a mixed, but overall positive response by students with more collaborative tutorials employing tasks promoting inquiry based, peer-to-peer, active, and ownership of learning activities. Preliminary results from student feedback supports evidence that students learn differently, and running tutorials focusing on only one pedagogical approached (typically didactic) may not be beneficial to all students. Further, preliminary data suggests that the learning / teaching style of both students and tutor are important to promoting deep learning in students. Data collection is still ongoing and scheduled for completion at the end of First Semester (Australian academic calendar). The final paper will examine in more detail the results and analysis of this project.

Pedagogy of feedback on student academic writing : research supervisor practices for the end of the research degree candidature

All academic writing is advanced with the benefit of feedback about the writing. In the case of the academic writing genres of the research proposal and the dissertation, feedback is usually provided by the research supervisor. Given that academic writing development is a process, and in the case of the research proposal and dissertation, writing which develops over time, it seems likely that the nature of feedback on drafts written early in the candidature may be different from feedback provided by the research supervisor later in a student’s candidature. ----- ----- When a research supervisor has been reading a student’s writing over a period of time, their own familiarity with the writing generates a risk to their ability to provide critical and objective feedback. Particularly by the end of a student’s candidature, the research supervisor’s familiarity with the work may cause them to miss elements of writing improvement. ----- ----- The author, as a research supervisor, has developed a feedback grid to facilitate feedback on the final drafts of a dissertation. This feedback grid is generated by the embedded promises in the early sections of the dissertation, which are then used to audit the content of the final sections of the dissertation to ascertain whether promises made have been fulfilled. This provides a strategy for the research supervisor to step back from the work and read the dissertation with the agenda of a dissertation examiner. ----- ----- The grid is one strategy within a broader pedagogy of providing feedback on writing samples.

The role of radiotherapy patients in provision of student interpersonal skills feedback

Background At Queensland University of Technology, student radiation therapists receive regular feedback from clinical staff relating to clinical interpersonal skills. Although this is of great value, there is anecdotal evidence that students communicate differently with patients when under observation. Purpose The aim of this pilot was to counter this perceived observer effect by allowing patients to provide students with additional feedback. Materials and methods Radiotherapy patients from two departments were provided with anonymous feedback forms relating to aspects of student interpersonal skills. Clinical assessors, mentors and students were also provided with feedback forms, including questions about the role of patient feedback. Patient perceptions of student performance were correlated with staff feedback and assessment scores. Results Results indicated that the feedback was valued by both students and patients. Students reported that the additional dimension focused them on communication, set goals for development and increased motivation. These changes derived from both feedback and study participation, suggesting that the questionnaires could be a useful teaching tool. Patients scored more generously than mentors, although there was agreement in relative grading. Conclusions The anonymous questionnaire is a convenient and valuable method of gathering patient feedback on students. Future iterations will determine the optimum timing for this method of feedback.

Improving criteria and feedback in student assessment in law

Spurred on by both the 1987 Pearce Report1 and the general changes to higher education spawned by the “Dawkins revolution” from 1988, there has been much critical self-evaluation leading to profound improvements to the quality of teaching in Australian law schools.2 Despite the changes there are still areas of general law teaching practice which have lagged behind recent developments in our understanding of what constitutes high quality teaching. One such area is assessment criteria and feedback. The project Improving Feedback in Student Assessment in Law is an attempt to remedy this. It aims to produce a manual containing key principles for the design of assessment and the provision of feedback, with practical yet flexible ideas and illustrations which law teachers may adopt or modify. Most of the examples have been developed by teachers at the University of Melbourne Law School. The project was supported in 1996 by a Committee for the Advancement of University Teaching grant and the manual will be published late in 1997.3 This note summarises the core principles which are elaborated further in the manual.

Enhanced student learning in accounting utilising web-based technology, peer-review feedback and reflective practices: A learning community approach to assessment

Higher education is becoming a major driver of economic competitiveness in an increasingly knowledge-driven global economy. Maintaining the competitive edge has seen an increase in public accountability of higher education institutions through the mechanism of ranking universities based on the quality of their teaching and learning outcomes. As a result, assessment processes are under scrutiny, creating tensions between standardisation and measurability and the development of creative and reflective learners. These tensions are further highlighted in the context of large undergraduate subjects, learner diversity and time-poor academics and students. Research suggests that high level and complex learning is best developed when assessment, combined with effective feedback practices, involves students as partners in these processes. This article reports on a four-phase, cross-institution and cross-discipline project designed to embed peer-review processes as part of the assessment in two large, undergraduate accounting classes. Using a social constructivist view of learning, which emphasises the role of both teacher and learner in the development of complex cognitive understandings, we undertook an iterative process of peer review. Successive phases built upon students’ feedback and achievements and input from language/learning and curriculum experts to improve the teaching and learning outcomes.

Feedback on student performance in the workplace: The role of workplace supervisors

This chapter highlights the im portance of feedback in work - integrated learning (WIL) , the key role of workplace supervisors, and the importance of continuous improvement in systems to support feedback processes. The paper proposes a definition of feedback and formative feedback, as well as approaches for providing industry feedback in W I L . It further reports on a case analysis based on workplace supervisors providing feedback to students in engineering and urban development , yielding certain insights into student pe rformance in the workplace, and m ore importantly, hi ghlighting the need to enhance the use of feedback processes. This is requir ed in a context where delivering feedback in WIL is generally acknowledge d to be complex, and where the role of the industry supervisor in appraising the performance of the student in the workplace needs to be very c learly defined in order for supervisor s ’ feedback to have optimal impact. F eedback in WIL i s set against the bac kdrop of recognizing the importance and complexity of stakeholder engagement in WIL in general, and the intricacy associated with the provision of feedback from industry supervisors in particular. Student self - assessment is briefly considered as a fu rther dimension of their participation in providing feedback on their own performance in the workplace . ( Asia - Pacific Journal of Cooperative Education, Special Issue, 2014, 15 (3 ), 241 - 25 2

'Take away from the dry sixties style marking': Lecturer and student perceptions and experiences of audio feedback

Providing audio feedback to assessment is relatively uncommon in higher education. However, published research suggests that it is preferred over written feedback by students but lecturers were less convinced. The aim of this paper is to examine further these findings in the context of a third year business ethics unit. Data was collected from two sources. The first is a series of in-depth, semi-structured interviews conducted with three lecturers providing audio feeback for the first time in Semester One 2011. The second source of data was drawn from the university student evaluation system. A total of 363 responses were used providing 'before' and 'after' perspectives about the effectiveness of audio feedback versus written feedback. Between 2005 and 2009 the survey data provided information about student attitudes to written assessment feedback (n=261). From 2010 onwards the data relates to audio (mp3) feedback (n=102). The analysis of he interview data indicated that introducing audio feedback should be done with care. The perception of the participating lecturers was mixed, ranging from sceptism to outright enthusiasm, but over time the overall approach became positive. It was found that particular attention needs to be paid to small (but important) technical details, and lecturers need to be convinced of its effectieness, especially that it is not necessarily more time consuming than providing written feedback. For students, the analysis revealed a clear preference for audio feedback. It is concluded that there is cause for concern and reason for optimism. It is a cause for concern because there is a possibility that scepticism on the part of academic staff seems to be based on assumptions about what students prefer and a concern about using the technology. There is reason for optimism because the evidence points towards students preferring audio feedback and as academic staff become more familiar with the technology the scepticism tends to evaporate. While this study is limited in scope, questions are raised about tackling negative staff perceptions of audio feedback that are worthy of further research.

Automated Feedback for 'Fill in the Gap' Programming Exercises

Timely feedback is a vital component in the learning process. It is especially important for beginner students in Information Technology since many have not yet formed an effective internal model of a computer that they can use to construct viable knowledge. Research has shown that learning efficiency is increased if immediate feedback is provided for students. Automatic analysis of student programs has the potential to provide immediate feedback for students and to assist teaching staff in the marking process. This paper describes a “fill in the gap” programming analysis framework which tests students’ solutions and gives feedback on their correctness, detects logic errors and provides hints on how to fix these errors. Currently, the framework is being used with the Environment for Learning to Programming (ELP) system at Queensland University of Technology (QUT); however, the framework can be integrated into any existing online learning environment or programming Integrated Development Environment (IDE)

Program Annotations: Feedback for Students Learning to Program

This paper presents a novel program annotation mechanism which enables students to obtain feedback from tutors on their programs in a far simpler and more efficient way than is possible with, for example, email. A common scenario with beginning students is to email tutors with copies of their malfunctioning programs. Unfortunately the emailed program often bears little resemblance to the program the student has been trying to make work; often it is incomplete, a different version and corrupted. We propose an annotation mechanism enabling students to simply and easily annotate their programs with comments asking for help. Similarly our mechanism enables tutors to view students’ programs and to reply to their comments in a simple and structured fashion. This means students can get frequent and timely feedback on their programs; tutors can provide such feedback efficiently, and hence students’ learning is greatly improved.

Harnessing assessment and feedback in the first year to support learning success, engagement and retention

It has been argued that intentional first year curriculum design has a critical role to play in enhancing first year student engagement, success and retention (Kift, 2008). A fundamental first year curriculum objective should be to assist students to make the successful transition to assessment in higher education. Scott (2006) has identified that ‘relevant, consistent and integrated assessment … [with] prompt and constructive feedback’ are particularly relevant to student retention generally; while Nicol (2007) suggests that ‘lack of clarity regarding expectations in the first year, low levels of teacher feedback and poor motivation’ are key issues in the first year. At the very minimum, if we expect first year students to become independent and self-managing learners, they need to be supported in their early development and acquisition of tertiary assessment literacies (Orrell, 2005). Critical to this attainment is the necessity to alleviate early anxieties around assessment information, instructions, guidance, and performance. This includes, for example:  inducting students thoroughly into the academic languages and assessment genres they will encounter as the vehicles for evidencing learning success; and  making expectations about the quality of this evidence clear. Most importantly, students should receive regular formative feedback of their work early in their program of study to aid their learning and to provide information to both students and teachers on progress and achievement. Leveraging research conducted under an ALTC Senior Fellowship that has sought to articulate a research-based 'transition pedagogy' (Kift & Nelson, 2005) – a guiding philosophy for intentional first year curriculum design and support that carefully scaffolds and mediates the first year learning experience for contemporary heterogeneous cohorts – this paper will discuss theoretical and practical strategies and examples that should be of assistance in implementing good assessment and feedback practices across a range of disciplines in the first year.

Changing relationships: activating student and staff engagement in a design studio

In the design studio learning environment, traditional student and staff expectations are of close contact teaching and learning. In recent years at QUT students have experienced reduced personal staff attention, and have increasingly felt “anonymous” and correspondingly disengaged, to the detriment of quality learning (Carbone 1998: 8; Biggs 2003). Concurrently, there has been a necessary increase in teaching by sessional staff at QUT with varied levels of experience and assurance. This paper outlines the first iteration of an action research project exploring whether changing the current QUT design studio student and staff relationships may lead to more engaged, dynamic learning environments. “Engagement” is understood as a primarily emotional, rather than operational student concern (Solomonides and Martin 2008; Austerlitz and Aravot 2007). The project inverted the standard QUT design studio teaching structure, and evaluated the new structure and activation of student engagement across four identified markers: attendance, participation, learning and performance (ACER 2009; NSSE 2005; Chapman 2003). Student and staff surveys and focus groups, corporate data, and informal feedback informed these evaluations. Overall, the results support the premise that when students and staff feel part of a reasonably-sized studio class with a dedicated lecturer and self-selected project, the majority are inclined to value these relationships, to feel actively engaged, and to experience some improvement in their learning and teaching performances.

Towards a classification of criterion referenced assessment models in mathematics courses : student and academic perspectives

We present the findings of a study into the implementation of explicitly criterion- referenced assessment in undergraduate courses in mathematics. We discuss students' concepts of criterion referencing and also the various interpretations that this concept has among mathematics educators. Our primary goal was to move towards a classification of criterion referencing models in quantitative courses. A secondary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. The data and feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, it did not alter the way the actually approached the assessment activity.

Closing the loop 21st century style : providing feedback on written assessment via MP3 recordings

Feedback on student performance, whether in the classroom or on written assignments, enables them to reflect on their understandings and restructure their thinking in order to develop more powerful ideas and capabilities. Research has identified a number of broad principles of good feedback practice. These include the provision of feedback that facilitates the development of reflection in learning; helps clarify what good performance is in terms of goals, criteria and expected standards; provides opportunities to close the gap between current and desired performance; delivers high quality information to students about their learning; and encourages positive motivational beliefs and self-esteem. However, high staff–student ratios and time pressures often result in a gulf between this ideal and reality. Whilst greater use of criteria referenced assessment has enabled an improvement in the extent of feedback being provided to students, this measure alone does not go far enough to satisfy the requirements of good feedback practice. Technology offers an effective and efficient means by which personalised feedback may be provided to students. This paper presents the findings of a trial of the use of the freely available Audacity program to provide individual feedback via MP3 recordings to final year Media Law students at the Queensland University of Technology on their written assignments. The trial has yielded wide acclaim by students as an effective means of explaining the exact reasons why they received the marks they were awarded, the things they did well and the areas needing improvement. It also showed that good feedback practice can be achieved without the burden of an increase in staff workload.

Development of a good practice guide to safeguard student learning engagement

This paper reports on the development of a good practice guide that will offer the higher education sector a framework for safeguarding student learning engagement. The good practice guide and framework are underpinned by a set of principles initially identified as themes in the social justice literature which were refined following the consolidation of data collected from eight selected “good practice” Australasian universities and feedback gathered at various forums and presentations. The good practice guide will provide the sector with examples of institutional wide efforts which respond to national priorities for student retention and will also provide exemplars of institutional practices for each principle to facilitate the uptake of sector-wide good practice. Participants will be provided with the opportunity to discuss the social justice principles, the draft good practice guide and identify the practical applications of the guide within individual institutions.