Reframe : QUT's Evaluation Framework 2014 Communications Plan

2014 is the year for embedding the Reframe: QUT’s Evaluation Framework as core business within the University, with the following providing an outline of the Learning and Teaching Unit’s (LTU) planned activities to support this implementation. LTU is continuing its commitment to intensive support and engagement with the academic and teaching focused community within QUT. The Academic Quality and Standards team have extended the existing communication and dissemination activities into a comprehensive communication plan for 2014, with major initiatives defined within this document. It should be noted that these activities are aligned to the discussions from the 2013 Integrated Management of Feedback (IMF) Steering Group and as endorsed by the University Learning and Teaching Committee at their meeting in February, 2014.

Improving criteria and feedback in student assessment in law

Spurred on by both the 1987 Pearce Report1 and the general changes to higher education spawned by the “Dawkins revolution” from 1988, there has been much critical self-evaluation leading to profound improvements to the quality of teaching in Australian law schools.2 Despite the changes there are still areas of general law teaching practice which have lagged behind recent developments in our understanding of what constitutes high quality teaching. One such area is assessment criteria and feedback. The project Improving Feedback in Student Assessment in Law is an attempt to remedy this. It aims to produce a manual containing key principles for the design of assessment and the provision of feedback, with practical yet flexible ideas and illustrations which law teachers may adopt or modify. Most of the examples have been developed by teachers at the University of Melbourne Law School. The project was supported in 1996 by a Committee for the Advancement of University Teaching grant and the manual will be published late in 1997.3 This note summarises the core principles which are elaborated further in the manual.

Framing student evaluations of university learning and teaching : discursive strategies and textual outcomes

Evaluation in higher education is an evolving social practice, that is, it involves what people, institutions and broader systems do and say, how they do and say it, what they value, the effects of these practices and values, and how meanings are ascribed. The textual products (verbal, written, visual, gestural) that inform and are produced by, for and through evaluative practices are important as they promulgate particular kinds of meanings and values in specific contexts. This paper reports on an exploratory study that sought to investigate, using discourse analysis, the types of evaluative practices that were ascribed value, and the student responses that ensued, in different evaluative instruments. Findings indicate that when a reflective approach is taken to evaluation, students’ responses are more considered, they interrogate their own engagement in the learning context and they are more likely to demonstrate reconstructive thought. These findings have implications for reframing evaluation as reflective learning.

CoWs, Google and online possibilities : creating digital posters to enhance student engagement

A new Bachelor of Science (BSc) course was introduced at Queensland University of Technology (QUT) in 2013 and focused on inquiry-based, collaborative and active learning. Two of the first year units required that students carry out a group poster assessment task. This poster provides a preliminary evaluation from an academic staff perspective of the assessment approach used, whereby students created digital posters to utilise the affordances of new learning spaces. The digital posters approach was first introduced to a group of academic staff from the Science and Engineering Faculty (SEF) in 2012 during a professional development program to explicitly develop skills and shared understandings of teaching in collaborative learning spaces (Steel & Andrews, 2012). Considerations were given to the pedagogical requirements of a poster assessment task, the affordances of the learning space and an identification of possible benefits of using Google Sites to create digital posters. Positive feedback from this group (as highlighted in the quotes shown) and subsequent approval from unit coordinators for two of the new first year BSc units meant that the approach was adopted for Semester 1, 2013 with approximately 360 students in each unit.

Teaching persuasive texts : building a language of evaluation through hedging and moderated intensification

We identify two persuasive writing techniques – hedging and intensification – that pose difficulty for students in the middle years. We use examples of student writing from 3000 work samples collected as part of a larger Australian Research Council Linkage Project, URLearning (2009–2013). To realise the effective power of rhetorical persuasion, students need to be explicitly taught a range of hedging techniques to use to their advantage, and an expanded lexicon that does not rely on intensifiers. Practical teaching tips are provided for teachers.

Final Report: University of Technology, Sydney Student Feedback Survey Review

This evaluation was commissioned by Martin Hanlon, Director of the Planning and Quality Unit of the University of Technology, Sydney (UTS) to investigate the Student Feedback Survey (SFS) system, engagement in stakeholder feedback and provide recommendations against the Terms of Reference.

An analysis of user engagement in student Facebook groups

Analysing the engagement of students in university-based Facebook groups can shed light on the nature of their learning experience and highlight leverage points to build on student success. While post-semester surveys and demographic participation data can highlight who was involved and how they subsequently felt about the experience, these techniques do not necessarily reflect real-time engagement. One way to gain insight into in-situ student experiences is by categorising the original posts and comments into predetermined frameworks of learning. This paper offers a systematic method of coding Facebook contributions within various engagement categories: motivation, discourse, cognition and emotive responses.

Scientists in Control: a progressive approach to preparing work ready graduates in QC, EQA and method evaluation

Introduction QC, EQA and method evaluation are integral to delivery of quality patient results. To ensure QUT graduates have a solid grounding in these key areas of practice, a theory-to-practice approach is used to progressively develop and consolidate these skills. Methods Using a BCG assay for serum albumin, each student undertakes an eight week project analysing two levels of QC alongside ‘patient’ samples. Results are assessed using both single rules and Multirules. Concomitantly with the QC analyses, an EQA project is undertaken; students analyse two EQA samples, twice in the semester. Results are submitted using cloud software and data for the full ‘peer group’ returned to students in spreadsheets and incomplete Youden plots. Youden plots are completed with target values and calculated ALP values and analysed for ‘lab’ and method performance. The method has a low-level positive bias, which leads to the need to investigate an alternative method. Building directly on the EQA of the first project and using the scenario of a lab that services renal patients, students undertake a method validation comparing BCP and BCG assays in another eight-week project. Precision and patient comparison studies allow students to assess whether the BCP method addresses the proportional bias of the BCG method and overall is a ‘better’ alternative method for analysing serum albumin, accounting for pragmatic factors, such as cost, as well as performance characteristics. Results Students develop understanding of the purpose and importance of QC and EQA in delivering quality results, the need to optimise testing to deliver quality results and importantly, a working knowledge of the analyses that go into ensuring this quality. In parallel to developing these key workplace competencies, students become confident, competent practitioners, able to pipette accurately and precisely and organise themselves in a busy, time pressured work environment.

Establishing a framework for transforming student engagement, success and retention in higher education institutions

This poster summarises the outcomes of a national project to develop and provide a holistic framework consisting of a series of sequential and increasingly sophisticated stages that will allow higher education institutions (HEIs) to manage and improve their student engagement and retention strategies/programs.

The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates

Objective: To evaluate the feasibility, reliability and acceptability of the mini clinical evaluation exercise (mini-CEX) for performance assessment among international medical graduates (IMGs). Design, setting and participants: Observational study of 209 patient encounters involving 28 IMGs and 35 examiners at three metropolitan teaching hospitals in New South Wales, Victoria and Queensland, September-December 2006. Main outcome measures: The reliability of the mini-CEX was estimated using generatisability (G) analysis, and its acceptability was evaluated by a written survey of the examiners and IMGs. Results: The G coefficient for eight encounters was 0.88, suggesting that the reliability of the mini-CEX was 0.90 for 10 encounters. Almost half of the IMGs (7/16) and most examiners (14/18) were satisfied with the mini-CEX as a learning tool. Most of the IMGs and examiners enjoyed the immediate feedback, which is a strong component of the tool. Conclusion: The mini-CEX is a reliable tool for performance assessment of IMGs, and is acceptable to and well received by both learners and supervisors.

Student perception of a new integrated anatomy practical program: Does students' prior learning make a difference?

While there is evidence that science and non-science background students display small differences in performance in basic and clinical sciences, early in a 4-year, graduate entry medical program, this lessens with time. With respect to anatomy knowledge, there are no comparable data as to the impact previous anatomy experience has on the student perception of the anatomy practical learning environment. A study survey was designed to evaluate student perception of the anatomy practical program and its impact on student learning, for the initial cohort of a new medical school. The survey comprised 19 statements requiring a response using a 5-point Likert scale, in addition to a free text opportunity to provide opinion of the perceived educational value of the anatomy practical program. The response rate for a total cohort of 82 students was 89%. The anatomy practical program was highly valued by the students in aiding their learning of anatomy, as indicated by the high mean scores for all statements (range: 4.04-4.7). There was a significant difference between the students who had and had not studied a science course prior to entering medicine, with respect to statements that addressed aspects of the course related to its structure, organization, variety of resources, linkage to problem-based learning cases, and fairness of assessment. Nonscience students were more positive compared to those who had studied science before (P levels ranging from 0.004 to 0.035). Students less experienced in anatomy were more challenged in prioritizing core curricular knowledge. © 2011 Wiley-Liss, Inc.

Asking the critical questions: An evaluation of social work students' experiences in an international placement

Preparing social work students for the demands of changing social environments and to promote student mobility and interest in overseas employment opportunities have resulted in an increasing demand for international social work placements. The literature describes numerous examples of social work programmes that offer a wide variety of international placements. However, research about the actual benefit of undertaking an overseas placement is scant with limited empirical evidence on the profile of students participating, their experience of the tasks offered, the supervisory practice and the outcomes for students' professional learning and career. This study contributes to the existing body of literature by exploring the relevance of international field placements for students and is unique in that it draws its sample from students who have graduated so provides a distinctive perspective in which to compare their international placement with their other placement/s as well as evaluating what were the benefits and drawbacks for them in terms of their careers, employment opportunities and current professional practice.

Student performance in field education placements: The findings of a 6-year Australian study of admissions data

This article reports on a 6-year study that examined the association between pre-admission variables and field placement performance in an Australian bachelor of social work program (N=463). Very few of the pre-admission variables were found to be significantly associated with performance. These findings and the role of the admissions process are discussed. In addition to the usual academic criteria, the authors urge schools to include a focus on nonacademic criteria during the admissions process and the ongoing educational program.

Dimensions of University Student Learning in Medicine and Pharmacy

Study orientations in higher education consist of various dimensions, such as approaches to learning, conceptions of learning and knowledge (i.e. epistemologies), self-regulation, and motivation. They have also been measured in different ways. The main orientations typically reported are reproducing and meaning orientations. The present study explored dimensions of study orientations, focusing in particular on pharmacy and medicine. New versions of self-report instruments were developed and tested in various contexts and in two countries. Furthermore, the linkages between study orientations and students epistemological development were explored. The context of problem-based (PBL) small groups was investigated in order to better understand how collaboration contributes to the quality of learning. The participants of Study I (n=66) were pharmacy students, who were followed during a three-year professionally oriented program in terms of their study orientations and epistemologies. A reproducing orientation to studying diminished during studying, whereas only a few students maintained their original level of meaning orientation. Dualism was found to be associated with a reproducing orientation. In Study II practices associated with deep and surface approaches to learning were measured in two differing ways, in order to better distinguish between what students believed to be useful in studying, and the extent to which they applied their beliefs to practice when preparing for examinations. Differences between domains were investigated by including a sample of Finnish and Swedish medical students (n=956) and a Finnish non-medical sample of university students (n=865). Memorizing and rote learning appeared as differing components of a surface approach to learning, while understanding, relating, and critical evaluation of knowledge emerged as aspects of a deep approach to learning. A structural model confirmed these results in both student samples. Study III explored a wide variety of dimensions of learning in medical education. Swedish medical students (n=280) answered the questionnaire. The deep approach to learning was strongly related to collaboration and reflective learning, whereas the surface approach was associated with novice-like views of knowledge and the valuing of certain and directly applicable knowledge. PBL students aimed at understanding, but also valued the role of memorization. Study IV investigated 12 PBL tutorial groups of students (n=116) studying microbiology and pharmacology in a medical school. The educational application was expected to support a deep approach to learning: Group members course grades in a final examination were related to the perceived functioning of the PBL tutorial groups. Further, the quality of cases that had been used as triggers for learning, was associated with the quality of small group functioning. New dimensions of study orientations were discovered. In particular, novel, finer distinctions were found within the deep approach component. In medicine, critical evaluation of knowledge appeared to be less valued than understanding and relating. Further, collaboration appeared to be closely related to the deep approach, and it was also important in terms of successful PBL studying. The results of the studies confirmed the previously found associations between approaches to learning and study success, but showed interesting context- and subgroup-related differences in this respect. Students ideas about the nature of knowledge and their approaches to learning were shown to be closely related. The present study expanded our understanding of the dimensions of study orientations, of their development, and their contextual variability in pharmacy and medicine.

Análisis y mejora de la gestión y organización del equipo Formula Student Bizkaia usando como referencia el modelo EFQM

[ES]El presente trabajo de fin de grado tiene como objetivo analizar y mejorar la gestión y organización del equipo Formula Student Bizkaia, de tal manera que le permita ser más competitivo frente al resto de universidades. Para su realización, en primer lugar, se exponen las distintas alternativas que existen para evaluar la calidad de gestión del equipo. En segundo lugar, una vez se haya escogido el modelo más conveniente, se prosigue al estudio de los distintos criterios clave en los que se basa. En tercer lugar, cuando se haya llevado a cabo el estudio de los puntos fuertes, así como evidencias y áreas a mejorar, se procede a realizar una evaluación que permita determinar las fortalezas y debilidades, para de esta manera potenciarlas y solventarlas, respectivamente. Por último, se presenta un desglose de los gastos que conlleva realizar la elaboración de un estudio de estas características, así como un desglose de las diferentes tareas a llevar a cabo y su respectiva duración.