Non-formal techniques for early assessment of design ideas for services

Designing systems for multiple stakeholders requires frequent collaboration with multiple stakeholders from the start. In many cases at least some stakeholders lack a professional habit of formal modeling. We report observations from student design teams as well as two case studies, respectively of a prototype for supporting creative communication to design objects, and of stakeholder-involvement in early design. In all observations and case studies we found that non-formal techniques supported strong collaboration resulting in deep understanding of early design ideas, of their value and of the feasibility of solutions.

Hunting and gathering : new imperatives in mapping and collecting student learning data to assure quality outcomes

Assurance of learning (AOL) is a quality enhancement and quality assurance process used in higher education. It involves a process of determining programme learning outcomes and standards, and systematically gathering evidence to measure students' performance on these. The systematic assessment of whole-of-programme outcomes provides a basis for curriculum development and management, continuous improvement, and accreditation. To better understand how AOL processes operate, a national study of university practices across one discipline area, business and management, was undertaken. To solicit data on AOL practice, interviews were undertaken with a sample of business school representatives (n = 25). Two key processes emerged: (1) mapping of graduate attributes and (2) collection of assurance data. External drivers such as professional accreditation and government legislation were the primary reasons for undertaking AOL outcomes but intrinsic motivators in relation to continuous improvement were also evident. The facilitation of academic commitment was achieved through an embedded approach to AOL by the majority of universities in the study. A sustainable and inclusive process of AOL was seen to support wider stakeholder engagement in the development of higher education learning outcomes.

Crossing Boundaries : Australian student teachers creating ‘The Global Teacher Exhibition’

This DVD describes a curriculum project embedded into the subject The Global Teacher (code: CLB049/LCB327, Faculty of Education, Queensland University of Technology). The Global Teacher is a subject within the undergraduate degree program for pre-service teachers and provides a global perspective on socio-political issues that shape education. The curriculum in The Global Teacher was designed around a collaborative partnership between Queensland University of Technology and State Library Queensland. Through this collaboration, State Library became not only a resource for information, but also helped to develop the pedagogical skills of the pre-service teaachers by guiding them in exhibiting and curating Global Teacher themes for a broader community-based audience. The collaboration became part of the assessment for The Global Teacher, requiring the pre-service teachers to visually translate their understandings of global educational issues into a public exhibition, which was held at State Library Queensland on 1st May, 2013. This DVD is a creative work explaining the stages of this collaborative project. It explores the learning outcomes achieved, using the voices of participants: the pre-service teachers, the QUT teacher educators and staff of State Library Queensland. A detailed description of this project is to be found at: http://libguides.library.qut.edu.au/content.php?pid=595206&sid=4908024&preview=1b455ed4f2c606d19702090f85d1f965

Literacy assessment that counts : mediating, interpreting and contesting translocal policy in a primary school

In Australia, as in many western education systems over the last two decades, discourses of accountability and performativity have reshaped education policy that has in turn reorganised the work of school leaders and teachers. One of the effects of this reorganisation is increased attention to the production, analysis and display of student achievement data. In this paper we examine in detail a sequence of the production and reading of literacy assessment data in a small Catholic school. Our analysis uses institutional ethnography’s concept of the ‘active text’, the text as occurring in a specific place and time even as it is articulated to social relations beyond its immediate context. Through this process we learn from those involved how their everyday work brings into being formalised, textually authorised processes in a local site that ensure the school meets accountability requirements while enabling teachers to resist standardisation of literacy teaching and assessment.

The role of radiotherapy patients in provision of student interpersonal skills feedback

Background At Queensland University of Technology, student radiation therapists receive regular feedback from clinical staff relating to clinical interpersonal skills. Although this is of great value, there is anecdotal evidence that students communicate differently with patients when under observation. Purpose The aim of this pilot was to counter this perceived observer effect by allowing patients to provide students with additional feedback. Materials and methods Radiotherapy patients from two departments were provided with anonymous feedback forms relating to aspects of student interpersonal skills. Clinical assessors, mentors and students were also provided with feedback forms, including questions about the role of patient feedback. Patient perceptions of student performance were correlated with staff feedback and assessment scores. Results Results indicated that the feedback was valued by both students and patients. Students reported that the additional dimension focused them on communication, set goals for development and increased motivation. These changes derived from both feedback and study participation, suggesting that the questionnaires could be a useful teaching tool. Patients scored more generously than mentors, although there was agreement in relative grading. Conclusions The anonymous questionnaire is a convenient and valuable method of gathering patient feedback on students. Future iterations will determine the optimum timing for this method of feedback.

Promoting law student well-being through the curriculum

The Australian tertiary sector is becoming increasingly concerned about the psychological well-being of its students.Empirical research in Australia indicates that more than one-third of law students suffer from psychological distress, and the competitive, isolated, adversarial learning environment at law school has been suggested as partly responsible (Brain and Mind Research Institute, 2009). This fellowship program has mobilised strategic change to improve the psychological health of law students. It has lead and stimulated advancement in the legal curriculum, its pedagogy, and assessment practice to better engage, motivate and support student learning of law, focussing on the potential of non-adversarial legal practice. A new conceptual framework for legal education has been developed, demonstrating the pursuit of excellence in the teaching of law, and raising the profile of learning and teaching in Australian law schools. In addition the fellowship has created a national community of practice around this issue through the Wellness Network for Law, and made significant contributions to research and scholarship in the field.

The automation of radiotherapy treatment planning assessment

Aim The assessment of treatment plans is an important component in the education of radiation therapists. The establishment of a grade for a plan is currently based on subjective assessment of a range of criteria. The automation of assessment could provide a number of advantages including faster feedback, reduced chance of human error, and simpler aggregation of past results. Method A collection of treatments planned by a cohort of 27 second year radiation therapy students were selected for quantitative evaluation. Treatment sites included the bladder, cervix, larynx, parotid and prostate, although only the larynx plans had been assessed in detail. The plans were designed with the Pinnacle system and exported using the DICOM framework. Assessment criteria included beam arrangement optimisation, volume contouring, target dose coverage and homogeneity, and organ-at-risk sparing. The in-house Treatment and Dose Assessor (TADA) software1 was evaluated for suitability in assisting with the quantitative assessment of these plans. Dose volume data were exported in per-student and per-structure data tables, along with beam complexity metrics, dose volume histograms, and reports on naming conventions. Results The treatment plans were exported and processed using TADA, with the processing of all 27 plans for each treatment site taking less than two minutes. Naming conventions were successfully checked against a reference protocol. Significant variations between student plans were found. Correlation with assessment feedback was established for the larynx plans. Conclusion The data generated could be used to inform the selection of future assessment criteria, monitor student development, and provide useful feedback to the students. The provision of objective, quantitative evaluations of plan quality would be a valuable addition to not only radiotherapy education programmes but also for staff development and potentially credentialing methods. New functionality within TADA developed for this work could be applied clinically to, for example, evaluate protocol compliance.

Evaluation of assessment in the context of work-based learning: Qualitative perspectives of new graduates

Aim Evaluation or assessment of competence is an important step to ensure the safety and efficacy of health professionals, including dietitians. Most competency-based assessment studies are focussed on valid and reliable methods of assessment for the preparation of entry-level dietitians, few papers have explored student dietitians’ perceptions of these evaluations. This study aimed to explore the perceptions of recent graduates from accredited nutrition and dietetics training programs in Australia. It also aimed to establish the relevance of competency-based assessment to adequately prepare them for entry-level work roles. Methods A purposive sample of newly-graduated dietitians with a range of assessment experiences and varied employment areas was recruited. A qualitative approach, using in-depth interviews with 13 graduates, with differing assessment experiences was undertaken. Graduates were asked to reflect upon their competency-based assessment experiences whilst a student. Data was thematically analysed by multiple authors. Results Four themes emerged from the data analysis: (i) Transparency and consistency are critical elements of work-based competency assessment. (ii) Students are willing to take greater responsibility in their assessment process. (iii) Work-based competency assessment prepares students for employment. (iv) The relationship between students and their assessors can impact on the student experience and their assessment performance. Conclusions Understanding this unique perspective of students can improve evaluation of future health professionals and assist in designing valid competency-based assessment approaches.

Development of an electronic role-play assessment initiative in bioscience for nursing students

Devising authentic assessments for subjects with large enrolments is a challenge. This study describes an electronic role-play assessment for approximately 600 first-year nursing students to learn and apply pathophysiology (bioscience) concepts to nursing practice. Students used Microsoft Office PowerPoint® to prepare electronic role-plays both between a nurse and patient, and between two nurses, thus simulating workplace scenarios. Student feedback demonstrated that respondents found this assessment useful for learning pathophysiology, and for applying pathophysiology to a nursing clinical setting. This electronic presentation circumvented issues associated with a traditional oral presentation such as embarrassment and logistics of scheduling groups, and rated well with students of non-English speaking background. The electronic role-play assessment initiative encouraged students to apply their bioscience knowledge to a clinical setting, and allowed students to conceptualise the importance of bioscience within both the nursing degree and the profession.

Framing student evaluations of university learning and teaching : discursive strategies and textual outcomes

Evaluation in higher education is an evolving social practice, that is, it involves what people, institutions and broader systems do and say, how they do and say it, what they value, the effects of these practices and values, and how meanings are ascribed. The textual products (verbal, written, visual, gestural) that inform and are produced by, for and through evaluative practices are important as they promulgate particular kinds of meanings and values in specific contexts. This paper reports on an exploratory study that sought to investigate, using discourse analysis, the types of evaluative practices that were ascribed value, and the student responses that ensued, in different evaluative instruments. Findings indicate that when a reflective approach is taken to evaluation, students’ responses are more considered, they interrogate their own engagement in the learning context and they are more likely to demonstrate reconstructive thought. These findings have implications for reframing evaluation as reflective learning.

CoWs, Google and online possibilities : creating digital posters to enhance student engagement

A new Bachelor of Science (BSc) course was introduced at Queensland University of Technology (QUT) in 2013 and focused on inquiry-based, collaborative and active learning. Two of the first year units required that students carry out a group poster assessment task. This poster provides a preliminary evaluation from an academic staff perspective of the assessment approach used, whereby students created digital posters to utilise the affordances of new learning spaces. The digital posters approach was first introduced to a group of academic staff from the Science and Engineering Faculty (SEF) in 2012 during a professional development program to explicitly develop skills and shared understandings of teaching in collaborative learning spaces (Steel & Andrews, 2012). Considerations were given to the pedagogical requirements of a poster assessment task, the affordances of the learning space and an identification of possible benefits of using Google Sites to create digital posters. Positive feedback from this group (as highlighted in the quotes shown) and subsequent approval from unit coordinators for two of the new first year BSc units meant that the approach was adopted for Semester 1, 2013 with approximately 360 students in each unit.

Development of an online role-play assessment initiative for nursing students in their bioscience studies

Devising assessment tasks for large units that embrace academic goals of authenticity and assessment variety can be a challenge. We developed an online Role-Play Assessment Initiative for first year nursing students in bioscience. Students responded to a case study by preparing two role-play dialogues: as a nurse with the patient, and between two nurses. The aims were to assess whether the students could: 1) understand the underlying disease process (pathophysiology) and relate it to clinical practice; 2) use language appropriate for lay and medical conversation; and 3) apply information using active learning. We conducted a student survey using quantitative questions (Likert scale: 1=strongly disagree to 5=strongly agree), and qualitative questions. 65 completed surveys were received. 80% of respondents agreed (includes agree or strongly agree) that it was a useful way to learn and understand pathophysiology of the case study. 86% agreed that it was useful to apply pathophysiology from lectures to a clinical setting. Overall, students found it enjoyable, which is beneficial for enhanced student engagement, and agreed that it allowed them to work well in a group (74% and 85%, respectively). Most qualitative suggestions for improvement related to group work, despite the encouraging response to group work in quantitative questions. Most positive comments surrounded different communication with a nurse compared with a patient. These results demonstrate that students developed deeper understanding of pathophysiology through active learning and were able to expand their nursing career skills during the role-play. Learning using role-play to simulate the workforce has fostered active learning.

Linking pedagogical knowledge practices and student outcomes in STEM education for primary schools

STEM education is a new frontier in Australia, particularly for primary schools. However, the E in STEM needs to have a stronger focus with science and mathematics concepts aligned to the presiding curricula. In addition, pedagogical knowledge practices such as planning, preparation, teaching strategies, assessment and so forth need to be connected to key concepts for developing a STEM education. One of the aims of this study was to understand how a pedagogical knowledge practice framework could be linked to student outcomes in STEM education. Specifically, this qualitative research investigated Year 4 students’ involvement in an integrated STEM education program that focused on science concepts (e.g., states of matter, testing properties of materials) and mathematics concepts (such as 3D shapes and metric measurements: millilitres, temperature, grams, centimetres) for designing, making and testing a strong and safe medical kit to insulate medicines at desirable temperatures. Eleven pedagogical knowledge practices (e.g., planning, preparation, teaching strategies, classroom management, and assessment) were used as a framework for understanding how teaching may be linked to student outcomes in STEM education. For instance, “planning” involved devising a student booklet as a resource for students to understand the tasks required of them, which also provided space for them to record ideas, results and information. Planning involved linking national and state curriculum documents to the STEM education activities. More studies are required around pedagogical knowledge frameworks to understand what students learn when involved in STEM education, particularly with the inclusion of engineering education.

Class participation as a learning and assessment strategy in law : facilitating students’ engagement, skills development and deep learning

Assessment has widely been described as being ‘at the centre of the student experience’. It would be difficult to conceive of the modern teaching university without it. Assessment is accepted as one of the most important tools that an educator can deploy to influence both what and how students learn. Evidence suggests that how students allocate time and effort to tasks and to developing an understanding of the syllabus is affected by the method of assessment utilised and the weighting it is given. This is particularly significant in law schools where law students may be more preoccupied with achieving high grades in all courses than their counterparts from other disciplines. However, well-designed assessment can be seen as more than this. It can be a vehicle for encouraging students to learn and engage more broadly than with the minimums required to complete the assessment activity. In that sense assessment need not merely ‘drive’ learning, but can instead act as a catalyst for further learning beyond what a student had anticipated. In this article we reconsider the potential roles and benefits in legal education of a form of interactive classroom learning we term assessable class participation (‘ACP’), both as part of a pedagogy grounded in assessment and learning theory, and as a platform for developing broader autonomous approaches to learning amongst students. We also consider some of the barriers students can face in ACP and the ways in which teacher approaches to ACP can positively affect the socio-emotional climates in classrooms and thus reduce those barriers. We argue that the way in which a teacher facilitates ACP is critical to the ability to develop positive emotional and learning outcomes for law students, and for teachers themselves.