Longitudinal Student Evaluations of a Postgraduate Unit Using Importance-Performance Analysis

The Australian tourism tertiary education sector operates in a competitive and dynamic environment, which necessitates a market orientation to be successful. Academic staff and management in the sector must regularly assess the perceptions of prospective and current students, and monitor the satisfaction levels of current students. This study is concerned with the setting and monitoring of satisfaction levels of current students, reporting the results of three longitudinal investigations of student satisfaction in a postgraduate unit. The study also addresses a limitation of a university’s generic teaching evaluation instrument. Importance-performance analysis (IPA) has been recommended as a simple but effective tool for overcoming the deficiencies of many student evaluation studies, which have generally measured only attribute importance or importance at the end of a semester. IPA was used to compare student expectations of the unit at the beginning of semester with their perceptions of performance ten weeks later. The first stage documented key benchmarks for which amendments to the unit based on student feedback could be evaluated during subsequent teaching periods.

Longitudinal evaluations of student satisfaction with a postgraduate unit using importance-performance analysis

The Australian tourism tertiary education sector operates in a competitive and dynamic environment, which necessitates a market orientation to be successful. Academic staff and management in the sector must regularly assess the perceptions of prospective and current students and monitor the satisfaction levels of current students. This study is concerned with the setting and monitoring of satisfaction levels of current students, reporting the results of three longitudinal investigations of student satisfaction in a postgraduate unit. The study also addresses a limitation of a university’s generic teaching evaluation instrument. Importance-Performance Analysis (IPA) has been recommended as a simple but effective tool for overcoming the deficiencies of many student evaluation studies, which have generally measured only attribute performance at the end of a semester. IPA was used to compare student expectations of the unit at the beginning of a semester with their perceptions of performance 10 weeks later. The first stage documented key benchmarks for which amendments to the unit based on student feedback could be evaluated during subsequent teaching periods.

Integration of medical images to the teaching of systematic pathology : an evaluation of relevance

A sound knowledge of pathological disease processes is required for professional practice within health professions. The project described in this paper reviewed the resources currently available for the delivery of systematic pathology tutorials. Additional complementary resources were developed and the inclusion of these additional learning resources in practical tutorial sessions was evaluated for their impact on student learning. Student evaluation of the learning resources was undertaken across one semester with two different cohorts of health profession students using questionnaires and focus group discussion. Both cohorts reported an enhancement to their understanding of pathological disease processes through the use of the additional resources. Results indicate student perception of the value of the resources correlates with staff perception and is independent of prior experiences.

Please take out your phones : on the spot solicitation of student feedback in class

The use of mobile devices such as smart phones and tablets in classrooms has been met with mixed sentiments. Some instructors and teachers see them as a distraction and regularly ban their usage. Others who see their potential to enhance learning have started to explore ways to integrate them into their teaching in an attempt to improve student engagement. In this paper we report on a pilot study that forms part of a university-wide project reconceptualising its approach to the student evaluation of learning and teaching. In a progressive decision to embrace mobile technology, the university decided to trial a smart phone app designed for students to check-in to class and leave feedback on the spot. Our preliminary findings from trialling the app indicate that the application establishes a more immediate feedback loop between students and teachers. However, the app’s impact depends on how feedback is shared with students and how the teaching team responds.

'Take away from the dry sixties style marking': Lecturer and student perceptions and experiences of audio feedback

Providing audio feedback to assessment is relatively uncommon in higher education. However, published research suggests that it is preferred over written feedback by students but lecturers were less convinced. The aim of this paper is to examine further these findings in the context of a third year business ethics unit. Data was collected from two sources. The first is a series of in-depth, semi-structured interviews conducted with three lecturers providing audio feeback for the first time in Semester One 2011. The second source of data was drawn from the university student evaluation system. A total of 363 responses were used providing 'before' and 'after' perspectives about the effectiveness of audio feedback versus written feedback. Between 2005 and 2009 the survey data provided information about student attitudes to written assessment feedback (n=261). From 2010 onwards the data relates to audio (mp3) feedback (n=102). The analysis of he interview data indicated that introducing audio feedback should be done with care. The perception of the participating lecturers was mixed, ranging from sceptism to outright enthusiasm, but over time the overall approach became positive. It was found that particular attention needs to be paid to small (but important) technical details, and lecturers need to be convinced of its effectieness, especially that it is not necessarily more time consuming than providing written feedback. For students, the analysis revealed a clear preference for audio feedback. It is concluded that there is cause for concern and reason for optimism. It is a cause for concern because there is a possibility that scepticism on the part of academic staff seems to be based on assumptions about what students prefer and a concern about using the technology. There is reason for optimism because the evidence points towards students preferring audio feedback and as academic staff become more familiar with the technology the scepticism tends to evaporate. While this study is limited in scope, questions are raised about tackling negative staff perceptions of audio feedback that are worthy of further research.

A review of non-traditional occupational therapy practice placement education : a focus on role-emerging and project placements

Practice placement education has been recognised as an integral and critical component of the training of occupational therapy students. Although there is an extensive body of literature on clinical education and traditional practice placement education models, there has been limited research on alternative placements.-------- This paper reviews the literature on various practice placement education models and presents a contemporary view on how it is currently delivered. The literature is examined with a particular focus on the increasing range of practice placement education opportunities, such as project and role-emerging placements. The drivers for non-traditional practice placement education include shortages of traditional placement options, health reform and changing work practices, potential for role development and influence on practice choice. The benefits and challenges of non-traditional practice placement education are discussed, including supervision issues, student evaluation, professional and personal development and the opportunity to practise clinical skills.--------- Further research is recommended to investigate occupational therapy graduates' perceptions of role-emerging and project placements in order to identify the benefits or otherwise of these placements and to contribute to the limited body of knowledge of emerging education opportunities.

The design and development of an undergraduate signal processing laboratory

The School of Electrical and Electronic Systems Engineering of Queensland University of Technology (like many other universities around the world) has recognised the importance of complementing the teaching of signal processing with computer based experiments. A laboratory has been developed to provide a "hands-on" approach to the teaching of signal processing techniques. The motivation for the development of this laboratory was the cliche "What I hear I remember but what I do I understand." The laboratory has been named as the "Signal Computing and Real-time DSP Laboratory" and provides practical training to approximately 150 final year undergraduate students each year. The paper describes the novel features of the laboratory, techniques used in the laboratory based teaching, interesting aspects of the experiments that have been developed and student evaluation of the teaching techniques

The development of a program to assist international nursing students to be prepared for clinical practice in the Australian health care context

Health education in Western Countries has grown considerably in the last decade and this has occurred for a number of reasons. Specifically Universities actively recruit International students as the health workforce becomes global; also it is much easier for students to move and study globally. Internationally there is a health workforce shortage and if students gain a degree in a reputable university their ability to work globally is improved significantly. However, when studying to practice in the health care field the student must undertake clinical practice in an acute or aged care setting. This can be a significant problem for students who are culturally and linguistically diverse in an English speaking country such as Australia. The issues that can arise stem from the language differences where communication, interpretation understanding and reading the cultural norms of the health care setting are major challenges for International students. To assist international students to be successful in their clinical education, an extra curriculum workshop program was developed to provide additional support. The program which runs twice each year includes on-campus interactive workshops that are complemented by targeted support provided for students and clinical staff who are supervising students’ practice experience in the workplace. As this is an English speaking country the workshop is based on practicing reading, writing, listening and speaking, as well as exploring basic health care concepts and cultural differences. This enables students to gain knowledge of and practice interpretation of cultural norms and expectations in a safe environment. This innovative series of interactive workshops in a highly student-centred learning environment combine education with role play and discussion with peers who are supported by culturally aware and competent Educators. Over the years it has been running, the program has been undertaken by an increasing number of students. In 2011, more than 100 students are expected to participate. Student evaluation of the program has confirmed that it has assisted the majority of them to be successful in their clinical studies. Effectiveness of the project is measured throughout the program and in follow up sessions. This ongoing information allows for continuous development of the program that serves to meet individual needs of the International student, the University and Service providers such as the hospitals. This feedback from students regarding their increased comprehension of the Australian colloquial Language, healthcare terminology, critical thinking and clinical skill development and a cultural awareness also enables them to maintain their feelings of self confidence and self esteem.

Towards an understanding of teacher judgement in the context of social moderation

Social moderation involves teachers gathering together to discuss their judgements of the quality of student work and to reach agreement regarding the standard awarded. This qualitative study conducted over a three-year period investigated the social practice of moderation and the influence on teachers’ judgements of students work. An initial survey of teachers’ understandings of moderation and standards, pre-interviews of teachers who participated in the moderation meetings, observations of these meetings with a particular focus on one teacher (focus teachers) comprised the data collection methods. Data analysis involved organising, matching, coding, identifying patterns and themes using a constant comparative method. Socio-cultural theories of learning and assessment underpinned the approach to data analysis and proved helpful in explaining the diverse influences on teachers’ judgements beyond the task criteria, and the progressive development of shared understandings through engaging in professional discussions of students’ work. The study revealed that the process is not clear and linear and is influenced by factors such as the representation of the standards and the knowledge base of the teachers.

Teaching undergraduate students community nursing : using action research to increase engagement and learning

Nurses play a pivotal role in responding to the changing needs of community health care. Therefore, nursing education must be relevant, responsive, and evidence based. We report a case study of curriculum development in a community nursing unit embedded within an undergraduate nursing degree. We used action research to develop, deliver, evaluate, and redesign the curriculum. Feedback was obtained through self-reflection, expert opinion from community stakeholders, formal student evaluation, and critical review. Changes made, especially in curriculum delivery, led to improved learner focus and more clearly linked theory and practice. The redesigned unit improved performance, measured with the university's student evaluation of feedback instrument (increased from 0.3 to 0.5 points below to 0.1 to 0.5 points above faculty mean in all domains), and was well received by teaching staff. The process confirmed that improved pedagogy can increase student engagement with content and perception of a unit as relevant to future practice.

Moving beyond traditional clinics : creating a new legal clinics through community partnerships

To overcome the challenge of finding placements for large student numbers, QUT has partnered with community organisations to enable students to work on community-based projects addressing a community need. Students work in interdisciplinary teams with the community organisation to resolve issues and identify solutions to suit the organisation and client base. This paper will describe the community engaged learning pedagogy that is employed in the subject and will consider the benefits and challenges to law students of working collaboratively and developing community relationships. Critical appraisal of the legal system and the role of lawyers and analysis of the professional and ethical responsibilities legal practitioners is a focus of the subject. Explicit emphasis is placed on developing a sense of social responsibility and inculcating a pro bono ethos. Students attend workshops on topics such as reflective practice, cultural competencies, client solutions, collaborative practice and ethical obligations. This paper will discuss the challenges in creating the new legal clinic subject, benefits to students and community partners, and the results of initial student evaluation of the subject.

Evaluating leadership development using the Most Significant Change technique

Course evaluations are now a serious matter for universities trying to meet stakeholder needs and expectations, quality assurance, improvements and strategic decision making. Typically, students are invited to participate in surveys on how well the design and delivery aspects meet predetermined learning objectives, quality of teaching, and the types of improvements needed for future deliveries. We used the Most Significant Change technique to gather data on the impact of a leadership course on 18 Pacific Islanders who completed a Master of Education (Educational Leadership). Participants' views highlighted impacts that were of significance to the students and their workplaces. The findings demonstrate that the Most Significant Change technique offers a more comprehensive understanding of the impact of leadership development courses.

Do instructional attributes pose multicollinearity problems? An empirical exploration

It is commonly perceived that variables ‘measuring’ different dimensions of teaching (construed as instructional attributes) used in student evaluation of teaching (SET) questionnaires are so highly correlated that they pose a serious multicollinearity problem for quantitative analysis including regression analysis. Using nearly 12000 individual student responses to SET questionnaires and ten key dimensions of teaching and 25 courses at various undergraduate and postgraduate levels for multiple years at a large Australian university, this paper investigates whether this is indeed the case and if so under what circumstances. This paper tests this proposition first by examining variance inflation factors (VIFs), across courses, levels and over time using individual responses; and secondly by using class averages. In the first instance, the paper finds no sustainable evidence of multicollinearity. While, there were one or two isolated cases of VIFs marginally exceeding the conservative threshold of 5, in no cases did the VIFs for any of the instructional attributes come anywhere close to the high threshold value of 10. In the second instance, however, the paper finds that the attributes are highly correlated as all the VIFs exceed 10. These findings have two implications: (a) given the ordinal nature of the data ordered probit analysis using individual student responses can be employed to quantify the impact of instructional attributes on TEVAL score; (b) Data based on class averages cannot be used for probit analysis. An illustrative exercise using level 2 undergraduate courses data suggests higher TEVAL scores depend first and foremost on improving explanation, presentation, and organization of lecture materials.

Using academagogy to meet the needs of millennial learners: A comparative case study

Higher education institutions across the world are experiencing a new generation of students, known as millennial learners. They are more technologically literate and digitally connected than previous generations of learners. To meet the teaching and learning needs of these learners, we must offer more deliberate and meaningful learning experiences and opportunities, where students can see the connections between new material and their own experiences and real world applications – an academagogic approach. This study compares the implementation of academagogy for two different groups of millennial learners – one a traditional face-to-face undergraduate Engineering unit, and the other a mixed-mode (online and face-to-face) undergraduate Design unit. The units are discussed in terms of their student evaluation results, both qualitative and quantitative, and in terms of their academic outcomes for students. Conclusions are drawn about the applicability of academagogy as a heuristic for improving teaching and learning across disciplines, as well as its strengths and limitations in terms of student results.

Audience response systems can facilitate communal course feedback

Background At Queensland University of Technology (QUT), the Bachelor of Radiation Therapy course evaluation has previously suffered from low online survey participation rates. A communal instantaneous feedback event using an audience response system (ARS) was evaluated as a potential solution to this problem. The aims of the project were to determine the extent to which this feedback event could be facilitated by ARS technology and to evaluate the impact the technology made on student satisfaction and engagement. Methods Students were invited to a timetabled session to provide feedback on individual study units and the course overall. They provided quantitative Likert-style responses to prompts for each unit and the course using an ARS as well as anonymous typed qualitative comments. Data collection was performed live so students were able to view collective class responses. This prompted further discussion and enabled a prospective action plan to be developed. To inform future ARS use, students were asked for their opinions on the feedback method. Results Despite technological difficulties, student evaluation indicated that all responders enjoyed the session and the opportunity to view the combined responses. All students felt that useful feedback was generated and that this method should be used in the future. The student attendance and response rates were high, and it was clear that the session had led to the development of some insightful qualitative feedback comments. Conclusions: An ARS contributed well to the collection of course feedback in a communal and interactive environment. Students found it enjoyable to use, and it helped to stimulate useful qualitative comments