Comparative evaluation of PBL and traditional lecturebased teaching in undergraduate engineering courses : evidence from controlled learning environment

Project and problem-based learning (PBL) has been widely recognised as an active, collaborative, cumulative and integrative learning approach that engages learners, motivates team creativity and centres on practical education. On the other hand, traditional lecture-tutorial teaching is often criticised for being a passive, surface learning and exam-focused approach. In spite of these evidence-based observations and claims over the years, the traditional lecture-tutorial teaching approach still dominates as the preferred teaching approach at Australian universities. This study sets up a control environment to compare these two teaching and learning approaches by analysing data from students' actual performance, course evaluation and expectation in two large undergraduate engineering courses in 2009 and 2010. The evidence reported in this study is broadly interesting in that both courses were taught by the same teaching staff using two entirely different learning and teaching approaches to the same cohort of students in the same semester within the same degree program. The analysis shows that there are significant differences between the students' actual performance, course evaluation and their expectation. Such conflicting differences may be some of the reasons that may negatively impact teaching staff deterring them from switching to PBL from traditional lecture-tutorial teaching.

eVALUate: an evaluation instrument for measuring students’ perceptions of their engagement and learning outcomes

In the current climate in Australian higher education, quality assurance in university teaching is a priority. In particular, the introduction of the Learning and Teaching Performance Fund (LTPF) has refocused attention on universities’ internal student evaluation survey instruments. This paper reports the development, validation and implementation of a new unit survey instrument which prompts students to reflect on what helps their achievement of unit learning outcomes, and to report their levels of motivation, engagement and overall satisfaction with a semester‐long course or unit of study. The instrument (eVALUate) was created from precepts reported in the research literature, current practices in evaluating teaching, and sound quality assurance practices appropriate to a university outcomes‐focused education paradigm.

Lecturing and other face-to-face teaching – too much or too little? An assessment based on student feedback and fail rates

Institutes of higher learning are tending to reduce the amount of face-to-face teaching that they offer, and particularly through the traditional pedagogical method of lecturing. There is ongoing debate about the educational value of lectures as a teaching approach, in terms of both whether they facilitate understanding of subject material and whether they augment the student educational experience. In this study, student evaluation of teaching scores plus academic outcome (percentage of students who fail) was assessed for 236 course units offered by a science faculty at an Australian university over the course of one year. These measures were related to the degree to which lectures and other face-to-face teaching were used in these units, controlling for factors such as class size, school and year level. An information-theoretic model selection approach was employed to identify the best models and predictors of student assessments and fail rates. All the top models of student feedback included a measure reflecting amount of face-to-face teaching, with the evaluation of quality of teaching being higher in units with higher proportions of lectures. However, these models explained only 12–20% of the variation in student evaluation scores, suggesting that many other factors come into play. By contrast, units with fewer lectures have lower failure rates. These results suggest that moving away from lectures and face-to-face teaching may not harm, and indeed may improve the number of students who pass the subject, but that this may be incurred at the expense of greater dissatisfaction in students' learning experience.

External school evaluation : teachers’ perspectives : two case studies from Northern Portugal

The external evaluation of non-higher education schools in Portugal has been developed by the General Inspectorate of Education since 2006. A first cycle of evaluation was completed, covering all educational units in continental Portugal up to 2011. The model of evaluation has since been subject to alterations, and a second cycle of evaluation is now coming to an end. The current model of evaluation is based on documental analysis, analysis of students’ results, and panel interviews with a variety of representatives of the school community, and addresses three domains: results, provision of educational service and management. This paper is part of an ongoing research project, developed by 6 universities and supported by the Portuguese Foundation for Science and Technology (PTDC/CPE-CED/116674/2010) which intends to analyse the impacts and effects this process of external evaluation has had on Portuguese schools. This project includes a variety of perspectives and methodologies. In particular, we will focus on two case studies undertaken in two schools from the northern region of Portugal, and more specifically on the perspectives expressed by the teachers of those schools. These particular schools were chosen because they have been evaluated twice and represent different educational levels (basic and secondary), contexts and results. These case studies included the analysis of documental data, interviews to key informants and a questionnaire directed to teachers (n = 141) – the latter will be the main focus of this paper. Teachers are essential elements of the school community when considering the impacts of external evaluation, as any changes directed at teaching practices, student evaluation, among others are only possible through their direct action and implication. Therefore, their perceptions on the process and its impacts are crucial to the understanding of what does and does not change in schools as a consequence of external evaluation. Although teachers’ opinions are not homogenous and each school reveals a number of differences when it comes to teachers’ perceptions of School Evaluation, it was possible to stress some areas as the most and as the least consensual. Teachers in both schools agree External School Evaluation (ESE) is useful for the identification of the schools’ strengths and weaknesses, values students’ external evaluation results, imposes a model for schools internal evaluation (and in fact contributes to the very existence of internal evaluation practices), and contributes to schools improvement. However teachers in both schools do not believe ESE contributes to teachers’ autonomy produces changes in how curriculum is managed, or leads to innovative teaching practices. These results point to a greater emphasis on change at the levels of school management, self-evaluation and particularly internal evaluation, but little impact on the teaching practices. We believe the classroom is at the core of school practices and teaching processes are essential to any measure of school quality and to their impacts on student learning.

Text analytics visualisation of Course Experience Questionnaire student comment data in science and technology

A version of the Course Experience Questionnaire (CEQ) has been included in the Graduate Careers Council of Australia national survey of university graduates from 1993 onward. In addition to the quantitative response items noted above, the CEQ also includes an invitation to respondents to write open-ended comments on the best aspects (BA) of their university course experience and those aspects most needing improvement (NI).

A text analytics evaluation of a first-year engineering project-based unit

In the undergraduate engineering program at Griffith University in Australia, the unit 1006ENG Design and Professional Skills aims to provide an introduction to engineering design and professional practice through a project-based learning (PBL) approach to problem solving. It provides students with an experience of PBL in the first-year of their programme. The unit comprises an underpinning lecture series, design work including group project activities, an individual computer-aided drawing exercise/s and an oral presentation. Griffith University employs a ‘Student Experience of Course’ (SEC) online survey as part of its student evaluation of teaching, quality improvement and staff performance management processes. As well as numerical response scale items, it includes the following two questions inviting open-ended text responses from students: i) What did you find particularly good about this course? and ii) How could this course be improved? The collection of textual data in in student surveys is commonplace, due to the rich descriptions of respondent experiences they can provide at relatively low cost. However, historically these data have been underutilised because they are time consuming to analyse manually, and there has been a lack of automated tools to exploit such data efficiently. Text analytics approaches offer analysis methods that result in visual representations of comment data that highlight key individual themes in these data and the relationships between those themes. We present a text analytics-based evaluation of the SEC open-ended comments received in the first two years of offer of the PBL unit 1006ENG. We discuss the results obtained in detail. The method developed and documented here is a practical and useful approach to analysing/visualising open-ended comment data that could be applied by others with similar comment data sets.

A review of non-traditional occupational therapy practice placement education : a focus on role-emerging and project placements

Practice placement education has been recognised as an integral and critical component of the training of occupational therapy students. Although there is an extensive body of literature on clinical education and traditional practice placement education models, there has been limited research on alternative placements.-------- This paper reviews the literature on various practice placement education models and presents a contemporary view on how it is currently delivered. The literature is examined with a particular focus on the increasing range of practice placement education opportunities, such as project and role-emerging placements. The drivers for non-traditional practice placement education include shortages of traditional placement options, health reform and changing work practices, potential for role development and influence on practice choice. The benefits and challenges of non-traditional practice placement education are discussed, including supervision issues, student evaluation, professional and personal development and the opportunity to practise clinical skills.--------- Further research is recommended to investigate occupational therapy graduates' perceptions of role-emerging and project placements in order to identify the benefits or otherwise of these placements and to contribute to the limited body of knowledge of emerging education opportunities.

The design and development of an undergraduate signal processing laboratory

The School of Electrical and Electronic Systems Engineering of Queensland University of Technology (like many other universities around the world) has recognised the importance of complementing the teaching of signal processing with computer based experiments. A laboratory has been developed to provide a "hands-on" approach to the teaching of signal processing techniques. The motivation for the development of this laboratory was the cliche "What I hear I remember but what I do I understand." The laboratory has been named as the "Signal Computing and Real-time DSP Laboratory" and provides practical training to approximately 150 final year undergraduate students each year. The paper describes the novel features of the laboratory, techniques used in the laboratory based teaching, interesting aspects of the experiments that have been developed and student evaluation of the teaching techniques

The development of a program to assist international nursing students to be prepared for clinical practice in the Australian health care context

Health education in Western Countries has grown considerably in the last decade and this has occurred for a number of reasons. Specifically Universities actively recruit International students as the health workforce becomes global; also it is much easier for students to move and study globally. Internationally there is a health workforce shortage and if students gain a degree in a reputable university their ability to work globally is improved significantly. However, when studying to practice in the health care field the student must undertake clinical practice in an acute or aged care setting. This can be a significant problem for students who are culturally and linguistically diverse in an English speaking country such as Australia. The issues that can arise stem from the language differences where communication, interpretation understanding and reading the cultural norms of the health care setting are major challenges for International students. To assist international students to be successful in their clinical education, an extra curriculum workshop program was developed to provide additional support. The program which runs twice each year includes on-campus interactive workshops that are complemented by targeted support provided for students and clinical staff who are supervising students’ practice experience in the workplace. As this is an English speaking country the workshop is based on practicing reading, writing, listening and speaking, as well as exploring basic health care concepts and cultural differences. This enables students to gain knowledge of and practice interpretation of cultural norms and expectations in a safe environment. This innovative series of interactive workshops in a highly student-centred learning environment combine education with role play and discussion with peers who are supported by culturally aware and competent Educators. Over the years it has been running, the program has been undertaken by an increasing number of students. In 2011, more than 100 students are expected to participate. Student evaluation of the program has confirmed that it has assisted the majority of them to be successful in their clinical studies. Effectiveness of the project is measured throughout the program and in follow up sessions. This ongoing information allows for continuous development of the program that serves to meet individual needs of the International student, the University and Service providers such as the hospitals. This feedback from students regarding their increased comprehension of the Australian colloquial Language, healthcare terminology, critical thinking and clinical skill development and a cultural awareness also enables them to maintain their feelings of self confidence and self esteem.

Towards an understanding of teacher judgement in the context of social moderation

Social moderation involves teachers gathering together to discuss their judgements of the quality of student work and to reach agreement regarding the standard awarded. This qualitative study conducted over a three-year period investigated the social practice of moderation and the influence on teachers’ judgements of students work. An initial survey of teachers’ understandings of moderation and standards, pre-interviews of teachers who participated in the moderation meetings, observations of these meetings with a particular focus on one teacher (focus teachers) comprised the data collection methods. Data analysis involved organising, matching, coding, identifying patterns and themes using a constant comparative method. Socio-cultural theories of learning and assessment underpinned the approach to data analysis and proved helpful in explaining the diverse influences on teachers’ judgements beyond the task criteria, and the progressive development of shared understandings through engaging in professional discussions of students’ work. The study revealed that the process is not clear and linear and is influenced by factors such as the representation of the standards and the knowledge base of the teachers.

Teaching undergraduate students community nursing : using action research to increase engagement and learning

Nurses play a pivotal role in responding to the changing needs of community health care. Therefore, nursing education must be relevant, responsive, and evidence based. We report a case study of curriculum development in a community nursing unit embedded within an undergraduate nursing degree. We used action research to develop, deliver, evaluate, and redesign the curriculum. Feedback was obtained through self-reflection, expert opinion from community stakeholders, formal student evaluation, and critical review. Changes made, especially in curriculum delivery, led to improved learner focus and more clearly linked theory and practice. The redesigned unit improved performance, measured with the university's student evaluation of feedback instrument (increased from 0.3 to 0.5 points below to 0.1 to 0.5 points above faculty mean in all domains), and was well received by teaching staff. The process confirmed that improved pedagogy can increase student engagement with content and perception of a unit as relevant to future practice.

Moving beyond traditional clinics : creating a new legal clinics through community partnerships

To overcome the challenge of finding placements for large student numbers, QUT has partnered with community organisations to enable students to work on community-based projects addressing a community need. Students work in interdisciplinary teams with the community organisation to resolve issues and identify solutions to suit the organisation and client base. This paper will describe the community engaged learning pedagogy that is employed in the subject and will consider the benefits and challenges to law students of working collaboratively and developing community relationships. Critical appraisal of the legal system and the role of lawyers and analysis of the professional and ethical responsibilities legal practitioners is a focus of the subject. Explicit emphasis is placed on developing a sense of social responsibility and inculcating a pro bono ethos. Students attend workshops on topics such as reflective practice, cultural competencies, client solutions, collaborative practice and ethical obligations. This paper will discuss the challenges in creating the new legal clinic subject, benefits to students and community partners, and the results of initial student evaluation of the subject.

Evaluating leadership development using the Most Significant Change technique

Course evaluations are now a serious matter for universities trying to meet stakeholder needs and expectations, quality assurance, improvements and strategic decision making. Typically, students are invited to participate in surveys on how well the design and delivery aspects meet predetermined learning objectives, quality of teaching, and the types of improvements needed for future deliveries. We used the Most Significant Change technique to gather data on the impact of a leadership course on 18 Pacific Islanders who completed a Master of Education (Educational Leadership). Participants' views highlighted impacts that were of significance to the students and their workplaces. The findings demonstrate that the Most Significant Change technique offers a more comprehensive understanding of the impact of leadership development courses.

Do instructional attributes pose multicollinearity problems? An empirical exploration

It is commonly perceived that variables ‘measuring’ different dimensions of teaching (construed as instructional attributes) used in student evaluation of teaching (SET) questionnaires are so highly correlated that they pose a serious multicollinearity problem for quantitative analysis including regression analysis. Using nearly 12000 individual student responses to SET questionnaires and ten key dimensions of teaching and 25 courses at various undergraduate and postgraduate levels for multiple years at a large Australian university, this paper investigates whether this is indeed the case and if so under what circumstances. This paper tests this proposition first by examining variance inflation factors (VIFs), across courses, levels and over time using individual responses; and secondly by using class averages. In the first instance, the paper finds no sustainable evidence of multicollinearity. While, there were one or two isolated cases of VIFs marginally exceeding the conservative threshold of 5, in no cases did the VIFs for any of the instructional attributes come anywhere close to the high threshold value of 10. In the second instance, however, the paper finds that the attributes are highly correlated as all the VIFs exceed 10. These findings have two implications: (a) given the ordinal nature of the data ordered probit analysis using individual student responses can be employed to quantify the impact of instructional attributes on TEVAL score; (b) Data based on class averages cannot be used for probit analysis. An illustrative exercise using level 2 undergraduate courses data suggests higher TEVAL scores depend first and foremost on improving explanation, presentation, and organization of lecture materials.

Using academagogy to meet the needs of millennial learners: A comparative case study

Higher education institutions across the world are experiencing a new generation of students, known as millennial learners. They are more technologically literate and digitally connected than previous generations of learners. To meet the teaching and learning needs of these learners, we must offer more deliberate and meaningful learning experiences and opportunities, where students can see the connections between new material and their own experiences and real world applications – an academagogic approach. This study compares the implementation of academagogy for two different groups of millennial learners – one a traditional face-to-face undergraduate Engineering unit, and the other a mixed-mode (online and face-to-face) undergraduate Design unit. The units are discussed in terms of their student evaluation results, both qualitative and quantitative, and in terms of their academic outcomes for students. Conclusions are drawn about the applicability of academagogy as a heuristic for improving teaching and learning across disciplines, as well as its strengths and limitations in terms of student results.