Course evaluation : reconfigurations for learning with learning management systems

The introduction of online delivery platforms such as learning management systems (LMS) in tertiary education has changed the methods and modes of curriculum delivery and communication. While course evaluation methods have also changed from paper-based in-class-administered methods to largely online-administered methods, the data collection instruments have remained unchanged. This paper reports on a small exploratory study of two tertiary-level courses. The study investigated why design of the instruments and methods to administer surveys in the courses are ineffective measures against the intrinsic characteristics of online learning. It reviewed the students' response rates of the conventional evaluations for the courses over an eight-year period. It then compared a newly developed online evaluation and the conventional methods over a two-year period. The results showed the response rates with the new evaluation method increased by more than 80% from the average of the conventional evaluations (below 30%), and the students' written feedback was more detailed and comprehensive than in the conventional evaluations. The study demonstrated the possibility that the LMS-based learning evaluation can be effective and efficient in terms of the quality of students' participation and engagement in their learning, and for an integrated pedagogical approach in an online learning environment.

Course/instructor evaluation : perspectives of part-time graduate students

This study explored 6 part-time graduate students' perspectives on course/instructor evaluation. The purpose was to explore whether a link exists between the evaluation for course and instructors as contained in the Faculty of Education courselinstructor evaluation form and the needs of part-time students enrolled in that program. The literature review provided contextual information concerning the 3 main subject areas based upon which the research questions were designed: learner needs in the context of part-time graduate students, courselinstructor eval~ation, and the potential lack of congruency between the 2. Using a semistructured interview process, participants identified criteria important or relevant to the evaluation process and incongruent with the course/instructor evaluation form. A qualitative research methodology using a grounded theory approach contributed to the theory on the nature of course evaluation instruments in a graduate program and addressed the notion of where power was situated within the evaluation process. Findings suggested that the concepts of relevance and the instructor's role that participants identified as important in their graduate learning experience were congruent with what they considered important components of the course/instructor evaluation form. Participants noted a lack of congruency between their expectations of a quality graduate learning experience and the format, content, intent, and timing of the evaluation process. The study confirmed that students did want a voice in the evaluation of their learning experience at both the course and program levels.

How should a vocational education and training course be evaluated?

The course evaluation process used by a large VET provider was evaluated using guidelines suggested by the course evaluation literature and feedback obtained from multiple stakeholders. A modified model is presented as an exemplar for course evaluation in the VET sector.

Course cohesion : an elusive goal for tertiary education

A program’s development and implementation in a higher education institution is usually launched with great fanfare, goodwill and a huge effort on the part of the whole development team to ensure a worthwhile cohesive set of learning experiences aligned to the desired course learning outcomes. It is often not long before the glue starts to come unstuck arising from staffing changes, subtle migration of course resources, opportunistic inclusions of “off the shelf” or unit based innovative teaching and learning approaches, and perhaps general poor attention to detail with regard to the impact of new introductions and electives. This paper presents an initial investigation into the elusive goal of achieving course cohesion. The authors consider building cohesion into a course as it is being designed through identified cohesion factors and in sustaining course cohesion through active leadership.

Audience response systems can facilitate communal course feedback

Background At Queensland University of Technology (QUT), the Bachelor of Radiation Therapy course evaluation has previously suffered from low online survey participation rates. A communal instantaneous feedback event using an audience response system (ARS) was evaluated as a potential solution to this problem. The aims of the project were to determine the extent to which this feedback event could be facilitated by ARS technology and to evaluate the impact the technology made on student satisfaction and engagement. Methods Students were invited to a timetabled session to provide feedback on individual study units and the course overall. They provided quantitative Likert-style responses to prompts for each unit and the course using an ARS as well as anonymous typed qualitative comments. Data collection was performed live so students were able to view collective class responses. This prompted further discussion and enabled a prospective action plan to be developed. To inform future ARS use, students were asked for their opinions on the feedback method. Results Despite technological difficulties, student evaluation indicated that all responders enjoyed the session and the opportunity to view the combined responses. All students felt that useful feedback was generated and that this method should be used in the future. The student attendance and response rates were high, and it was clear that the session had led to the development of some insightful qualitative feedback comments. Conclusions: An ARS contributed well to the collection of course feedback in a communal and interactive environment. Students found it enjoyable to use, and it helped to stimulate useful qualitative comments

Faculty Instructional Quality Assessment Practices Incorporating Teaching Portfolios and Peer Review with Student Course Evaluations

Program directors and department chairs require different means of assessing faculty quality due to the unreliability of student course evaluation data. This report outlines alternative strategies for review committees to assess faculty instructional quality. This report also details incorporation of annual performance reviews for tenure-track faculty into tenure decisions.

A university extension course in leprosy: telemedicine in the Amazon for primary healthcare

There is a high prevalence of leprosy in the Amazon region of Brazil. We have developed a distance education course in leprosy for training staff of the Family Health Teams (FHTs). The course was made available through a web portal. Tele-educational resources were mediated by professors and coordinators, and included the use of theoretical content available through the web, discussion lists, Internet chat, activity diaries, 3-D video animations (Virtual Human on Leprosy), classes in video streaming and case simulation. Sixty-five FHT staff members were enrolled. All of them completed the course and 47 participants received a certificate at the end of the course. At the end of the course, 48 course-evaluation questionnaires were answered. A total of 47 participants (98%) considered the course as excellent. The results demonstrate the feasibility of an interactive, tele-education model as an educational resource for staff in isolated regions. Improvements in diagnostic skills should increase diagnostic suspicion of leprosy and may contribute to early detection.

PBL and B-Learning for Civil Engineering Students in a Transportation Course

The results of a pedagogical strategy implemented at the University of Sao Paulo at Sao Carlos are presented and discussed. The initiative was conducted in a transportation course offered to Civil Engineering students. The approach is a combination of problem-based learning and project-based learning (PBL) and blended-learning (B-learning). Starting in 2006, a different problem was introduced every year. From 2009 on, however, the problem-based learning concept was expanded to project-based learning. The performance of the students was analyzed using the following elements: (1) grades in course activities; (2) answers from a questionnaire designed for course evaluation; and (3) cognitive maps made to assess the effects of PBL through the comparison of the responses provided by the students involved and those not involved in the experiment. The results showed positive aspects of the method, such as a strong involvement of several students with the subject. A gradual increase in the average scores obtained by the students in the project activities (from 6.77 in 2006 to 8.24 in 2009) was concomitant with a better evaluation of these activities and of the course as a whole (90 and 97% of options "Good" or "Very good" in 2009, respectively). A growing interest in the field of transportation engineering as an alternative for further studies was also noticed. DOI: 10.1061/(ASCE)EI.1943-5541.0000115. (C) 2012 American Society of Civil Engineers.

The effect of mentoring electronic mail on student achievement and attitudes in a graduate course in educational research

This exploratory study of a classroom with mentoring and neutral e-mail was conducted in a public commuter state university in South Florida between January 1996 and April 1996. Sixteen males and 83 females from four graduate level educational research classes participated in the study.^ Two main hypotheses were tested. Hypothesis One was that those students receiving mentoring e-mail messages would score significantly higher on an instrument measuring attitude toward educational research (ATERS) than those not receiving mentoring e-mail messages. Hypothesis Two was that those students receiving mentoring e-mail would score significantly higher on objective exams covering the educational research material than those not receiving mentoring e-mail.^ Results of factorial analyses of variance showed no significant differences between the treatment groups in achievement or in attitudes toward educational research. Introverts had lower attitudes and lower final exam grades in both groups, although introverts in the mentored group scored higher than those introverts in the neutral group.^ A t test of the means of total response to e-mail from the researcher showed a significant difference between the mentored and neutral e-mail groups. Introverts responded more often than extraverts in both groups.^ Teacher effect was significant in determining class response to e-mail messages. Responses were most frequent in the researcher's classes.^ Qualitative analyses of the e-mail and course evaluation survey and of the content of e-mail messages received by the researcher were then grouped into basic themes and discussed.^ A qualitative analysis of an e-mail and course evaluation survey revealed that students from both the neutral and mentoring e-mail groups appreciated teacher feedback. A qualitative analysis of the mentoring and neutral e-mail replies divided the responses into those pertaining to the class, such as test and research paper questions, and more personal items, such as problems in the class and personal happenings.^ At this point in time, e-mail is not a standard way of communicating in classes in the college of education at this university. As this technology tool of communication becomes more popular, it is anticipated that replications of this study will be warranted. ^

Adherence, adaptation and acceptance of elderly chronic heart failure patients to receiving healthcare via telephone-monitoring

Background: Although the potential to reduce hospitalisation and mortality in chronic heart failure (CHF) is well reported, the feasibility of receiving healthcare by structured telephone support or telemonitoring is not. Aims: To determine; adherence, adaptation and acceptability to a national nurse-coordinated telephone-monitoring CHF management strategy. The Chronic Heart Failure Assistance by Telephone Study (CHAT). Methods: Triangulation of descriptive statistics, feedback surveys and qualitative analysis of clinical notes. Cohort comprised of standard care plus intervention (SC + I) participants who completed the first year of the study. Results: 30 GPs (70% rural) randomised to SC + I recruited 79 eligible participants, of whom 60 (76%) completed the full 12 month follow-up period. During this time 3619 calls were made into the CHAT system (mean 45.81 SD ± 79.26, range 0-369), Overall there was an adherence to the study protocol of 65.8% (95% CI 0.54-0.75; p = 0.001) however, of the 60 participants who completed the 12 month follow-up period the adherence was significantly higher at 92.3% (95% CI 0.82-0.97, p ≤ 0.001). Only 3% of this elderly group (mean age 74.7 ±9.3 years) were unable to learn or competently use the technology. Participants rated CHAT with a total acceptability rate of 76.45%. Conclusion: This study shows that elderly CHF patients can adapt quickly, find telephone-monitoring an acceptable part of their healthcare routine, and are able to maintain good adherence for a least 12 months. © 2007.

Evaluating leadership development using the Most Significant Change technique

Course evaluations are now a serious matter for universities trying to meet stakeholder needs and expectations, quality assurance, improvements and strategic decision making. Typically, students are invited to participate in surveys on how well the design and delivery aspects meet predetermined learning objectives, quality of teaching, and the types of improvements needed for future deliveries. We used the Most Significant Change technique to gather data on the impact of a leadership course on 18 Pacific Islanders who completed a Master of Education (Educational Leadership). Participants' views highlighted impacts that were of significance to the students and their workplaces. The findings demonstrate that the Most Significant Change technique offers a more comprehensive understanding of the impact of leadership development courses.

Studying industrial design : Exploring the experience of female students

Despite a significant increase in the number of women enrolling and graduating from design courses, the reality is that women remain ‘invisible’ in the design profession (Bruce, 1985). Over two decades ago, Bruce and Lewis (1990) argued that women were less likely than men to be designers due to three key gendered hurdles: the completion of a design degree, getting a design job and obtaining success in a design job. This paper focuses specifically on Australian women’s experience of hurdle one: the completion of a design degree, utilising industrial design as a case study. Semi-structured interview questions (exploring issues such as experience in class and the workshop, accessibility of the course and content, types of projects etc) were recorded and transcribed verbatim, with a thematic analysis conducted to better understand women’s experiences in completing their industrial design degree. This paper focuses on one key theme “navigating the design studio”, which comprises of three sub-themes: design skill development, the workshop experience and course evaluation. These findings highlight the need to understand the educational experience to ensure female designers remain motivated and eventually employable.

The practical examination and how it correlates with the triple jump, tutorial, and written examination

This study explored the relationship between the practical examination and other course evaluation methods~ specifically, the triple jump, tutorial, and written examination. Studies correlating academic and clinical grades tended to indicate that they may not be highly correlated because each evaluation process contributes different kinds of information regarding student knowledge, skills, and attitudes. Six hypotheses were generated stating a positive relationship between the four evaluation methods. A correlation matrix was produced of the Pearson Product Moment correlation co-efficients on the four evaluation methods in the second and third year Occupational Therapy Technique and Clinical Problem Solving courses of the 1988 and 1989 graduates (n~45). The results showed that the highest correlations existed between the triple jump and the tutorial grades and the lowest correlations existed between the practical examination and written examination grades. Not all of the correlations~ however~ reached levels of significance. The correlations overall. though, were only low to moderate at best which indicates that the evaluation methods may be measuring different aspects of student learning. This conclusion supports the studies researched. The implications and significance of this study is that it will assist the faculty in defining what the various evaluation methods measure which will in turn promote more critical input into curriculum development for the remaining years of the program.

Adequação do currículo do curso de pedagogia: habilitação em administração escolar e orientação educacional: ao mercado de trabalho

O objetivo deste trabalho foi o de estudar o currículo do Curso de Pedagogia (habilitação em Administração Escolar e Orientação Educacional) da Faculdade de Educação da Universidade do Amazonas. Procurou-se examinar também a relação existente entre a formação e as condições de desempenho de Administradores Escolares e Orientadores Educacionais atuando nas Escolas Estaduais de 1º e 2º graus, na cidade de Manaus, Estado do Amazonas. O trabalho consta de duas etapas: a primeira contém a importância do estudo, as hipóteses que fundamentaram o mesmo e a fundamentação teórica, onde nos detivemos principalmente nas figuras dos dois especialistas: o Adm.Esc. e o O.E., em seus aspectos teóricos e atribuições legais; a segunda constitue-se de uma Pesquisa de campo, onde foram utilizados três instrumentos, abrangendo as seguites áreas: avaliação dos cursos; metodologia; estágio supervisionado; avaliação do rendimento; realidade social e condições de atuação. A população estudada consta de 80 elementos exercendo as funções de Diretores, Administradores Escolares e Orientadores Educacionais. Os resultados obtidos evidenciaram um hiato entre a formação oferecida pela Faculdade de Educação da Universidade do Amazonas, e as condições de atuaçao dos profissionais em estudo. Os Adm. Esc. (em função de Assessor) estão exercendo funções para as quais não necessitam formação em nível superior; os O.E. não dispõem de condições para uma atuação efetiva, dentro do que é preconizado pelos teóricos da Orientação Educacional; os profissionais estudados apontam como principal ponto de estrangulamento em seu desempenho o distanciamento entre a teoria e a prática oferecidas nos cursos de graduação; notou-se que os O.E. mostraram-se mais críticos quanto à formação recebida, do que os Adm. Esc. Sugere-se no final do trabalho, pesquisas que visem aprofundar o conhecimento da realidade educacional pesquisada.