The Effect of using online homework exercises on student achievement in a Quantitative methods course

Abstract: Quantitative Methods (QM) is a compulsory course in the Social Science program in CEGEP. Many QM instructors assign a number of homework exercises to give students the opportunity to practice the statistical methods, which enhances their learning. However, traditional written exercises have two significant disadvantages. The first is that the feedback process is often very slow. The second disadvantage is that written exercises can generate a large amount of correcting for the instructor. WeBWorK is an open-source system that allows instructors to write exercises which students answer online. Although originally designed to write exercises for math and science students, WeBWorK programming allows for the creation of a variety of questions which can be used in the Quantitative Methods course. Because many statistical exercises generate objective and quantitative answers, the system is able to instantly assess students’ responses and tell them whether they are right or wrong. This immediate feedback has been shown to be theoretically conducive to positive learning outcomes. In addition, the system can be set up to allow students to re-try the problem if they got it wrong. This has benefits both in terms of student motivation and reinforcing learning. Through the use of a quasi-experiment, this research project measured and analysed the effects of using WeBWorK exercises in the Quantitative Methods course at Vanier College. Three specific research questions were addressed. First, we looked at whether students who did the WeBWorK exercises got better grades than students who did written exercises. Second, we looked at whether students who completed more of the WeBWorK exercises got better grades than students who completed fewer of the WeBWorK exercises. Finally, we used a self-report survey to find out what students’ perceptions and opinions were of the WeBWorK and the written exercises. For the first research question, a crossover design was used in order to compare whether the group that did WeBWorK problems during one unit would score significantly higher on that unit test than the other group that did the written problems. We found no significant difference in grades between students who did the WeBWorK exercises and students who did the written exercises. The second research question looked at whether students who completed more of the WeBWorK exercises would get significantly higher grades than students who completed fewer of the WeBWorK exercises. The straight-line relationship between number of WeBWorK exercises completed and grades was positive in both groups. However, the correlation coefficients for these two variables showed no real pattern. Our third research question was investigated by using a survey to elicit students’ perceptions and opinions regarding the WeBWorK and written exercises. Students reported no difference in the amount of effort put into completing each type of exercise. Students were also asked to rate each type of exercise along six dimensions and a composite score was calculated. Overall, students gave a significantly higher score to the written exercises, and reported that they found the written exercises were better for understanding the basic statistical concepts and for learning the basic statistical methods. However, when presented with the choice of having only written or only WeBWorK exercises, slightly more students preferred or strongly preferred having only WeBWorK exercises. The results of this research suggest that the advantages of using WeBWorK to teach Quantitative Methods are variable. The WeBWorK system offers immediate feedback, which often seems to motivate students to try again if they do not have the correct answer. However, this does not necessarily translate into better performance on the written tests and on the final exam. What has been learned is that the WeBWorK system can be used by interested instructors to enhance student learning in the Quantitative Methods course. Further research may examine more specifically how this system can be used more effectively.

Prévoir la Réussite des Étudiants en Techniques d’inhalothérapie et d’anesthésie

Abstract: Respiratory therapists must be able to care for their patients safely, efficiently and competently. They manage critically ill patients on life support systems. As a member of the anesthesia team they are responsible for the vulnerable patient undergoing surgery. Within all areas of the hospital they are called upon to make decisions and judgements concerning patient treatment. The environment that is found in the modern clinical setting is often stressful and demanding. The respiratory and anesthesia technology program has the responsibility of preparing competent practioners who graduate not only with a broad knowledge base but with the affective competencies that are required to meet these challenges. Faculty and clinical instructors in the program of Respiratory and Anesthesia Technology have been troubled by rising attrition rates and weak performance of students. It is apparent that this is not a problem unique to Vanier College. The rationale for this study was multi-fold; to establish a definition of student success, to determine whether pre-admission academic abilities can predict success in the program and whether scores on a professional behavioural aptitudes tool can predict success in the clinical year of the program. Predictors were sought that could be used either in the pre-program admission policies or during the course of study in order to ensure success throughout the program and beyond. A qualitative analysis involving clinical instructors and faculty (n=5) was carried out to explore what success signified for a student in the respiratory and anesthesia program. While this process revealed that a student who obtained a grade above 77.5% was considered “successful”, the concept surrounding success was a much more complex issue. Affective as well as cognitive and psychomotor abilities complete the model of the successful student. Appropriate behaviour and certain character traits in a respiratory therapy student are considered to be significant elements leading to success. Assessment of students in their clinical year of the respiratory & anesthesia technology program currently include little measurement of abilities in the affective domain, and the resulting grade becomes primarily a measure of academic and procedural skills. A quantitative study of preadmission records and final program grades was obtained from a single cohort of respiratory and anesthesia technology students who began the program in 2005 and graduated in 2008 (n=16). Data was collected and a descriptive analysis (analysis of variance, Pearson correlation) was used to determine the relationship between preadmission grades and success. The lack of association between the high school grades and grades in the program ran contrary to some of the findings in the literature and it can be cautiously inferred that preadmission grades do not predict success in the program. To ascertain the predictive significance of evaluating professional behavioural skills and success in clinical internship, a behaviour assessment tool was used by clinical instructors and faculty to score each student during a rotation in their third year of the program which was clinical internship. The results of this analysis showed that a moderately strong association could be made between a high score on the behavior assessment tool and final clinical grades. Therefore this tool may be effective in predicting success in the clinical year of the program. Refining the admissions process to meet the challenge and responsibility of turning out graduates who are capable of meeting the needs of the profession is difficult but essential. The capacity to predict which students possess the affective competencies necessary to cope and succeed in their clinical year is conceivably more important than their academic abilities. Although these preliminary findings contribute, to some degree, to the literature that exists concerning methods of predicting success in a respiratory and anesthesia technology program, much data is still unknown. Further quantitative and qualitative research is required using a broader population base to substantiate the findings of this small study.||Résumé: Les inhalothérapeutes doivent être capables de prodiguer des soins à leurs patients d’une manière sécuritaire, efficace et compétente. Ils/elles peuvent être appelé(e)s à gérer les soins aux personnes gravement malades branchées à un respirateur artificiel. En tant que membres de l’équipe d’anesthésie, ils/elles sont responsables des patients qui subissent une chirurgie. Ils/elles sont sollicité(e)s par tous les secteurs de l’hôpital pour décider ou juger des traitements à apporter aux malades. L’environnement dans lequel ils/elles travaillent est souvent stressant et exigeant. Le programme de Techniques d’inhalothérapie et d’anesthésie vise à former des inhalothérapeutes compétent(e)s qui possèdent non seulement les connaissances propres à la discipline mais également les aptitudes affectives nécessaires pour faire face à ces défis. Les enseignant(e)s et instructeur(e)s cliniques en Techniques d’inhalothérapie et d’anesthésie sont préoccupé(e)s par le taux d’abandon croissant et la faible performance des étudiant(e)s dans le programme. Il semble que ce problème ne soit pas unique au Collège Vanier. Le but de cette recherche est multiple : définir ce qu’est «réussir» pour les étudiant(e)s de ce programme; déterminer si les aptitudes scolaires acquises avant l’admission au programme peuvent aider à prévoir le succès des étudiant(e)s dans le programme; et si les résultats obtenus à un test mesurant les aptitudes comportementales professionnelles permettent de prévoir le succès des étudiant(e)s dans le stage clinique du programme. On a essayé d’identifier des facteurs qui pourraient être utilisés dans les politiques d’admission au programme ou celles régissant le cheminement dans le programme qui permettraient d’assurer le succès au cours du programme et par la suite. Une analyse qualitative a été conduite auprès des instructeur(e)s cliniques et des enseignant(e)s (n=5) afin d’étudier la notion de « réussite » des étudiant(e)s dans le programme. Bien qu’un(e) étudiant(e) ayant obtenu une note supérieure à 77.5% soit considéré(e) comme ayant « réussi », la notion de « réussite » est beaucoup plus complexe. Des aptitudes affectives, autant que cognitives et psychomotrices complètent le modèle d’un(e) étudiant(e) ayant réussi. Un comportement approprié et certains traits de caractère sont considérés comme des facteurs importants pour la réussite d’un(e) étudiant(e) en techniques d’inhalothérapie et d’anesthésie. L’évaluation qui se fait actuellement des étudiant(e)s dans le stage clinique du programme ne porte que peu sur les aptitudes affectives, et le résultat obtenu témoigne essentiellement des aptitudes scolaires et procédurales. Une analyse quantitative des dossiers des étudiant(e)s avant leur admission au programme et leurs notes finales a été conduite auprès d’une cohorte d’étudiant(e)s ayant commencé le programme en 2005 et gradué en 2008 (n=16). Des données ont été recueillies et une analyse descriptive (analyse de la variance, corrélation de Pearson) ont été faites afin de déterminer l’existence d’un lien entre les notes obtenues au secondaire et celles obtenues dans le programme. L’absence de corrélation entre les deux catégories de notes va à l’encontre de certaines recherches publiées et on peut déduire avec réserve que les notes obtenues avant l’admission au programme ne permettent pas de prévoir la réussite dans le programme. Afin de vérifier la portée de l’évaluation du comportement professionnel et de la réussite en milieu clinique quant à la prévision de réussite dans le programme, une méthode d’évaluation du comportement a été appliquée par les instructeurs(e) cliniques et les enseignant(e)s pour évaluer chaque étudiant(e) au cours d’une rotation dans leur troisième année de stage clinique. Les résultats de cette analyse ont démontré qu’une corrélation moyennement forte pouvait être faite entre une bonne note à l’évaluation comportementale et les notes finales du stage clinique. Perfectionner le processus d’admission au programme afin d’assumer la responsabilité de former des diplômé(e)s capables de répondre aux besoins de la profession est difficile mais essentiel. Avoir les moyens de prévoir quels/quelles étudiant(e)s ont les compétences affectives nécessaires pour faire face à la réussite de leur année de stage clinique est peut être plus important que d’avoir les aptitudes scolaires. Bien que ces observations préliminaires contribuent, à un certain degré, à la littérature existante sur les méthodes de prévoir la réussite dans le programme d’inhalothérapie et d’anesthésie, plusieurs données restent inconnues. Une recherche quantitative et qualitative plus élaborée, conduite sur un échantillon plus large de la population, est nécessaire afin de corroborer les résultats de cette étude limitée.