Development of the Heat and Energy Concept Inventory: Preliminary Results on the Prevalence and Persistence of Engineering Students' Misconceptions

BACKGROUND Students frequently hold a number of misconceptions related to temperature, heat and energy. There is not currently a concept inventory with sufficiently high internal reliability to assess these concept areas for research purposes. Consequently, there is little data on the prevalence of these misconceptions amongst undergraduate engineering students. PURPOSE (HYPOTHESIS) This work presents the Heat and Energy Concept Inventory (HECI) to assess prevalent misconceptions related to: (1) Temperature vs. Energy, (2) Temperature vs. Perceptions of Hot and Cold, (3) Factors that affect the Rate vs. Amount of Heat Transfer and (4) Thermal Radiation. The HECI is also used to document the prevalence of misconceptions amongst undergraduate engineering students. DESIGN/METHOD Item analysis, guided by classical test theory, was used to refine individual questions on the HECI. The HECI was used in a one group, pre-test-post-test design to assess the prevalence and persistence of targeted misconceptions amongst a population of undergraduate engineering students at diverse institutions. RESULTS Internal consistency reliability was assessed using Kuder-Richardson Formula 20; values were 0.85 for the entire instrument and ranged from 0.59 to 0.76 for the four subcategories of the HECI. Student performance on the HECI went from 49.2% to 54.5% after instruction. Gains on each of the individual subscales of the HECI, while generally statistically significant, were similarly modest. CONCLUSIONS The HECI provides sufficiently high estimates of internal consistency reliability to be used as a research tool to assess students' understanding of the targeted concepts. Use of the instrument demonstrates that student misconceptions are both prevalent and resistant to change through standard instruction.

Toward equity through participation in Modeling Instruction in introductory university physics

We report the results of a five year evaluation of the reform of introductory calculus-based physics by implementation of Modeling Instruction (MI) at Florida International University (FIU), a Hispanic-serving institution. MI is described in the context of FIU’s overall effort to enhance student participation in physics and science broadly. Our analysis of MI from a “participationist” perspective on learning identifies aspects of MI including conceptually based instruction, culturally sensitive instruction, and cooperative group learning, which are consistent with research on supporting equitable learning and participation by students historically under-represented in physics (i.e., Black, Hispanic, women). This study uses markers of conceptual understanding as measured by the Force Concept Inventory (FCI) and odds of success as measured by the ratio of students completing introductory physics and earning a passing grade (i.e., C− or better) by students historically under-represented in physics to reflect equity and participation in introductory physics. FCI pre and post scores for students in MI are compared with lecture-format taught students. Modeling Instruction students outperform students taught in lecture-format classes on post instruction FCI (61.9% vs 47.9%, p

THE EFFECTS OF MODELING INSTRUCTION ON STUDENT LEARNING OF A NEWTONIAN FORCE CONCEPT

This study explores the effects of modeling instruction on student learning in physics. Multiple representations grounded in physical contexts were employed by students to analyze the results of inquiry lab investigations. Class whiteboard discussions geared toward a class consensus following Socratic dialogue were implemented throughout the modeling cycle. Lab investigations designed to address student preconceptions related to Newton’s Third Law were implemented. Student achievement was measured based on normalized gains on the Force Concept Inventory. Normalized FCI gains achieved by students in this study were comparable to those achieved by students of other novice modelers. Physics students who had taken a modeling Intro to Physics course scored significantly higher on the FCI posttest than those who had not. The FCI results also provided insight into deeply rooted student preconceptions related to Newton’s Third Law. Implications for instruction and the design of lab investigations related to Newton’s Third Law are discussed.

Uncovering misconceptions through text analysis

Assessing students’ conceptual understanding of technical content is important for instructors as well as students to learn content and apply knowledge in various contexts. Concept inventories that identify possible misconceptions through validated multiple-choice questions are helpful in identifying a misconception that may exist, but do not provide a meaningful assessment of why they exist or the nature of the students’ understanding. We conducted a case study with undergraduate students in an electrical engineering course by testing a validated multiple-choice response concept inventory that we augmented with a component for students to provide written explanations for their multiple-choice selection. Results revealed that correctly chosen multiple-choice selections did not always match correct conceptual understanding for question testing a specific concept. The addition of a text-response to multiple-choice concept inventory questions provided an enhanced and meaningful assessment of students’ conceptual understanding and highlighted variables associated with current concept inventories or multiple choice questions.

Insights into students' conceptual understanding using textual analysis: A case study in signal processing

Concept inventory tests are one method to evaluate conceptual understanding and identify possible misconceptions. The multiple-choice question format, offering a choice between a correct selection and common misconceptions, can provide an assessment of students' conceptual understanding in various dimensions. Misconceptions of some engineering concepts exist due to a lack of mental frameworks, or schemas, for these types of concepts or conceptual areas. This study incorporated an open textual response component in a multiple-choice concept inventory test to capture written explanations of students' selections. The study's goal was to identify, through text analysis of student responses, the types and categorizations of concepts in these explanations that had not been uncovered by the distractor selections. The analysis of the textual explanations of a subset of the discrete-time signals and systems concept inventory questions revealed that students have difficulty conceptually explaining several dimensions of signal processing. This contributed to their inability to provide a clear explanation of the underlying concepts, such as mathematical concepts. The methods used in this study evaluate students' understanding of signals and systems concepts through their ability to express understanding in written text. This may present a bias for students with strong written communication skills. This study presents a framework for extracting and identifying the types of concepts students use to express their reasoning when answering conceptual questions.

Newtonen mugimenduaren legeen azterketa Lehen Hezkuntzan

[Eus] Lan honetan Lehen Hezkuntzako hirugarren zikloko ikasleek fisikaren barnean Newtonen mugimenduen legeei buruz zer dakitenaren azterketa kuantitatibo bat egitea izan da. Horretarako, Gernika-Lumoko Seber Altube ikastolako 94 ikasleri Lehen Hezkuntzako ikasleentzat moldatutako FCI (Force Concept Inventory) testa pasatu zitzaien, ondoren emaitzak aztertzeko. Egindako ikerketan aurkitutako emaitzek, aurretiaz gai honi buruz izan diren lanekin bat egiten dute, generoaren araberako ezberdintasunak baztertuz eta Newtonen legeen artean emaitzen arteko ezberdintasun esanguratsuak azalduz.

La cohérence conceptuelle d’étudiants collégiaux en mécanique newtonienne et en métrologie

Cette thèse porte sur l’évaluation de la cohérence du réseau conceptuel démontré par des étudiants de niveau collégial inscrits en sciences de la nature. L’évaluation de cette cohérence s’est basée sur l’analyse des tableaux de Burt issus des réponses à des questionnaires à choix multiples, sur l’étude détaillée des indices de discrimination spécifique qui seront décrits plus en détail dans le corps de l’ouvrage et sur l’analyse de séquences vidéos d’étudiants effectuant une expérimentation en contexte réel. Au terme de ce projet, quatre grands axes de recherche ont été exploré. 1) Quelle est la cohérence conceptuelle démontrée en physique newtonienne ? 2) Est-ce que la maîtrise du calcul d’incertitude est corrélée au développement de la pensée logique ou à la maîtrise des mathématiques ? 3) Quelle est la cohérence conceptuelle démontrée dans la quantification de l’incertitude expérimentale ? 4) Quelles sont les procédures concrètement mise en place par des étudiants pour quantifier l’incertitude expérimentale dans un contexte de laboratoire semi-dirigé ? Les principales conclusions qui ressortent pour chacun des axes peuvent se formuler ainsi. 1) Les conceptions erronées les plus répandues ne sont pas solidement ancrées dans un réseau conceptuel rigide. Par exemple, un étudiant réussissant une question sur la troisième loi de Newton (sujet le moins bien réussi du Force Concept Inventory) montre une probabilité à peine supérieure de réussir une autre question sur ce même sujet que les autres participants. De nombreux couples de questions révèlent un indice de discrimination spécifique négatif indiquant une faible cohérence conceptuelle en prétest et une cohérence conceptuelle légèrement améliorée en post-test. 2) Si une petite proportion des étudiants ont montré des carences marquées pour les questions reliées au contrôle des variables et à celles traitant de la relation entre la forme graphique de données expérimentales et un modèle mathématique, la majorité des étudiants peuvent être considérés comme maîtrisant adéquatement ces deux sujets. Toutefois, presque tous les étudiants démontrent une absence de maîtrise des principes sous-jacent à la quantification de l’incertitude expérimentale et de la propagation des incertitudes (ci-après appelé métrologie). Aucune corrélation statistiquement significative n’a été observée entre ces trois domaines, laissant entendre qu’il s’agit d’habiletés cognitives largement indépendantes. Le tableau de Burt a pu mettre en lumière une plus grande cohérence conceptuelle entre les questions de contrôle des variables que n’aurait pu le laisser supposer la matrice des coefficients de corrélation de Pearson. En métrologie, des questions équivalentes n’ont pas fait ressortir une cohérence conceptuelle clairement démontrée. 3) L’analyse d’un questionnaire entièrement dédié à la métrologie laisse entrevoir des conceptions erronées issues des apprentissages effectués dans les cours antérieurs (obstacles didactiques), des conceptions erronées basées sur des modèles intuitifs et une absence de compréhension globale des concepts métrologiques bien que certains concepts paraissent en voie d’acquisition. 4) Lorsque les étudiants sont laissés à eux-mêmes, les mêmes difficultés identifiées par l’analyse du questionnaire du point 3) reviennent ce qui corrobore les résultats obtenus. Cependant, nous avons pu observer d’autres comportements reliés à la mesure en laboratoire qui n’auraient pas pu être évalués par le questionnaire à choix multiples. Des entretiens d’explicitations tenus immédiatement après chaque séance ont permis aux participants de détailler certains aspects de leur méthodologie métrologique, notamment, l’emploi de procédures de répétitions de mesures expérimentales, leurs stratégies pour quantifier l’incertitude et les raisons sous-tendant l’estimation numérique des incertitudes de lecture. L’emploi des algorithmes de propagation des incertitudes a été adéquat dans l’ensemble. De nombreuses conceptions erronées en métrologie semblent résister fortement à l’apprentissage. Notons, entre autres, l’assignation de la résolution d’un appareil de mesure à affichage numérique comme valeur de l’incertitude et l’absence de procédures d’empilement pour diminuer l’incertitude. La conception que la précision d’une valeur numérique ne peut être inférieure à la tolérance d’un appareil semble fermement ancrée.

Jeu sérieux : étude de l'effet de l'intégration d'un modèle didactique dans un jeu vidéo sur l'apprentissage des joueurs.

Atout pour la mise en œuvre, atout pour l’évaluation et de manière plus évidente atout pour la motivation, le jeu sérieux se veut une solution pédagogique pertinente dans un contexte d’éducation formel ou informel. Au niveau de la recherche, on peut se questionner quant à la valeur pédagogique d’une telle approche ainsi que sur ses principaux atouts. Dans notre projet, nous nous sommes intéressés plus particulièrement à l’apport du scénario pédagogique dans un jeu sérieux. En utilisant le jeu vidéo Mecanika, développé dans le cadre d’une maîtrise en didactique à l’UQAM et basé sur un questionnaire reconnu permettant d’identifier les conceptions des élèves en mécanique, le Force Concept Inventory (HESTENES et al., 1992), nous tenterons d’extraire l'élément principal du scénario pédagogique afin d’en évaluer l’effet sur l’apprentissage. Notre méthodologie a permis de comparer les performances d’élèves de cinquième secondaire ayant utilisé deux versions différentes du jeu. Dans un premier temps, les résultats obtenus confirment ceux observés par Boucher Genesse qui étaient déjà supérieurs à ceux habituellement cités dans les recherches impliquant le FCI. Nous avons aussi observé qu’il semble exister une relation significative entre le plaisir à jouer et l’apprentissage, ainsi qu’une relation significative entre le nombre d’interactions et la version du jeu sur le gain, ce qui confirme que le jeu produit un effet qui s’ajoute à celui du professeur. La présence d’étoiles dans le jeu original a suscité plus d’actions des élèves que la version orientée simulation qui en est démunie, ce qui semble indiquer que l’utilisation d’un jeu sérieux favorise l’implication des élèves. Cependant, l’absence d’effet significatif associé à la suppression des étoiles indique que la scénarisation n’est peut-être pas la principale cause des apprentissages observés dans le jeu Mecanika. Le choix des autres éléments présents dans chaque tableau doit aussi être considéré. Des recherches futures seraient nécessaires pour mieux comprendre ce qui favorise les apprentissages

Diez años de evaluación de la enseñanza-aprendizaje de la mecánica de Newton en escuelas de ingeniería españolas : rendimiento académico y presencia de preconceptos.

Resumen tomado de la publicación

Effect of Participant Gender, Professor Gender, and Activity Interventions on the Understanding of Concepts in Thermodynamics

Misconceptions about heat and temperature have been seen across all educational levels, even in undergraduate engineering courses. One way these misconceptions can be remediated is through instructional methods, such as inquiry-based activities. Performance on assessments in sciences and engineering has been found to vary when gender is taken into consideration. The purpose of the current study was to investigate the effects of participant gender, professor gender, and level of inquiry-based activities on the conceptual understanding of 247 undergraduate engineering students in thermodynamics. A pre-test post-test design was used. Conceptual understanding of thermodynamics was measured by students’ scores on the Concept Inventory for Engineering Thermodynamics (CIET; Vigeant, Prince & Nottis, 2011). Inquiry-based activities were developed by the researchers and given to professors who determined if they would do all, some, or none of them as they taught. Significant differences were found among participants of different gender, different gender of the professor instructing the course, and level of inquiry-based activity. The participants who were exposed to all of the activities provided didsignificantly better on the post-test than those who were only exposed to some or none of the activities. The results from this current study indicated that differences in gender, professorgender, and level of inquiry-based activity has an effect on undergraduate engineering students’ conceptual understanding of thermodynamics. Future research should investigate more factorsthat contribute to lower representation of women in the engineering field.

The Repairing of Misconceptions through the Use of Inquiry-Based Activities

Misconceptions exist in all fields of learning and develop through a person’s preconception of how the world works. Students with misconceptions in chemical engineering are not capable of correctly transferring knowledge to a new situation and will likely arrive at an incorrect solution. The purpose of this thesis was to repair misconceptions in thermodynamics by using inquiry-based activities. Inquiry-based learning is a method of teaching that involves hands-on learning and self-discovery. Previous work has shown inquiry-based methods result in better conceptual understanding by students relative to traditional lectures. The thermodynamics activities were designed to guide students towards the correct conceptual understanding through observing a preconception fail to hold up through an experiment or simulation. The developed activities focus on the following topics in thermodynamics: “internal energy versus enthalpy”, “equilibrium versus steady state”, and “entropy”. For each topic, two activities were designed to clarify the concept and assure it was properly grasped. Each activity was coupled with an instructions packet containing experimental procedure as well as pre- and post-analysis questions, which were used to analyze the effect of the activities on the students’ responses. Concept inventories were used to monitor students’ conceptual understanding at the beginning and end of the semester. The results did not show a statistically significant increase in the overall concept inventory scores for students who performed the activities compared to traditional learning. There was a statistically significant increase in concept area scores for “internal energy versus enthalpy” and “equilibrium versus steady state”. Although there was not a significant increase in concept inventory scores for “entropy”, written analyses showed most students’ misconceptions were repaired. Students transferred knowledge effectively and retained most of the information in the concept areas of “internal energy versus enthalpy” and “equilibrium versus steady state”.

The Effect of Grade Point Average on Undergraduates' Understanding of Heat Transfer

Engineering students continue to develop and show misconceptions due to prior knowledge and experiences (Miller, Streveler, Olds, Chi, Nelson, & Geist, 2007). Misconceptions have been documented in students’ understanding of heat transfer(Krause, Decker, Niska, Alford, & Griffin, 2003) by concept inventories (e.g., Jacobi,Martin, Mitchell, & Newell, 2003; Nottis, Prince, Vigeant, Nelson, & Hartsock, 2009). Students’ conceptual understanding has also been shown to vary by grade point average (Nottis et al., 2009). Inquiry-based activities (Nottis, Prince, & Vigeant, 2010) haveshown some success over traditional instructional methods (Tasoglu & Bakac, 2010) in altering misconceptions. The purpose of the current study was to determine whether undergraduate engineering students’ understanding of heat transfer concepts significantly changed after instruction with eight inquiry-based activities (Prince & Felder, 2007) supplementing instruction and whether students’ self reported GPA and prior knowledge, as measured by completion of specific engineering courses, affected these changes. The Heat and Energy Concept Inventory (Prince, Vigeant, & Nottis, 2010) was used to assess conceptual understanding. It was found that conceptual understanding significantly increased from pre- to post-test. It was also found that GPA had an effect on conceptual understanding of heat transfer; significant differences were found in post-test scores onthe concept inventory between GPA groups. However, there were mixed results when courses previously taken were analyzed. Future research should strive to analyze how prior knowledge effects conceptual understanding and aim to reduce the limitations of the current study such as, sampling method and methods of measuring GPA and priorknowledge.

Assessment of a high school geological field course

Fieldwork is supportive of students’ natural inquiry abilities. Educational research findings suggest that instructors can foster the growth of thinking skills and promote science literacy by incorporating active learning strategies (McConnel et al, 2003). Huntoon (2001) states that there is a need to determine optimal learning strategies and to document the procedure of assessing those optimal geoscience curricula. This study seeks to determine if Earth Space II, a high school geological field course, can increase students’ knowledge of selected educational objectives. This research also seeks to measure any impact Earth Space II has on students’ attitude towards science. Assessment of the Earth Space II course objectives provided data on the impact of field courses on high school students’ scientific literacy, scientific inquiry skills, and understanding of selected course objectives. Knowledge assessment was done using a multiple choice format test, the Geoscience Concept Inventory, and an open-ended format essay test. Attitude assessment occurred by utilizing a preexisting science attitude survey. Both knowledge assessments items showed a positive effect size from the pretest to the posttest. The essay exam effect size was 17 and the Geoscience Concept Inventory effect size was 0.18. A positive impact on students’ attitude toward science was observed by an increase in the overall mean Likert value from the pre-survey to the post-survey.

INVESTIGATION OF STRATEGIES TO PROMOTE EFFECTIVE TEACHER PROFESSIONAL DEVELOPMENT EXPERIENCES IN EARTH SCIENCE

This dissertation serves as a call to geoscientists to share responsibility with K-12 educators for increasing Earth science literacy. When partnerships are created among K-12 educators and geoscientists, the synergy created can promote Earth science literacy in students, teachers, and the broader community. The research described here resulted in development of tools that can support effective professional development for teachers. One tool is used during the planning stages to structure a professional development program, another set of tools supports measurement of the effectiveness of a development program, and the third tool supports sustainability of professional development programs. The Michigan Teacher Excellence Program (MiTEP), a Math/Science Partnership project funded by the National Science Foundation, served as the test bed for developing and testing these tools. The first tool, the planning tool, is the Earth Science Literacy Principles (ESLP). The ESLP served as a planning tool for the two-week summer field courses as part of the MiTEP program. The ESLP, published in 2009, clearly describe what an Earth science literate person should know. The ESLP consists of nine big ideas and their supporting fundamental concepts. Using the ESLP for planning a professional development program assisted both instructors and teacher-participants focus on important concepts throughout the professional development activity. The measurement tools were developed to measure change in teachers’ Earth science content-area knowledge and perceptions related to teaching and learning that result from participating in a professional development program. The first measurement tool, the Earth System Concept Inventory (ESCI), directly measures content-area knowledge through a succession of multiple-choice questions that are aligned with the content of the professional development experience. The second measurement, an exit survey, collects qualitative data from teachers regarding their impression of the professional development. Both the ESCI and the exit survey were tested for validity and reliability. Lesson study is discussed here as a strategy for sustaining professional development in a school or a district after the end of a professional development activity. Lesson study, as described here, was offered as a formal course. Teachers engaged in lesson study worked collaboratively to design and test lessons that improve the teachers’ classroom practices. Data regarding the impact of the lesson study activity were acquired through surveys, written documents, and group interviews. The data are interpreted to indicate that the lesson study process improved teacher quality and classroom practices. In the case described here, the lesson study process was adopted by the teachers’ district and currently serves as part of the district’s work in Professional Learning Communities, resulting in ongoing professional development throughout the district.

Undergraduate Engineering Students' Understanding of Heat, Temperature, and Radiation

Difficulty understanding heat and temperature concepts has been recognized in engineering education. Confusion has been shown to persist after instruction. The purpose of this study was to determine whether undergraduate engineering students’ knowledge of four heat transfer concept areas significantly changed with instruction and whether this varied by major and GPA. Two hundred twenty-eight undergraduate engineering students from six institutions were assessed prior to and after instruction. Results showed significant improvement in most concept areas but mean scores were below mastery. Previously documented misconceptions persisted after instruction. Significant differences were found by major and GPA. Suggestions for future research provided.