Using textual analysis with Concept Inventories to identify root causes of misconceptions

Engineers must have deep and accurate conceptual understanding of their field and Concept inventories (CIs) are one method of assessing conceptual understanding and providing formative feedback. Current CI tests use Multiple Choice Questions (MCQ) to identify misconceptions and have undergone reliability and validity testing to assess conceptual understanding. However, they do not readily provide the diagnostic information about students’ reasoning and therefore do not effectively point to specific actions that can be taken to improve student learning. We piloted the textual component of our diagnostic CI on electrical engineering students using items from the signals and systems CI. We then analysed the textual responses using automated lexical analysis software to test the effectiveness of these types of software and interviewed the students regarding their experience using the textual component. Results from the automated text analysis revealed that students held both incorrect and correct ideas for certain conceptual areas and provided indications of student misconceptions. User feedback also revealed that the inclusion of the textual component is helpful to students in assessing and reflecting on their own understanding.

Teacher education for high-poverty schools : keeping the bar high

For some time now, there has been a focus, both in Australia and internationally, on quality teaching as a fundamental component that affects the educational outcomes of all students. The question of how teacher education programs in Australia prepare effective teachers to work across all school settings-including low-SES schools-has been elevated to national prominence by data from the 20 12 Programme for International Student Assessment (PIS A), which revealed a fall in Australian students' world ranking across Mathematics, Reading and Science. Education is commonly acknowledged as a "foundation capability" that improves a person's life chances, including employment prospects, and it is widely understood to be a "route out of disadvantage" (McLachlan, Gilfillan, and Gordon 20 13). The Australian Bureau of Statistics 201 1- 12 data suggest that around 2.6 million (11.8%) Australians currently live under the poverty line (Phillips et a!. 2012, 8). According to the Organisation for Economic Cooperation and Development (OECD), despite the significant effects teachers have on student performance, disadvantaged schools are not always staffed with the highest quality teachers (see Darling-Hammond, 2006).

Emotional experiences of preservice science teachers in online learning : the formation, disruption and maintenance of social bonds

The enactment of learning to become a science teacher in online mode is an emotionally charged experience. We attend to the formation, maintenance and disruption of social bonds experienced by online preservice science teachers as they shared their emotional online learning experiences through blogs, or e-motion diaries, in reaction to videos of face-to-face lessons. A multi-theoretic framework drawing on microsociological perspectives of emotion informed our hermeneutic interpretations of students’ first-person accounts reported through an e-motion diary. These accounts were analyzed through our own database of emotion labels constructed from the synthesis of existing literature on emotion across a range of fields of inquiry. Preservice science teachers felt included in the face-to-face group as they watched videos of classroom transactions. The strength of these feelings of social solidarity were dependent on the quality of the video recording. E-motion diaries provided a resource for interactions focused on shared emotional experiences leading to formation of social bonds and the alleviation of feelings of fear, trepidation and anxiety about becoming science teachers. We offer implications to inform practitioners who wish to improve feelings of inclusion amongst their online learners in science education.

Selection procedures for module discovery : exploring evolutionary algorithms for cognitive science

Evolutionary algorithms are playing an increasingly important role as search methods in cognitive science domains. In this study, methodological issues in the use of evolutionary algorithms were investigated via simulations in which procedures were systematically varied to modify the selection pressures on populations of evolving agents. Traditional roulette wheel, tournament, and variations of these selection algorithms were compared on the “needle-in-a-haystack” problem developed by Hinton and Nowlan in their 1987 study of the Baldwin effect. The task is an important one for cognitive science, as it demonstrates the power of learning as a local search technique in smoothing a fitness landscape that lacks gradient information. One aspect that has continued to foster interest in the problem is the observation of residual learning ability in simulated populations even after long periods of time. Effective evolutionary algorithms balance their search effort between broad exploration of the search space and in-depth exploitation of promising solutions already found. Issues discussed include the differential effects of rank and proportional selection, the tradeoff between migration of populations towards good solutions and maintenance of diversity, and the development of measures that illustrate how each selection algorithm affects the search process over generations. We show that both roulette wheel and tournament algorithms can be modified to appropriately balance search between exploration and exploitation, and effectively eliminate residual learning in this problem.

Navigating pathways for academic staff development : implications for institutions and academic ranks

BACKGROUND As engineering schools adopt outcomes - focused learning approaches in response to government expectations and industry requirements of graduates capable of learning and applying knowledge in different contexts, university academics must be capable of developing and delivering programs that meet these requirements. Those academics are increasingly facing challenges in progressing their research and also acquiring different skill sets to meet the learning and teaching requirements. PURPOSE The goal of this study was to identify the types of development and support structures in place for academic staff, especially early career ones, and examine how the type of institution and the rank or role of the staff member affects these structures. DESIGN/METHOD We conducted semi - structured interviews with 21 individuals in a range of positions pertaining to teaching and learning in engineering education. Open coding was used to identify main themes from the guiding questions raised in the interviews and refined to address themes relevant to the development of institutional staff . The interview data was then analysed based on the type of institution and the rank/ role of the participant. RESULTS While development programs that focus on improving teaching and learning are available, the approach on using these types of programs differed based on staff perspective. Fewer academics, regardless of rank/role, had knowledge of support structures related to other areas of scholarship, e.g. disciplinary research, educational research, learning the institutional culture. The type of institution also impacted how they weighted and encouraged multiple forms of scholarship. We found that academic staff holding higher ranking positions, e.g. dean or associate dean, were not only concerned with the success of their respective programs, but also in how to promote other academic staff participation throughout the process. CONCLUSIONS The findings from this stud y extend the premise that developing effective academic staff ultimately leads to more effective institutions and successful graduates and accomplishing this requires staff buy - in at multiple stages of instructional and program development. Staff and administration developing approaches for educational innovation together (Besterfield - Sacre et al., 2014) and getting buy - in from all academic staff to invest in engineering education development will ultimately lead to more successful engineering graduates.

Using automated text analysis to evaluate students’ conceptual understanding

Background: A major challenge for assessing students’ conceptual understanding of STEM subjects is the capacity of assessment tools to reliably and robustly evaluate student thinking and reasoning. Multiple-choice tests are typically used to assess student learning and are designed to include distractors that can indicate students’ incomplete understanding of a topic or concept based on which distractor the student selects. However, these tests fail to provide the critical information uncovering the how and why of students’ reasoning for their multiple-choice selections. Open-ended or structured response questions are one method for capturing higher level thinking, but are often costly in terms of time and attention to properly assess student responses. Purpose: The goal of this study is to evaluate methods for automatically assessing open-ended responses, e.g. students’ written explanations and reasoning for multiple-choice selections. Design/Method: We incorporated an open response component for an online signals and systems multiple-choice test to capture written explanations of students’ selections. The effectiveness of an automated approach for identifying and assessing student conceptual understanding was evaluated by comparing results of lexical analysis software packages (Leximancer and NVivo) to expert human analysis of student responses. In order to understand and delineate the process for effectively analysing text provided by students, the researchers evaluated strengths and weakness for both the human and automated approaches. Results: Human and automated analyses revealed both correct and incorrect associations for certain conceptual areas. For some questions, that were not anticipated or included in the distractor selections, showing how multiple-choice questions alone fail to capture the comprehensive picture of student understanding. The comparison of textual analysis methods revealed the capability of automated lexical analysis software to assist in the identification of concepts and their relationships for large textual data sets. We also identified several challenges to using automated analysis as well as the manual and computer-assisted analysis. Conclusions: This study highlighted the usefulness incorporating and analysing students’ reasoning or explanations in understanding how students think about certain conceptual ideas. The ultimate value of automating the evaluation of written explanations is that it can be applied more frequently and at various stages of instruction to formatively evaluate conceptual understanding and engage students in reflective

Virtual 3D models of insects for accelerated quarantine control

We learn from the past that invasive species have caused tremendous damage to native species and serious disruption to agricultural industries. It is crucial for us to prevent this in the future. The first step of this process is to identify correctly an invasive species from native ones. Current identification methods, relying on mainly 2D images, can result in low accuracy and be time consuming. Such methods provide little help to a quarantine officer who has time constraints to response when on duty. To deal with this problem, we propose new solutions using 3D virtual models of insects. We explain how working with insects in the 3D domain can be much better than the 2D domain. We also describe how to create true-color 3D models of insects using an image-based 3D reconstruction method. This method is ideal for quarantine control and inspection tasks that involve the verification of a physical specimen against known invasive species. Finally we show that these insect models provide valuable material for other applications such as research, education, arts and entertainment. © 2013 IEEE.

Capturing natural-colour 3D models of insects for species discovery and diagnostics

Collections of biological specimens are fundamental to scientific understanding and characterization of natural diversity - past, present and future. This paper presents a system for liberating useful information from physical collections by bringing specimens into the digital domain so they can be more readily shared, analyzed, annotated and compared. It focuses on insects and is strongly motivated by the desire to accelerate and augment current practices in insect taxonomy which predominantly use text, 2D diagrams and images to describe and characterize species. While these traditional kinds of descriptions are informative and useful, they cannot cover insect specimens "from all angles" and precious specimens are still exchanged between researchers and collections for this reason. Furthermore, insects can be complex in structure and pose many challenges to computer vision systems. We present a new prototype for a practical, cost-effective system of off-the-shelf components to acquire natural-colour 3D models of insects from around 3 mm to 30 mm in length. ("Natural-colour" is used to contrast with "false-colour", i.e., colour generated from, or applied to, gray-scale data post-acquisition.) Colour images are captured from different angles and focal depths using a digital single lens reflex (DSLR) camera rig and two-axis turntable. These 2D images are processed into 3D reconstructions using software based on a visual hull algorithm. The resulting models are compact (around 10 megabytes), afford excellent optical resolution, and can be readily embedded into documents and web pages, as well as viewed on mobile devices. The system is portable, safe, relatively affordable, and complements the sort of volumetric data that can be acquired by computed tomography. This system provides a new way to augment the description and documentation of insect species holotypes, reducing the need to handle or ship specimens. It opens up new opportunities to collect data for research, education, art, entertainment, biodiversity assessment and biosecurity control. © 2014 Nguyen et al.

“I was proud of myself that I didn’t give up and I did it”: Experiences of pride and triumph in learning science

The role that specific emotions, such as pride and triumph, play during instruction in science education is an under-researched field of study. Emotions are recognized as central to learning yet little is known about the way in which they are produced in naturalistic settings, how emotions relate to classroom learning during interactions, and what antecedent factors are associated with emotional experiences during instruction. Data sources for the study include emotion diaries, student written artifacts, video recordings of class interactions, and interviews. Emotions produced in the moment during classroom interactions are analyzed from video data and audio data through a novel theoretical framework related to the sociology of human emotions. These direct observations are compared with students’ recollected emotional experiences reported through emotion diaries and interviews. The study establishes links between pride and triumph within classroom interactions and instructional tasks during learning episodes in a naturalistic setting. We discuss particular classroom activities that are associated with justified feelings of pride and triumph. More specifically, classroom events associated with these emotions were related to understanding science concepts, social interactions, and achieving success on challenging tasks.

Science, Technology, Engineering, and Maths (STEM)

"Historically, science had a place in education before the time of Plato and Aristotle (e.g., Stonehenge). Technology gradually increased since early human inventions (e.g., indigenous tools and weapons), rose up dramatically through the industrial revolution and escalated exponentially during the twentieth and twenty-first centuries, particularly with the advent of the Internet. Engineering accomplishments were evident in the constructs of early civil works, including roads and structural feats such as the Egyptian pyramids. Mathematics was not as clearly defined BC (Seeds 2010), but was utilized for more than two millennia (e.g., Archimedes, Kepler, and Newton) and paved its way into education as an essential scientific tool and a way of discovering new possibilities. Hence, combining science, technology, engineering, and mathematics (STEM) areas should not come as a surprise but rather as a unique way of packaging what has been ..."--Publisher Website

Creating good lives through computer games

In this chapter we aim to explore how videogames can lead to improvements in wellbeing. Following Keyes (2007) and Huppert and So (2012) we view wellbeing as a multidimensional concept with both hedonic and eudaimonic aspects. In this chapter we take a broad approach in terms of exploring the impact of videogames on the psychological, social, and physical components of wellbeing. We explore how videogames have been shown to have an impact in each of these domains. Although there is a great deal of evidence for the actual and potential positive impacts of videogames, there are many unanswered questions regarding the situations in which there is likely to be an impact of videogame play on wellbeing, as well as the aspects of wellbeing that are likely to be impacted by videogame play. We conclude the chapter by outlining the key questions for future research. Our focus in this chapter is on the positive influences of videogames. We do not explore research on contexts in which negative impacts are possible or subgroups for which videogames could cause harm. However, these questions are obviously important and we see balanced engagement with age-appropriate videogames as a key prerequisite for any of the wellbeing benefits discussed below.

Analysing student programs in the PHP intelligent tutoring system

Programming is a subject that many beginning students find difficult. The PHP Intelligent Tutoring System (PHP ITS) has been designed with the aim of making it easier for novices to learn the PHP language in order to develop dynamic web pages. Programming requires practice. This makes it necessary to include practical exercises in any ITS that supports students learning to program. The PHP ITS works by providing exercises for students to solve and then providing feedback based on their solutions. The major challenge here is to be able to identify many semantically equivalent solutions to a single exercise. The PHP ITS achieves this by using theories of Artificial Intelligence (AI) including first-order predicate logic and classical and hierarchical planning to model the subject matter taught by the system. This paper highlights the approach taken by the PHP ITS to analyse students’ programs that include a number of program constructs that are used by beginners of web development. The PHP ITS was built using this model and evaluated in a unit at the Queensland University of Technology. The results showed that it was capable of correctly analysing over 96 % of the solutions to exercises supplied by students.

Art and science as creative catalysts

Science, Art and Science Art collaborations are generally presented and understood in terms of their products. We argue that the process of Science art can be a significant, even principal benefit of these collaborations, even though it may be largely invisible to anyone other than the collaborators. Hosting the Centenary of Canberra Science Art Commission at the Commonwealth Scientific and Industrial Research Organisation (CSIRO) has shown us that while Science and Art pursue orthogonal dimensions of creativity and innovation, collaborators can combine these directions to access new areas of imagination and ideas.

Evolution of self-reporting methods for identifying discrete emotions in science classrooms

Emotion researchers have grappled with challenging methodological issues in capturing emotions of participants in naturalistic settings such as school or university classrooms. Self-reporting methods have been used frequently, yet these methods are inadequate when used alone. We argue that the self-reporting methods of emotion diaries and cogenerative dialogues can be helpful in identifying in-the-moment emotions when used in conjunction with the microanalysis of video recordings of classroom events. We trace the evolution of our use of innovative self-reporting methods through three cases from our research projects, and propose new directions for our ongoing development and application of these methods in both school and university classrooms.

Personas of students accessing a peer-facilitated support for learning program

STIMulate is a support for learning program at the Queensland University of Technology in Brisbane, Australia. The program provides assistance in mathematics, science and information technology for undergraduate students. This paper develops personas - archetypal users - that represent the attitudes and motivations of students that utilise STIMulate (in particular, the IT stream). Seven different personas were developed based on interviews gathered from Peer Learning Facilitators (PLF) who are experienced students that have excelled in relevant subject areas. The personas were then validated by a PLF focus group. Developing the personas enabled us to better understand the characteristics and needs of the students using the STIMulate program, enabling a more critical analysis of the quality of the service provided.