Using automated text analysis to evaluate students’ conceptual understanding

Background: A major challenge for assessing students’ conceptual understanding of STEM subjects is the capacity of assessment tools to reliably and robustly evaluate student thinking and reasoning. Multiple-choice tests are typically used to assess student learning and are designed to include distractors that can indicate students’ incomplete understanding of a topic or concept based on which distractor the student selects. However, these tests fail to provide the critical information uncovering the how and why of students’ reasoning for their multiple-choice selections. Open-ended or structured response questions are one method for capturing higher level thinking, but are often costly in terms of time and attention to properly assess student responses. Purpose: The goal of this study is to evaluate methods for automatically assessing open-ended responses, e.g. students’ written explanations and reasoning for multiple-choice selections. Design/Method: We incorporated an open response component for an online signals and systems multiple-choice test to capture written explanations of students’ selections. The effectiveness of an automated approach for identifying and assessing student conceptual understanding was evaluated by comparing results of lexical analysis software packages (Leximancer and NVivo) to expert human analysis of student responses. In order to understand and delineate the process for effectively analysing text provided by students, the researchers evaluated strengths and weakness for both the human and automated approaches. Results: Human and automated analyses revealed both correct and incorrect associations for certain conceptual areas. For some questions, that were not anticipated or included in the distractor selections, showing how multiple-choice questions alone fail to capture the comprehensive picture of student understanding. The comparison of textual analysis methods revealed the capability of automated lexical analysis software to assist in the identification of concepts and their relationships for large textual data sets. We also identified several challenges to using automated analysis as well as the manual and computer-assisted analysis. Conclusions: This study highlighted the usefulness incorporating and analysing students’ reasoning or explanations in understanding how students think about certain conceptual ideas. The ultimate value of automating the evaluation of written explanations is that it can be applied more frequently and at various stages of instruction to formatively evaluate conceptual understanding and engage students in reflective

Understanding welding practices on shipyards : an ethnographic study for designing an interactive robot welder

The aim of this ethnographic study was to understand welding practices in shipyard environments with the purpose of designing an interactive welding robot that can help workers with their daily job. The robot is meant to be deployed for automatic welding on jack-up rig structures. The design of the robot turns out to be a challenging task due to several problematic working conditions on the shipyard, such as dust, irregular floor, high temperature, wind variations, elevated working platforms, narrow spaces, and circular welding paths requiring a robotic arm with more than 6 degrees of freedom. Additionally, the environment is very noisy and the workers – mostly foreigners – have a very basic level of English. These two issues need to be taken into account when designing the interactive user interface for the robot. Ideally, the communication flow between the two parties involved should be as frictionless as possible. The paper presents the results of our field observations and welders’ interviews, as well as our robot design recommendation for the next project stage.

Australia’s first Allied Health Prescribing Program – exploring participants' understanding and confidence in clinical therapy choices for patient management

BACKGROUND Globally there are emerging trends for non-medical health professionals to expand their scope of practice into prescribing. The NPS Prescribing Competencies Framework and the Health Professionals Prescribing Pathway Program are recent initiatives to assist with implementation of prescribing for allied health professionals (AHPs). For AHPs to become prescribers, training programmes must be designed to extend their knowledge of medicines information and medicine management principles with the aim of optimising medicines related outcomes for patients. AIM To explore the understanding and confidence in clinical therapeutic choices for patient management of those AHPs enrolled in the Allied Health Prescribing Training Program Module One: Introduction to clinical therapeutics for prescribers, delivered by Queensland University of Technology, Brisbane. METHOD A pre-post survey was developed to explore key themes around understanding and confidence in selecting therapeutic choices for patients with varying complexities of conditions. Data were collected from participants in week one and 13 of the module via an online survey using a five-point Likert scale (1 = Strongly Agree (SA) to 5 = Strongly Disagree (SD)). RESULTS In the pre-Module survey the AHPs had a limited degree (D/SD) of understanding and confidence regarding the safe and effective use of medicines and appropriate therapeutic choices for managing patients, particularly with complex patients. This improved significantly in the post Module survey (A/SA).

Understanding empathy : its origins and measurement

The paper addresses the topic that was important at the World Economic Forum in Davos (Jan. 2014)namely 'Empathetic leadership in organisations'. Research is presented supporting the work of Cambridge researcher, Simon Baron-Cohen and his theory of Systemizers and Empathizers.An argument is made for the importance of world decision makers having an empathetic approach to international and national decision making. The article contends that their work will directly impact on the health and future of life on our planet.

The unequal distribution of road trauma in China : understanding risk factors to improve public health

The World Health Organization identifies road trauma as a major public health issue in every country; most notably among low-to-middle income countries. More than 90% of all road fatalities occur in these countries, although they have only 48% of all registered vehicles [1]. Unprecedented focus has been placed on reducing the global road trauma burden through the United Nations Decade of Action for Road Safety (2011-2020). China is rapidly transitioning from a nation of bicycle riders and pedestrians to one where car ownership and use is increasing. This transition presents important public health, mobility, and safety challenges. Rapid motorisation has resulted in an increased road trauma burden, shouldered disproportionately among the population. Vulnerable road users (bicyclists, pedestrians, and motorcyclists) are of particular concern, representing 70% of all road-related fatalities [1]. Furthermore, those at greatest risk of sustaining a crash-related disability are: male, older, less educated, and earning a lower income [2] and residing in urban areas [3], with higher fatality rates in north-western poorer provinces [3]. Speeding is a key factor in road crashes in China [1, 4] and is one of two risk factors targeted in the Bloomberg Philanthropies-funded Global Road Safety Program operating in two Chinese cities over five year [5] to which the first author has provided expert advice. However, little evidence exists to help understand the factors underpinning speeding behaviour. Previous research conducted by the authors in Beijing and Hangzhou explored personal, social, and legal factors relating to speeding to assist in better understanding the motivations for non-compliance with speed limits. Qualitative and quantitative research findings indicated that speeding is relatively common, including self-reported travel speeds of greater than 30 km/hour above posted speed limits [6], and that the road safety laws and enforcement practices may, in some circumstances, contribute to this [7]. Normative factors were also evident; the role of friends, family members and driving instructors were influential. Additionally, using social networks to attempt to avoid detection and penalty was reported, thereby potentially reinforcing community perceptions that speeding is acceptable [8, 9]. The authors established strong collaborative links with the Chinese Academy of Sciences and Zhejiang Police College to conduct this research. The first author has worked in both institutions for extended time periods and recognises that research must include an understanding of culturally-relevant issues if road safety is to improve in China. Future collaborations to assist in enhancing our understanding of such issues are welcomed. References [1] World Health Organization. (2009). Global status report on road safety: Time for action; Geneva. [2] Chen, H., Du, W., & Li, N. (2013). The socioeconomic inequality in traffic-related disability among Chinese adults: the application of concentration index. Accident Analysis & Prevention, 55(101-106). [3] Wang, S. Y., Li, Y. H., Chi, G. B., Xiao, S. Y., Ozanne-Smith, J., Stevenson, M., & Phillips, M. (2008). Injury-related fatalities in China: an under-recognised public-health problem. The Lancet (British edition), 372(9651), 1765-1773. [4] He, J., King, M. J., Watson, B., Rakotonirainy, A., & Fleiter, J. J. (2013). Speed enforcement in China: National, provincial and city initiatives and their success. Accident Analysis & Prevention, 50, 282-288. [5] Bhalla, K., Li, Q., Duan, L., Wang, Y., Bishai, D., & Hyder, A. A. (2013). The prevalence of speeding and drink driving in two cities in China: a mid project evaluation of ongoing road safety interventions. Injury, 44, 49-56. doi:10.1016/S0020-1383(13)70213-4. [6] Fleiter, J. J., Watson, B., & Lennon, A. (2013). Awareness of risky behaviour among Chinese drivers. Peer-reviewed paper presented at 23rd Canadian Multidisciplinary Road Safety Conference, Montréal, Québec. [7] Fleiter, J. J., Watson, B., Lennon, A., King, M. J., & Shi, K. (2009). Speeding in Australia and China: A comparison of the influence of legal sanctions and enforcement practices on car drivers. Peer-reviewd paper presented at Australasian Road Safety Research Policing Education Conference, Sydney. [8] Fleiter, J. J., Watson, B., Lennon, A., King, M. J., & Shi, K. (2011). Social influences on drivers in China. Journal of the Australasian College of Road Safety, 22(2), 29-36. [9] Fleiter, J. J., Watson, B., Guan, M. Q., Ding, J. Y., & Xu, C. (2013). Characteristics of Chinese Drivers Attending a Mandatory Training Course Following Licence Suspension. Peer-reviewed paper presented at Road Safety on Four Continents, Beijing, China.

Children's understanding of sustainability in their home and kindergarten

This article presents an ongoing study within early childhood education for sustainability at the Department for Early Childhood Education in Stavanger, in collaboration with the School of Early Childhood, Queensland University of Technology, in Brisbane, Australia. The study commenced in 2014, and will compare the responses to interviews of young children (4-5 years) in Brisbane with those of children in Stavanger, in order to find out ways to enhance learning about sustainability topics, to identify which kind of environmental/sustainability activities are memorable for young children, and to obtain mutual inspiration about early childhood education for sustainability from the different countries.

Reason of state and a new notion of fortune : Historicising the risk society

Since the 1980s the concept of risk has produced a large and diverse volume of sociological research. Ulrich Beck’s groundbreaking risk society thesis provides a particularly engaging contribution, since it seems that nearly every sociological account of risk engages with this work. For Beck, we are living in second modernity – a new epoch that breaks with pre-modernity and industrial society due to the centrality, incalculability and reflexivity of globalised risk. While Beck’s theory is compelling, a reading of other theorists such as Foucault (2007[1978]) and Hacking (1975,1990) suggests that a difficulty with Beck’s work is that in attempting to explain what is novel about risk in contemporary times, he too quickly passes over the complexities and ruptures of historical change that impact on the history and contingency of risk. This paper begins by presenting a brief analysis of the present state of risk by introducing Beck’s historical narrative of risk from pre-modernity to the risk society; it then outlines the challenges with the “risk as epoch” argument by considering a range of literature, which suggests risk has a more complex history than proposed by Beck; and finally it highlights the value in examining strategies of statecraft in early modern Europe, specifically Machiavelli’s The Prince (2008[1513]) and Giovanni Botero’s political treatise, Della Ragion di Stato (1956[1589]) – as a means of more thoroughly understanding how our current concept of risk emerges. In doing so, this paper seeks to open up new trajectories in the historicisation of risk for other interested scholars.

Development of fourth-grade students’ understanding of experimental and theoretical probability

Students explored variation and expectation in a probability activity at the end of the first year of a 3-year longitudinal study across grades 4-6. The activity involved experiments in tossing coins both manually and with simulation using the graphing software, TinkerPlots. Initial responses indicated that the students were aware of uncertainty, although an understanding of chance concepts appeared limited. Predicting outcomes of 10 tosses reflected an intuitive notion of equiprobability, with little awareness of variation. Understanding the relationship between experimental and theoretical probability did not emerge until multiple outcomes and representations were generated with the software.

Politics and the Press in Indonesia : Understanding an Evolving Political Culture

This book explores the evolving political culture in Indonesia, by discussing the country's dominant political philosophies, then showing how those philosophies affect the working lives of ordinary Indonesian citizens. It focuses in particular on the working lives of news journalists, a group that occupies a strategic social and political position.

Unified user experience model enabling a more comprehensive understanding of emotional experience design

Design deals with improving the lives of people. As such interactions with products, interfaces, and systems should facilitate not only usable and practical concerns but also mediate emotionally meaningful experiences. This paper presents an integrated and comprehensive model of experience, labeled 'Unified User Experience Model', covering the most prominent perspectives from across the design field. It is intended to support designers from different disciplines to consider the complexity of user experience. The vision of the model is to support both the analysis of existing products, interfaces, and systems, as well as the development of new designs that take into account this complexity. In essence, we hope the model can enable designers to develop more marketable, appropriate, and enhanced products to improve experiences and ultimately the lives of people.

Understanding preservice teachers’ development of pedagogical knowledge practices when co-teaching primary science to peers

Preservice teachers articulate the need for more teaching experiences for developing their practices, however, extending beyond existing school arrangements may present difficulties. Thus, it is important to understand preservice teachers’ development of pedagogical knowledge practices when in the university setting. This mixed-method study investigated 48 second-year preservice teachers’ development of pedagogical knowledge practices as a result of co-teaching primary science to peers. Data were collected through a survey, video-recorded lessons, extended written responses and researcher observations. The study showed how these preservice teachers demonstrated 9 of 11 pedagogical knowledge practices within the co-teaching arrangement. However, research is needed to determine the level of development on each pedagogical knowledge practice and how these practices can be transferred into authentic primary classroom settings.

Gen Y recruitment: Understanding graduate intentions to join an organisation using the Theory of Planned Behaviour

Purpose Using the Theory of Planned Behaviour (TPB) framework, this study explored whether the standard TPB constructs explained variance in Gen Y individuals’ intentions to join their ideal organisation. Design/methodology/approach A mixed methods approach was used featuring qualitative and quantitative methods. Findings The overall TPB model accounted for a significant 51.6% of the variance in intention to join one’s ideal organisation in the next six months with the significant predictors in the model being subjective norm and perceived behaviour control but not attitude. Research limitations/implications Using graduating students from a single Australian university sample may mean that the current findings may not extend to all Gen Y individuals. The current study has demonstrated the explanatory utility of the TPB in relation to graduate Gen Y’s intention to join their ideal organisation, providing further evidence of the robustness of the TPB framework in an organisational setting. Practical implications These findings have implications for enhancing understanding of the most effective recruitment processes for Gen Y students entering the workforce. The findings could inform recruitment policies and strategies to attract Gen Y applicants. Originality/value To our knowledge this study is the first application of the TPB to this topic. The current research extends the recruitment literature with a theoretically-based investigation. Identification of factors which inform organisational recruitment strategies, allow organisations to stand out from their competitors and potentially achieve a larger application pool from which to select the best human capital and sustain competitive advantage.

Investigative trends in understanding penalty-kick performance in association football: An ecological dynamics perspective

There have been different approaches to studying penalty-kick performance in association football. In this paper, the authors synthesize key findings within an ecological dynamics theoretical framework. According to this theoretical perspective, information is the cornerstone for understanding the dynamics of action regulation in penalty-kick performance. Research suggests that investigators need to identify the information sources that are most relevant to penalty-kick performance. An important task is to understand how constraints can channel (i.e. change, emphasize or mask) information sources used to regulate upcoming actions and how the influence of these constraints is expressed in players' behavioural dynamics. Due to the broad range of constraints influencing penalty-kick performance, it is recommended that future research adopts an interdisciplinary focus on performance assessment to overcome the current lack of representativeness in penalty-kick experimental designs. Such an approach would serve to capture the information-based control of action of both players as components of this dyadic system in competitive sport.