Using automated text analysis to evaluate students’ conceptual understanding

Background: A major challenge for assessing students’ conceptual understanding of STEM subjects is the capacity of assessment tools to reliably and robustly evaluate student thinking and reasoning. Multiple-choice tests are typically used to assess student learning and are designed to include distractors that can indicate students’ incomplete understanding of a topic or concept based on which distractor the student selects. However, these tests fail to provide the critical information uncovering the how and why of students’ reasoning for their multiple-choice selections. Open-ended or structured response questions are one method for capturing higher level thinking, but are often costly in terms of time and attention to properly assess student responses. Purpose: The goal of this study is to evaluate methods for automatically assessing open-ended responses, e.g. students’ written explanations and reasoning for multiple-choice selections. Design/Method: We incorporated an open response component for an online signals and systems multiple-choice test to capture written explanations of students’ selections. The effectiveness of an automated approach for identifying and assessing student conceptual understanding was evaluated by comparing results of lexical analysis software packages (Leximancer and NVivo) to expert human analysis of student responses. In order to understand and delineate the process for effectively analysing text provided by students, the researchers evaluated strengths and weakness for both the human and automated approaches. Results: Human and automated analyses revealed both correct and incorrect associations for certain conceptual areas. For some questions, that were not anticipated or included in the distractor selections, showing how multiple-choice questions alone fail to capture the comprehensive picture of student understanding. The comparison of textual analysis methods revealed the capability of automated lexical analysis software to assist in the identification of concepts and their relationships for large textual data sets. We also identified several challenges to using automated analysis as well as the manual and computer-assisted analysis. Conclusions: This study highlighted the usefulness incorporating and analysing students’ reasoning or explanations in understanding how students think about certain conceptual ideas. The ultimate value of automating the evaluation of written explanations is that it can be applied more frequently and at various stages of instruction to formatively evaluate conceptual understanding and engage students in reflective

University life and Australian homes : three case studies of international Muslim students in Brisbane

Despite a significant increase in enrolments of postgraduate international Muslim students within Australian universities, little is known about their perceptions of life within Australian homes while undertaking their studies. The aim of this study is to investigate the ways in which students’ cultural and religious traditions affect their use of domestic spaces within the homes in which they reside. The research found that participants faced some minor difficulties in achieving privacy, maintaining modesty and extending hospitality while able to perform their daily activities in Australian designed homes. The findings suggest that greater research attention needs to be given to the development of Australian home designs that are adaptable to the needs of a multicultural society. Australian society encompasses diverse cultural customs and requirements with respect to home design, and these are yet to be explored.

The facilitator files : supporting the supporters of nursing students

Nurse education in Australia has undergone a substantial transformation in the course of its professional history. The use of experiential teaching and learning approaches in both pre and post registration courses has escalated dramatically and resulted in the extensive use of hospital based clinical experiences for student learning. The role of the clinical facilitator is used to provide stimulation to the affective and cognitive domains of student professional development and additionally is often used as a vehicle for student assessment. While most tertiary nursing schools utilise hospital clinical experiences widely, few studies have examined whether clinical facilitators have the necessary tools to effectively support and assess students in the clinical environment. Although employed for their clinical expertise and understanding of specific clinical contexts, facilitators are expected to have an understanding of experiential learning, how it informs the undergraduate curriculum and be able to support the development of neophyte nurses as they learn to apply theoretical concepts to the increasingly complex realm of patient care. Additionally, clinical facilitators are often employed on a casual basis and as a consequence of the casualisation of the facilitator workforce, universities are continually recruiting new clinical facilitators to these positions. This paper will explore a trial of facilitator education offered to clinical facilitators at the Queensland University of Technology, School of Nursing campus in Brisbane, Australia. It will report on the use of a series of student/facilitator “vignettes” created specifically for the purpose of facilitator education and how these interactions have been embedded within a wider educational package to support facilitators in their development as facilitators rather than teachers of nursing education. Finally, it will discuss the implications of facilitator preparedness on their ability to support and assess nursing students as they engage in the important transition from neophyte to registered nurse.

Do e-learning applications click with students globally? A study of the development and adoption of WriteCite.com

This paper traces the process of invention of adoption of WriteCite.com which is an internet based learning tool used by students to format citation lists. A number of theoretical issues relating to development and deployment of online learning tools are addressed in this paper. A significant finding in this paper is that adoption rate of new technologies amongst students, in this case for WriteCite.com, is not immediately global in nature despite the internet's ubiquity and the existence of internationally accepted bibliographic styles. Rather, and in contrast to neo theories of technology internationalisation, early-stage adoption tends to be geo-centric and concentrated in the US. In the case of WriteCite.com, which has not been sponsored by an academic institution, the findings reflect voluntary rates of globa adoption. From an applied perspective, the process of identifying globally contestable markets is relevant to developers of online learning tools.

A multiple case study of NAPLAN numeracy testing of Year 9 students in three Queensland secondary schools

This thesis reports on a multiple case study of the actions of three Queensland secondary schools in the context of Year 9 NAPLAN numeracy testing, focusing on their administrative practices, curriculum, pedagogy and assessment. It was established that schools have found it both challenging and costly to operate in an environment of educational reform generally, and NAPLAN testing in particular. The lack of a common understanding of numeracy and the substantial demands of implementing the Australian Curriculum have impacted on schools' ability to prepare students appropriately for NAPLAN numeracy tests. It was concluded that there is scope for schools to improve their approaches to NAPLAN numeracy testing in a way that maximises learning as well as test outcomes.

Facebook for adults ? A longitudinal study of MBA student’s use of an enterprise social network for collaborative learning

The rapid pace of social media means that our understanding of the way in which it facilitates the learning process continues to lag. The findings of a longitudinal study of an executive MBA cohort over the period of eight months in their use of the social media application is presented. Over time the ownership and use of the Yammer site shifted to become student driven and facilitated. The motivations behind the site’s use, perceived advantages and disadvantages and changes in usage patterns are documented. The case provides a useful insight into the way in which students used this technology to facilitate their learning goals and how patterns of behaviour changed in response to the changing needs of the cohort.

Developing young students’ meta-representational competence through integrated mathematics and science investigations

This paper describes students’ developing meta-representational competence, drawn from the second phase of a longitudinal study, Transforming Children’s Mathematical and Scientific Development. A group of 21 highly able Grade 1 students was engaged in mathematics/science investigations as part of a data modelling program. A pedagogical approach focused on students’ interpretation of categorical and continuous data was implemented through researcher-directed weekly sessions over a 2-year period. Fine-grained analysis of the developmental features and explanations of their graphs showed that explicit pedagogical attention to conceptual differences between categorical and continuous data was critical to development of inferential reasoning.

Development of fourth-grade students’ understanding of experimental and theoretical probability

Students explored variation and expectation in a probability activity at the end of the first year of a 3-year longitudinal study across grades 4-6. The activity involved experiments in tossing coins both manually and with simulation using the graphing software, TinkerPlots. Initial responses indicated that the students were aware of uncertainty, although an understanding of chance concepts appeared limited. Predicting outcomes of 10 tosses reflected an intuitive notion of equiprobability, with little awareness of variation. Understanding the relationship between experimental and theoretical probability did not emerge until multiple outcomes and representations were generated with the software.

Linking pedagogical knowledge practices and student outcomes in STEM education for primary schools

STEM education is a new frontier in Australia, particularly for primary schools. However, the E in STEM needs to have a stronger focus with science and mathematics concepts aligned to the presiding curricula. In addition, pedagogical knowledge practices such as planning, preparation, teaching strategies, assessment and so forth need to be connected to key concepts for developing a STEM education. One of the aims of this study was to understand how a pedagogical knowledge practice framework could be linked to student outcomes in STEM education. Specifically, this qualitative research investigated Year 4 students’ involvement in an integrated STEM education program that focused on science concepts (e.g., states of matter, testing properties of materials) and mathematics concepts (such as 3D shapes and metric measurements: millilitres, temperature, grams, centimetres) for designing, making and testing a strong and safe medical kit to insulate medicines at desirable temperatures. Eleven pedagogical knowledge practices (e.g., planning, preparation, teaching strategies, classroom management, and assessment) were used as a framework for understanding how teaching may be linked to student outcomes in STEM education. For instance, “planning” involved devising a student booklet as a resource for students to understand the tasks required of them, which also provided space for them to record ideas, results and information. Planning involved linking national and state curriculum documents to the STEM education activities. More studies are required around pedagogical knowledge frameworks to understand what students learn when involved in STEM education, particularly with the inclusion of engineering education.

Student perceptions: Improving the educational experiences of high school students on the autism spectrum

Listening to and reflecting on the voices and personal stories of adolescent students with an autism spectrum disorder (ASD) is critically important to developing more inclusive approaches to their education. This article considers the experiences of nine adolescents with an ASD on their inclusive education in a large urban secondary school in Australia. These educational experiences were mapped onto four themes emanating from a similar study by Humphrey and Lewis from the United Kingdom. The results from both studies suggest that although students with ASD are having positive and enabling educational experiences, a number of common inhibitors continue to prevent them from taking full advantage of their schooling. By listening to the voices of students with ASD, specific enablers and inhibitors to promoting successful educational experiences are identified, and recommendations for practice are put forward to better support the education not only of students with ASD but all students.

Increasing the confidence and knowledge of occupational therapy and physiotherapy students when communicating with people with aphasia: A pre–post intervention study

This study investigated occupational therapy and physiotherapy students' level of confidence and knowledge of strategies for communicating with people with aphasia (PWA) before and after a communication partner-training (CPT) program. Twenty-eight physiotherapy and occupational therapy students participated in a pre–post CPT program. Students completed a customized mixed-methods questionnaire before and after the intervention. The CPT program involved a lecture about effective communication strategies followed by a conversation with PWA to practice strategies learnt. Before CPT, students were not confident with the possibility of communicating with PWA. Students demonstrated rudimentary knowledge of supported conversation, identifying a maximum of five strategies for communicating effectively with PWA. Following intervention, students demonstrated increased confidence. Students' knowledge of effective communication strategies improved, with students identifying a maximum of 16 suitable strategies post-training. The results suggest that occupational therapy and physiotherapy students have potential to benefit from practical training in supported communication with PWA, which may assist them during placements in clinical settings with neurological patients or subsequent employment.

Afterschool MediaClub: Critical literacy in a high diversity, high poverty urban setting

In this chapter we present analyses of data produced with young people in an afterschool digital literacy program for 9 – 12 year olds. The young people were students at a high diversity, high poverty outer suburban elementary school in Queensland, Australia. The club was part of the URLearning research project (2010-14). In the classroom-based component of the project we worked with teachers to develop intellectually substantive and critical digital literacy practice. MediaClub was in some ways complementary to the classroom component; it was designed to skill up interested kids as digital media experts not only for their families and communities, but also for the classroom. Given the critical literacy traditions established in Australian schools, we approached MediaClub with certain critical expectations. In this chapter we look at what ensued, highlighting unanticipated critical outcomes at a time of heightened struggle over English curriculum. Critical literacy has been part of official English curriculum in Queensland since the early 1990s. The approach has been primarily text analytic, concerned with giving students access to genres of power and tools for understanding the ideological work of language through text. Many ideas for translating this normative critical project into classroom practice have been developed for use from the earliest elementary grades onwards. However, curricular space for critical literacy is under pressure. Amongst other things, this reflects both the development of Australia’s first national curriculum and the construction of a regimen of national literacy testing. At MediaClub we found a certain resistance to learning activities which were “too much like school”. However, in a context of increased control of teachers’ and students’ work in the classroom, MediaClub evolved as a learning space that can be understood in critical terms. Our experience in this regard might be of interest to teachers and researchers in high diversity high poverty settings that are strongly controlled through increasingly prescriptive – even scripted – pedagogies.