Using automated text analysis to evaluate students’ conceptual understanding

Background: A major challenge for assessing students’ conceptual understanding of STEM subjects is the capacity of assessment tools to reliably and robustly evaluate student thinking and reasoning. Multiple-choice tests are typically used to assess student learning and are designed to include distractors that can indicate students’ incomplete understanding of a topic or concept based on which distractor the student selects. However, these tests fail to provide the critical information uncovering the how and why of students’ reasoning for their multiple-choice selections. Open-ended or structured response questions are one method for capturing higher level thinking, but are often costly in terms of time and attention to properly assess student responses. Purpose: The goal of this study is to evaluate methods for automatically assessing open-ended responses, e.g. students’ written explanations and reasoning for multiple-choice selections. Design/Method: We incorporated an open response component for an online signals and systems multiple-choice test to capture written explanations of students’ selections. The effectiveness of an automated approach for identifying and assessing student conceptual understanding was evaluated by comparing results of lexical analysis software packages (Leximancer and NVivo) to expert human analysis of student responses. In order to understand and delineate the process for effectively analysing text provided by students, the researchers evaluated strengths and weakness for both the human and automated approaches. Results: Human and automated analyses revealed both correct and incorrect associations for certain conceptual areas. For some questions, that were not anticipated or included in the distractor selections, showing how multiple-choice questions alone fail to capture the comprehensive picture of student understanding. The comparison of textual analysis methods revealed the capability of automated lexical analysis software to assist in the identification of concepts and their relationships for large textual data sets. We also identified several challenges to using automated analysis as well as the manual and computer-assisted analysis. Conclusions: This study highlighted the usefulness incorporating and analysing students’ reasoning or explanations in understanding how students think about certain conceptual ideas. The ultimate value of automating the evaluation of written explanations is that it can be applied more frequently and at various stages of instruction to formatively evaluate conceptual understanding and engage students in reflective

A qualitative study of mental health practices with Culturally and Linguistically Diverse (CALD) clients

Practitioners are expected to be culturally competent in order to offer appropriate counselling to culturally and linguistically diverse (CALD) clients in Australia. However, not much is known about their practices and therapeutic experiences with CALD clients. Using a qualitative approach, 15 mental health professionals from different disciplines were interviewed about the counselling process as well as their practice with CALD clients. Participants who had worked with CALD clients in a range of settings were invited to participate. Themes identified by the thematic analysis highlighted the importance of awareness and skills over knowledge, suggesting that respect and rapport could compensate for a lack of cultural knowledge. Further, practitioners reported that knowing where to find cultural information as it was required was more important than being expected to know this cultural knowledge. The findings supported concepts from existing models of multicultural competence and identified concrete practices associated with these constructs, thereby narrowing the gap between the theory and practice of cultural competence.

Teaching and assessment of prescribing medicines : how does pharmacy perform?

BACKGROUND Prescribing is a complex task, requiring specific knowledge and skills combined with effective, context-specific clinical reasoning. Prescribing errors can result in significant morbidity and mortality. For all professions with prescribing rights, a clear need exists to ensure students graduate with a well-defined set of prescribing skills, which will contribute to competent prescribing. AIM To describe the methods employed to teach and assess the principles of effective prescribing across five non-medical professions at Queensland University of Technology. METHOD The NPS National Prescribing Competencies Framework (PCF) was used as the prescribing standard. A curriculum mapping exercise was undertaken to determine how well the PCF was addressed across the disciplines of paramedic science, pharmacy, podiatry, nurse practitioner and optometry. Identified gaps in teaching and/or assessment were noted. RESULTS Prescribing skills and knowledge are taught and assessed using a range of methods across disciplines. A multi-modal approach is employed by all disciplines. The Pharmacy discipline uses more tutorial sessions to teach prescribing principles and relies less on case studies and clinical appraisal to assess prescribing when compared to other disciplines. Within the pharmacy discipline approximately 90% of the PCF competencies are taught and assessed. This compares favourably with the other disciplines. CONCLUSION Further work is required to establish a practical, effective approach to the assessment of prescribing competence especially between the university and clinical settings. Effective and reliable assessment of prescribing undertaken by students in diverse settings remains challenging.

Individual differences in the processing of punishment and reward cues : an application to road safety messages

This thesis examined the extent to which individual differences, as conceptualised by the revised Reinforcement Sensitivity Theory, influenced young drivers' information processing and subsequent acceptance of anti-speeding messages. Using a multi-method approach, the findings highlighted the utility of combining objective measures (a cognitive response time task and electroencephalography) with self-report measures to assess message processing and message acceptance, respectively. This body of research indicated that responses to anti-speeding messages may differ depending on an individual's personality disposition. Overall, the research provided further insight into the development of message strategies to target high risk drivers.

Beyond competencies : using a capability framework in developing practice standards for advanced practice nursing

AIM This paper presents a discussion on the application of a capability framework for advanced practice nursing standards/competencies. BACKGROUND There is acceptance that competencies are useful and necessary for definition and education of practice-based professions. Competencies have been described as appropriate for practice in stable environments with familiar problems. Increasingly competencies are being designed for use in the health sector for advanced practice such as the nurse practitioner role. Nurse practitioners work in environments and roles that are dynamic and unpredictable necessitating attributes and skills to practice at advanced and extended levels in both familiar and unfamiliar clinical situations. Capability has been described as the combination of skills, knowledge, values and self-esteem which enables individuals to manage change, be flexible and move beyond competency. DESIGN A discussion paper exploring 'capability' as a framework for advanced nursing practice standards. DATA SOURCES Data were sourced from electronic databases as described in the background section. IMPLICATIONS FOR NURSING As advanced practice nursing becomes more established and formalized, novel ways of teaching and assessing the practice of experienced clinicians beyond competency are imperative for the changing context of health services. CONCLUSION Leading researchers into capability in health care state that traditional education and training in health disciplines concentrates mainly on developing competence. To ensure that healthcare delivery keeps pace with increasing demand and a continuously changing context there is a need to embrace capability as a framework for advanced practice and education.

Estimating second language productive vocabulary size: A Capture-Recapture approach

This study provides validity evidence for the Capture-Recapture (CR) method, borrowed from ecology, as a measure of second language (L2) productive vocabulary size (PVS). Two separate “captures” of productive vocabulary were taken using written word association tasks (WAT). At Time 1, 47 bilinguals provided at least 4 associates to each of 30 high-frequency stimulus words in English, their first language (L1), and in French, their L2. A few days later (Time 2), this procedure was repeated with a different set of stimulus words in each language. Since the WAT was used, both Lex30 and CR PVS scores were calculated in each language. Participants also completed an animacy judgment task assessing the speed and efficiency of lexical access. Results indicated that, in both languages, CR and Lex30 scores were significantly positively correlated (evidence of convergent validity). CR scores were also significantly larger in the L1, and correlated significantly with the speed of lexical access in the L2 (evidence of construct validity). These results point to the validity of the technique for estimating relative L2 PVS. However, CR scores are not a direct indication of absolute vocabulary size. A discussion of the method’s underlying assumptions and their implications for interpretation are provided.

Establishing statistical foundations early : data modeling with young learners

This paper addresses research from a three-year longitudinal study that engaged children in data modeling experiences from the beginning school year through to third year (6-8 years). A data modeling approach to statistical development differs in several ways from what is typically done in early classroom experiences with data. In particular, data modeling immerses children in problems that evolve from their own questions and reasoning, with core statistical foundations established early. These foundations include a focus on posing and refining statistical questions within and across contexts, structuring and representing data, making informal inferences, and developing conceptual, representational, and metarepresentational competence. Examples are presented of how young learners developed and sustained informal inferential reasoning and metarepresentational competence across the study to become “sophisticated statisticians”.

Childhood atopic dermatitis : a cross-sectional study of relationships between child and parent factors, atopic dermatitis management, and disease severity

Background: Successful management of atopic dermatitis poses a significant and ongoing challenge to parents of affected children. Despite frequent reports of child behaviour problems and parenting difficulties, there is a paucity of literature examining relationships between child behaviour and parents' confidence and competence with treatment. Objectives: To examine relationships between child, parent, and family variables, parents' self-efficacy for managing atopic dermatitis, self-reported performance of management tasks, observed competence with providing treatment, and atopic dermatitis severity. Design: Cross-sectional study design. Participants A sample of 64 parent-child dyads was recruited from the dermatology clinic of a paediatric tertiary referral hospital in Brisbane, Australia. Methods: Parents completed self-report questionnaires examining child behaviour, parents' adjustment, parenting conflict, parents' relationship satisfaction, and parents' self-efficacy and self-reported performance of key management tasks. Severity of atopic dermatitis was assessed using the Scoring Atopic Dermatitis index. A routine home treatment session was observed, and parents' competence in carrying out the child's treatment assessed. Results: Pearson's and Spearman's correlations identified significant relationships (p< .05) between parents' self-efficacy and disease severity, child behaviour difficulties, parent depression and stress, parenting conflict, and relationship satisfaction. There were also significant relationships between each of these variables and parents' self-reported performance of management tasks. More profound child behaviour difficulties were associated with more severe atopic dermatitis and greater parent stress. Using multiple linear regressions, significant proportions of variation in parents' self-efficacy and self-reported task performance were explained by child behaviour difficulties and parents' formal education. Self-efficacy emerged as a likely mediator for relationships between both child behaviour and parents' education, and self-reported task performance. Direct observation of treatment sessions revealed strong relationships between parents' treatment competence and parents' self-efficacy, outcome expectations, and self-reported task performance. Less competent task performance was also associated with greater parent-reported child behaviour difficulties, parent depression and stress, parenting conflict, and relationship dissatisfaction. Conclusion: This study revealed the importance of child behaviour to parents' confidence and practices in the context of atopic dermatitis management. Children with more severe atopic dermatitis are at risk of presenting with challenging behaviour problems and their parents struggle to manage the condition successfully.

Enterprise business technology governance: A validation of three technology-governance competencies for boards of directors

The competent leadership and governance of digital transformation needs to involve the board of directors. The reported lack of such capability in boards is becoming a pressing issue. Underpinning leadership in such transformation are the competencies to effectively govern Enterprise Technology (ETG). In this paper we take the position that ETG competencies are essential in boards because competent enterprise business technology governance has been shown to contribute to increased revenue, profit, and returns. We report the industry validation processes of a set of three board-of-director competencies needed for effective ETG related to strategy and planning; investment and risk; and, innovation and value creation. We conclude that gaps in board ETG competence remain.

Examining how projections about the future of society are related to present-day climate change action

Global climate change will affect all domains of person-environment relations. Tackling climate change will require social change that can be motivated by people’s imaginings of the future of their society where such social change has occurred. We use the “collective futures” framework to examine whether beliefs about the future of society are related to present-day intentions to take climate change action. Participants from two Brazilian samples imagined their society in 2050 where climate change was mitigated and then rated how this future society would differ from Brazilian society today in terms of societal-level dysfunction and development and personal-level traits and values. To the extent that participants believed preventing climate change would result in societal development and more competence traits, they were more willing to engage in environmental citizenship activities. Individual differences in future time perspective also impacted environmental citizenship intention. Societal development and consideration of future consequences seem to be distinct routes by which future thinking influence climate change action.

Collective futures: How projections about the future of society are related to actions and attitudes supporting social change

We identified the active ingredients in people’s visions of society’s future (“collective futures”) that could drive political behavior in the present. In eight studies (N = 595), people imagined society in 2050 where climate change was mitigated (Study 1), abortion laws relaxed (Study 2), marijuana legalized (Study 3), or the power of different religious groups had increased (Studies 4-8). Participants rated how this future society would differ from today in terms of societal-level dysfunction and development (e.g., crime, inequality, education, technology), people’s character (warmth, competence, morality), and their values (e.g., conservation, self-transcendence). These measures were related to present-day attitudes/intentions that would promote/prevent this future (e.g., act on climate change, vote for a Muslim politician). A projection about benevolence in society (i.e., warmth/morality of people’s character) was the only dimension consistently and uniquely associated with present-day attitudes and intentions across contexts. Implications for social change theories, political communication, and policy design are discussed.

Self-regulation support through music

A critical dimension of early learning competence in the year prior to school is self-regulation. Self-regulation enables children to manage their emotions and direct their attention, thinking, and actions to meet adaptive goals. These skills enhance young children's readiness to learn.

Strenthening information literacy through collaboration: Implications for AIM

Because of its size, its excellent VET history, and its emerging higher education provision, AIM is in a special position to be an exemplar of good practice in the VET-HE transition. Many dual sector providers, by virtue of their size, tend to focus on higher education, on the assumption that VET ‘competence’ implies that their VET entrants to HE are confident and capable in information literacy skills. While this is only one of the many challenges that such students face in their undergraduate programs, it is the most critical for most of them in their quest for academic success. All students (school leavers, gap-year participants, articulating, mature age) entering HE will face specific challenges. For articulating students, the nature of credit transfer arrangements will often mean they commence studies in units that are not designated first year units. In this case, the embedded support structures are not as prominent. The existing literature is not consistent in reports on the rates of completion, retention and attrition of articulating students. There is some evidence that VET-qualified students have higher retention rates than school leavers [1], but limited information literacy skills can lead to attrition [2].

Perceived safety, quality and cultural competency of maternity care for culturally and linguistically diverse women in Queensland

Various policies, plans, and initiatives have been implemented to provide safe, quality, and culturally competent care to patients within Queensland’s healthcare system. A series of models of maternity care are available in Queensland that range from standard public care to private midwifery care. The current study aimed to determine whether identifying as Culturally or Linguistically Diverse (CALD) was associated with the perceived safety, quality, and cultural competency of maternity care from a consumer perspective, and to identify specific needs and preferences of CALD maternity care consumers. Secondary analysis of data collected in the Having a Baby in Queensland Survey 2012 was used to compare the experiences of 655 CALD women to those of 4049 non-CALD women in Queensland, Australia, across three stages of maternity care: pregnancy, labour and birth, and after birth. After adjustment for model of maternity care received and socio-demographic characteristics, CALD women were significantly more likely than non-CALD women to experience suboptimal staff technical competence in pregnancy, overall perceived safety in pregnancy and labour/birth, and interpersonal sensitivity in pregnancy and labour/birth. Approximately 50% of CALD women did not have the choice to use a translator or interpreter, or the gender of their care provider, during labour and birth. Thirteen themes of preferences and needs of CALD maternity care consumers based on ethnicity, cultural beliefs, or traditions were identified, however, these were rarely met. Findings imply that CALD women in Queensland experience disadvantageous maternity care with regards to perceived staff technical competence, safety, and interpersonal sensitivity, and receive care that lacks cultural competence. Improved access to support persons, continuity and choice of carer, and staff availability and training is recommended.