Assessment of simulated clinical skills and distance students: can we do it better?

Australian universities have traditionally been able to supplement clinical education, for undergraduate nursing courses, delivered on placement with weekly clinical teaching in the simulated environment. The Objective Structured Clinical Assessment (OSCA) tool has been used in this simulated environment to assess clinical skills. Recently, however, online delivery of undergraduate nursing courses has become more common. The move from an internal mode of teaching to an online external mode is seen worldwide and poses challenges to staff and students as well as changing the teaching and learning culture of institutions (Philip and Wozniak, 2009). This cultural shift and the resulting diminishing timeframe for students to acquire and practice simulated clinical skills imply that it may become necessary to rethink assessment forms such as the OSCA assessment. This study examines whether or not the OSCA tool developed by Bujack et al. (1991a) is the best tool to be used in this new context, where online teaching is supplemented by very short, annual, intensive periods of study. Skills acquisition theories dictate that time is required to produce an ideal skills acquisition environment (Quinn, 2000) but the time constraints placed on students in such intensive periods of study could influence skills acquisition. This cross-sectional qualitative study used semi-structured interviews and focus groups to collect data. 65% of the nursing faculty participated in the study. The teaching of the Bachelor of Nursing (BN) occurred on two campuses and staff from both areas participated. This group of nurse academics was employed across the range of academic levels (from lecturer to professor) at the University. Data analysis followed a generic thematic analysis framework. Findings in this study show that there are a variety of attitudes and underpinning beliefs amongst staff in relation to the OSCAs. Doubts were raised in regard to the suitability of the use of the OSCA tool in this setting. It also became apparent during this study that the OSCA tool possibly serves purposes other than an assessment tool.

Nurse academics perceptions of the efficacy of the OSCA tool

The use of Objective Structured Clinical Examination/Objective Structured Clinical Assessment (OSCE/OSCA) has been well documented. How assessors currently view the process, and if the OSCA tool still fulfils the assessment requirements, is unclear. In this study, the beliefs and expectations of assessors towards the assessment tool used in an undergraduate nursing degree to assess clinical skills was investigated. A cross-sectional study used semi-structured interviews and focus groups with 16 lecturers in nursing from a rural Australian university. This represents 65% of the total nurse academic staff employed there. The key issues that the academic staff raised reflect those from previous studies, such as the use of OSCA as formative assessment or a quality check process before the clinical practice. The OSCAs were seen as a good assessment tool, which gave students the opportunity to receive feedback on their performance in relation to clinical skills. The drawbacks identified in relation to the use of OSCAs were that the OSCA was seen as stressful to students. This drawback was thought to be further compounded if there was a lack of congruence regarding essential criteria between assessors. If not adequately addressed these drawbacks will erode the potential the OSCA tool has to foster uniformity, which was one of the main reasons for its implementation. .

A trial of e-simulation of sudden patient deterioration (FIRST2ACT WEB) on student learning

BACKGROUND: High-fidelity simulation pedagogy is of increasing importance in health professional education; however, face-to-face simulation programs are resource intensive and impractical to implement across large numbers of students. OBJECTIVES: To investigate undergraduate nursing students' theoretical and applied learning in response to the e-simulation program-FIRST2ACT WEBTM, and explore predictors of virtual clinical performance. DESIGN AND SETTING: Multi-center trial of FIRST2ACT WEBTM accessible to students in five Australian universities and colleges, across 8 campuses. PARTICIPANTS: A population of 489 final-year nursing students in programs of study leading to license to practice. METHODS: Participants proceeded through three phases: (i) pre-simulation-briefing and assessment of clinical knowledge and experience; (ii) e-simulation-three interactive e-simulation clinical scenarios which included video recordings of patients with deteriorating conditions, interactive clinical tasks, pop up responses to tasks, and timed performance; and (iii) post-simulation feedback and evaluation. Descriptive statistics were followed by bivariate analysis to detect any associations, which were further tested using standard regression analysis. RESULTS: Of 409 students who commenced the program (83% response rate), 367 undergraduate nursing students completed the web-based program in its entirety, yielding a completion rate of 89.7%; 38.1% of students achieved passing clinical performance across three scenarios, and the proportion achieving passing clinical knowledge increased from 78.15% pre-simulation to 91.6% post-simulation. Knowledge was the main independent predictor of clinical performance in responding to a virtual deteriorating patient R(2)=0.090, F(7, 352)=4.962, p<0.001. DISCUSSION: The use of web-based technology allows simulation activities to be accessible to a large number of participants and completion rates indicate that 'Net Generation' nursing students were highly engaged with this mode of learning. CONCLUSION: The web-based e-simulation program FIRST2ACTTM effectively enhanced knowledge, virtual clinical performance, and self-assessed knowledge, skills, confidence, and competence in final-year nursing students.

Educational stress among Chinese adolescents : measurement, risk factors and associations with mental health

Academic pressure among adolescents is a major risk factor for poor mental health and suicide and other harmful behaviours. While this is a worldwide phenomenon, it appears to be especially pronounced in China and other East Asian countries. Despite a growing body of research into adolescent mental health in recent years, the multiple constructs within the ‘educational stress’ phenomenon have not been clearly articulated in Chinese contexts. Further, the individual, family, school and peer influencing factors for educational stress and its associations with adolescent mental health are not well understood. An in-depth investigation may provide important information for the ongoing educational reform in Mainland China with a special focus on students’ mental health and wellbeing. The primary goal of this study was to examine the relative contribution of educational stress to poor mental health, in comparison to other well-known individual, family, school and peer factors. Another important task was to identify significant risk factors for educational stress. In addition, due to the lack of a culturally suitable instrument for educational stress in this population, a new tool – the Educational Stress Scale for Adolescents (ESSA) was initially developed in this study and tested for reliability and validity. A self-administered questionnaire was used to collect information from convenient samples of secondary school students in Shandong, China. The pilot survey was conducted with 347 students (grades 8 and 11) to test the psychometric properties of the ESSA and other scales or questions in the questionnaire. Based on factor analysis and reliability and validity testing, the 16-item scale (the ESSA) with five factors showed adequate to good internal consistency, 2-week test-retest reliability, and satisfactory concurrent and predictive validity. Its factor structure was further demonstrated in the main survey with a confirmatory factor analysis illustrating a good fit of the proposed model based on a confirmatory factor analysis. The reliabilities of other scales and questions were also adequate to be used in this study. The main survey was subsequently conducted with a sample of 1627 secondary school (grades 7-12) students to examine the influencing factors of educational stress and its associations with mental health outcomes, including depression, happiness and suicidal behaviours. A wide range of individual, family, school and peer factors were found to have a significant association with the total ESSA and subscale scores. Most of the strong factors for academic stress were school or study-related, including rural school location, low school connectedness, perceived poor academic grades and frequent emotional conflicts with teachers and peers. Unexpectedly, family and parental factors, such as parental bonding, family connectedness and conflicts with parents were found to have little or no association with educational stress. Educational stress was the most predictive variable for depression, but was not strongly associated with happiness. It had a strong association with suicide ideation but not with suicide attempts. Among five subscales of the ESSA, ‘Study despondency’ score had the strongest associations with these mental health measures. Surprising, two subscales, ‘Self-expectation’ and ‘Worry about grades’ showed a protective effect on suicidal behaviours. An additional analysis revealed that although academic pressure was the most commonly reported reason for suicidal thinking, the occurrence of problems in peer relationships such as peer teasing and bullying, and romantic problems had a much stronger relationship with actual attempts. This study provides some insights into the nature and health implications of educational stress among Chinese adolescents. Findings in this study suggest that interventions on educational stress should focus on school environment and academic factors. Intervention programs focused on educational stress may have a high impact on the prevalence of common mental disorders such as depression. Efforts to increase perceived happiness however should cover a wider range of individual, family and school factors. The importance of healthy peer relationships should be adequately emphasised in suicide prevention. In addition, the newly developed scale (the ESSA) demonstrates sound psychometric properties and is expected to be used in future research into academic-related stress among secondary school adolescents.

Enhancing students' self-concepts and related constructs: A critical longitudinal analysis capitalising on and combining promising enhancement techniques for educational settings

Enhancing children's self-concepts is widely accepted as a critical educational outcome of schooling and is postulated as a mediating variable that facilitates the attainment of other desired outcomes such as improved academic achievement. Despite considerable advances in self-concept research, there has been limited progress in devising teacher-administered enhancement interventions. This is unfortunate as teachers are crucial change agents during important developmental periods when self-concept is formed. The primary aim of the present investigation is to build on the promising features of previous self-concept enhancement studies by: (a) combining two exciting research directions developed by Burnett and Craven to develop a potentially powerful cognitive-based intervention; (b) incorporating recent developments in theory and measurement to ensure that the multidimensionality of self-concept is accounted for in the research design; (c) fully investigating the effects of a potentially strong cognitive intervention on reading, mathematics, school and learning self-concepts by using a large sample size and a sophisticated research design; (d) evaluating the effects of the intervention on affective and cognitive subcomponents of reading, mathematics, school and learning self-concepts over time to test for differential effects of the intervention; (e) modifying and extending current procedures to maximise the successful implementation of a teacher-mediated intervention in a naturalistic setting by incorporating sophisticated teacher training as suggested by Hattie (1992) and including an assessment of the efficacy of implementation; and (f) examining the durability of effects associated with the intervention.

An examination of the relationship between emotional intelligence, leadership style and perceived leadership outcomes in Australian educational institutions

In the field of leadership studies transformational leadership theory (e.g., Bass, 1985; Avolio, Bass, & Jung, 1995) has received much attention from researchers in recent years (Hughes, Ginnet, & Curphy, 2009; Hunt, 1999). Many previous studies have found that transformational leadership is related to positive outcomes such as the satisfaction, motivation and performance of followers in organisations (Judge & Piccolo, 2004; Lowe, Kroeck, & Sivasubramaniam, 1996), including in educational institutions (Chin, 2007; Leithwoood & Jantzi, 2005). Hence, it is important to explore constructs that may predict leadership style in order to identify potential transformational leaders in leadership assessment and selection procedures. Several researchers have proposed that emotional intelligence (EI) is one construct that may account for hitherto unexplained variance in transformational leadership (Mayer, 2001; Watkin, 2000). Different models of EI exist (e.g., Goleman, 1995, 2001; Bar-On, 1997; Mayer & Salovey, 1997) but momentum is growing for the Mayer and Salovey (1997) model to be considered the most useful (Ashkanasy & Daus, 2005; Daus & Ashkanasy, 2005). Studies in non-educational settings claim to have found that EI is a useful predictor of leadership style and leader effectiveness (Harms & Crede, 2010; Mills, 2009) but there is a paucity of studies which have examined the Mayer and Salovey (1997) model of EI in educational settings. Furthermore, other predictor variables have rarely been controlled in previous studies and only self-ratings of leadership behaviours, rather than multiple ratings, have usually been obtained. Therefore, more research is required in educational settings to answer the question: to what extent is the Mayer and Salovey (1997) model of EI a useful predictor of leadership style and leadership outcomes? This project, set in Australian educational institutions, was designed to move research in the field forward by: using valid and reliable instruments, controlling for other predictors, obtaining an adequately sized sample of real leaders as participants and obtaining multiple ratings of leadership behaviours. Other variables commonly used to predict leadership behaviours (personality factors and general mental ability) were assessed and controlled in the project. Additionally, integrity was included as another potential predictor of leadership behaviours as it has previously been found to be related to transformational leadership (Parry & Proctor-Thomson, 2002). Multiple ratings of leadership behaviours were obtained from each leader and their supervisors, peers and followers. The following valid and reliable psychological tests were used to operationalise the variables of interest: leadership styles and perceived leadership outcomes (Multifactor Leadership Questionnaire, Avolio et al., 1995), EI (Mayer–Salovey–Caruso Emotional Intelligence Test, Mayer, Salovey, & Caruso, 2002), personality factors (The Big Five Inventory, John, Donahue, & Kentle, 1991), general mental ability (Wonderlic Personnel Test-Quicktest, Wonderlic, 2003) and integrity (Integrity Express, Vangent, 2002). A Pilot Study (N = 25 leaders and 75 raters) made a preliminary examination of the relationship between the variables included in the project. Total EI, the experiential area, and the managing emotions and perceiving emotions branches of EI, were found to be related to transformational leadership which indicated that further research was warranted. In the Main Study, 144 leaders and 432 raters were recruited as participants to assess the discriminant validity of the instruments and examine the usefulness of EI as a predictor of leadership style and perceived leadership outcomes. Scores for each leadership scale across the four rating levels (leaders, supervisors, peers and followers) were aggregated with the exception of the management-by-exception active scale of transactional leadership which had an inadequate level of interrater agreement. In the descriptive and measurement component of the Main Study, the instruments were found to demonstrate adequate discriminant validity. The impact of role and gender on leadership style and EI were also examined, and females were found to be more transformational as leaders than males. Females also engaged in more contingent reward (transactional leadership) behaviours than males, whilst males engaged in more passive/avoidant leadership behaviours than females. In the inferential component of the Main Study, multiple regression procedures were used to examine the usefulness of EI as a predictor of leadership style and perceived leadership outcomes. None of the EI branches were found to be related to transformational leadership or the perceived leadership outcomes variables included in the study. Openness, emotional stability (the inverse of neuroticism) and general mental ability (inversely) each predicted a small amount of variance in transformational leadership. Passive/avoidant leadership was inversely predicted by the understanding emotions branch of EI. Overall, EI was not found to be a useful predictor of leadership style and leadership outcomes in the Main Study of this project. Implications for researchers and human resource practitioners are discussed.

Triangular Linguistic Fuzzy based performance measurement model for supply chain to assess leanness attributes

Increasing global competitiveness worldwide has forced manufacturing organizations to produce high-quality products more quickly and at a competitive cost. In order to reach these goals, they need good quality components from suppliers at optimum price and lead time. This actually forced all the companies to adapt different improvement practices such as lean manufacturing, Just in Time (JIT) and effective supply chain management. Applying new improvement techniques and tools cause higher establishment costs and more Information Delay (ID). On the contrary, these new techniques may reduce the risk of stock outs and affect supply chain flexibility to give a better overall performance. But industry people are unable to measure the overall affects of those improvement techniques with a standard evaluation model .So an effective overall supply chain performance evaluation model is essential for suppliers as well as manufacturers to assess their companies under different supply chain strategies. However, literature on lean supply chain performance evaluation is comparatively limited. Moreover, most of the models assumed random values for performance variables. The purpose of this paper is to propose an effective supply chain performance evaluation model using triangular linguistic fuzzy numbers and to recommend optimum ranges for performance variables for lean implementation. The model initially considers all the supply chain performance criteria (input, output and flexibility), converts the values to triangular linguistic fuzzy numbers and evaluates overall supply chain performance under different situations. Results show that with the proposed performance measurement model, improvement area for each variable can be accurately identified.

Understanding teachers' reporting of child sexual abuse : measurement methods matter

The present study considered factors influencing teachers' reporting of child sexual abuse (CSA). Conducted in three Australian jurisdictions with different reporting laws and policies, the study focused on teachers' actual past and anticipated future reporting of CSA. A sample of 470 teachers within randomly selected rural and urban schools was surveyed, to identify training and experience; knowledge of reporting legislation and policy; attitudes; and reporting practices. Factors influencing actual past reporting and anticipated future reporting were identified using logistic regression modelling. This is the first study to simultaneously examine the effect of important influences in reporting practice using both retrospective and prospective approaches across jurisdictions with different reporting laws. Teachers who have actually reported CSA in the past are more likely have higher levels of policy knowledge, and hold more positive attitudes towards reporting CSA along three specific dimensions: commitment to the reporting role; confidence in the system's effective response to their reporting; and they are more likely to be able to override their concerns about the consequences of their reporting. Teachers indicating intention to report hypothetical scenarios are more likely to hold reasonable grounds for suspecting CSA, to recognise that significant harm has been caused to the child, to know that their school policy requires a report, and to be able to override their concerns about the consequences of their reporting.

Paving the way for the use of the SDQ in economic evaluations of school-based population health interventions: an empirical analysis of the external validity of SDQ mapping algorithms to the CHU9D in an educational setting

Purpose
The Strengths and Difficulties Questionnaire (SDQ) is a behavioural screening tool for children. The SDQ is increasingly used as the primary outcome measure in population health interventions involving children, but it is not preference based; therefore, its role in allocative economic evaluation is limited. The Child Health Utility 9D (CHU9D) is a generic preference-based health-related quality of-life measure. This study investigates the applicability of the SDQ outcome measure for use in economic evaluations and examines its relationship with the CHU9D by testing previously published mapping algorithms. The aim of the paper is to explore the feasibility of using the SDQ within economic evaluations of school-based population health interventions.
Methods
Data were available from children participating in a cluster randomised controlled trial of the school-based roots of empathy programme in Northern Ireland. Utility was calculated using the original and alternative CHU9D tariffs along with two SDQ mapping algorithms. t tests were performed for pairwise differences in utility values from the preference-based tariffs and mapping algorithms.
Results
Mean (standard deviation) SDQ total difficulties and prosocial scores were 12 (3.2) and 8.3 (2.1). Utility values obtained from the original tariff, alternative tariff, and mapping algorithms using five and three SDQ subscales were 0.84 (0.11), 0.80 (0.13), 0.84 (0.05), and 0.83 (0.04), respectively. Each method for calculating utility produced statistically significantly different values except the original tariff and five SDQ subscale algorithm.
Conclusion
Initial evidence suggests the SDQ and CHU9D are related in some of their measurement properties. The mapping algorithm using five SDQ subscales was found to be optimal in predicting mean child health utility. Future research valuing changes in the SDQ scores would contribute to this research.

A multidimensional measurement approach and analysis of children's motivation for reading, attributional style, and reading achievement

The present study investigates the usefulness of a multi-method approach to the measurement of reading motivation and achievement. A sample of 127 elementary and middle-school children aged 10 to 14 responded to measures of motivation, attributions, and achievement both longitudinally and in a challenging reading context. Novel measures of motivation and attributions were constructed, validated, and utilized to examine the relationship between ~ motivation, attributions, and achievement over a one-year period (Study I). The impact of classroom contexts and instructional practices was also explored through a study of the influence of topic interest and challenge on motivation, attributions, and persistence (Study II), as well as through interviews with children regarding motivation and reading in the classroom (Study III). Creation and validation of novel measures of motivation and attributions supported the use of a self-report measure of motivation in situation-specific contexts, and confirmed a three-factor structure of attributions for reading performance in both hypothetical and situation-specific contexts. A one-year follow up study of children's motivation and reading achievement demonstrated declines in all components of motivation beginning at age 10 through 12, and particularly strong decreases in motivation with the transition to middle school. Past perceived competence for reading predicted current achievement after controlling for past achievement, and showed the strongest relationships with reading-related skills in both elementary and middle school. Motivation and attributions were strongly related, and children with higher motivation Fulmer III displayed more adaptive attributions for reading success and failure. In the context of a developmentally inappropriate challenging reading task, children's motivation for reading, especially in terms of perceived competence, was threatened. However, interest in the story buffered some ofthe negative impacts of challenge, sustaining children's motivation, adaptive attributions, and reading persistence. Finally, children's responses during interviews outlined several emotions, perceptions, and aspects of reading tasks and contexts that influence reading motivation and achievement. Findings revealed that children with comparable motivation and achievement profiles respond in a similar way to particular reading situations, such as excessive challenge, but also that motivation is dynamic and individualistic and can change over time and across contexts. Overall, the present study outlines the importance of motivation and adaptive attributions for reading success, and the necessity of integrating various methodologies to study the dynamic construct of achievement motivation.

Equality of educational opportunity employing Pisa data: taking both achievement and access into account

La medición de la desigualdad de oportunidades con las bases de PISA implican varias limitaciones: (i) la muestra sólo representa una fracción limitada de las cohortes de jóvenes de 15 años en los países en desarrollo y (ii) estas fracciones no son uniformes entre países ni entre periodos. Lo anterior genera dudas sobre la confiabilidad de estas mediciones cuando se usan para comparaciones internacionales: mayor equidad puede ser resultado de una muestra más restringida y más homogénea. A diferencia de enfoques previos basados en reconstrucción de las muestras, el enfoque del documento consiste en proveer un índice bidimensional que incluye logro y acceso como dimensiones del índice. Se utilizan varios métodos de agregación y se observan cambios considerables en los rankings de (in) equidad de oportunidades cuando solo se observa el logro y cuando se observan ambas dimensiones en las pruebas de PISA 2006/2009. Finalmente se propone una generalización del enfoque permitiendo otras dimensiones adicionales y otros pesos utilizados en la agregación.

On the measurement of achievement goals: critique, illustration, and application

The authors identified several specific problems with the measurement of achievement goals in the current literature and illustrated these problems, focusing primarily on A. J. Elliot and H. A. McGregor's (2001) Achievement Goal Questionnaire (AGQ). They attended to these problems by creating the AGQ-Revised and conducting a study that examined the measure's structural validity and predictive utility with 229 (76 male, 150 female, 3 unspecified) undergraduates. The hypothesized factor and dimensional structures of the measure were confirmed and shown to be superior to a host of alternatives. The predictions were nearly uniformly supported with regard to both the antecedents (need for achievement and fear of failure) and consequences (intrinsic motivation and exam performance) of the 4 achievement goals. In discussing their work, the authors highlight the importance and value of additional precision in the area of achievement goal measurement. (PsycINFO Database Record (c) 2012 APA, all rights reserved)(journal abstract)

An evaluation of the effect of an educational intervention for Australian social workers on competence in delivering brief cognitive behavioural strategies : a randomised controlled trial

Background Broad community access to high quality evidence-based primary mental health care is an ongoing challenge around the world. In Australia one approach has been to broaden access to care by funding psychologists and other allied health care professionals to deliver brief psychological treatments to general practitioners' patients. To date, there has been a scarcity of studies assessing the efficacy of social worker delivered psychological strategies. This study aims to build the evidence base by evaluating the impact of a brief educational intervention on social workers' competence in delivering cognitive behavioural strategies (strategies derived from cognitive behavioural therapy). Methods A randomised controlled trial design was undertaken with baseline and one-week follow-up measurement of both objective and self-perceived competence. Simulated consultations with standardised depressed patients were recorded on videotape and objective competence was assessed by blinded reviewers using the Cognitive Therapy Scale. Questionnaires completed by participants were used to measure self-perceived competence. The training intervention was a 15 hour face-to-face course involving presentations, video example consultations, written materials and rehearsal of skills in pairs. Results 40 Melbourne-based (Australia) social workers enrolled and were randomised and 9 of these withdrew from the study before the pre training simulated consultation. 30 of the remaining 31 social workers (97%) completed all phases of the intervention and evaluation protocol (16 from intervention and 14 from control group). The intervention group showed significantly greater improvements than the control group in objective competence (mean improvement of 14.2 (7.38-21.02) on the 66 point Cognitive Therapy Scale) and in subjective confidence (mean improvement of 1.28 (0.84-1.72) on a 5 point Likert scale). On average, the intervention group improved from below to above the base competency threshold on the Cognitive Therapy Scale whilst the control group remained below. Conclusions Social workers can attain significant improvements in competency in delivering cognitive behavioural strategies from undertaking brief face to face training. This is relevant in the context of health reforms that involve social worker delivery of evidence based psychological care. Further research is required to assess how these improvements in competence translate into performance in practice and clinical outcomes for patients.