Standardizing the assessment of clinical competence: an overview of intensive care course design.

Rationale for the development of the Certificate in Health Studies: Intensive Care and High Dependency for Adults course developed at Queens University Belfast, Northern Ireland. Structure and content of clinical module reviewed. Clinical assessment strategy discussed. Focus on the utilization of a standardized portfolio, individualized learning contract and objective structured clinical examination (OSCE) to evaluate clinical competence. Evaluation of OSCE as an assessment tool and of the course provision.

Abstract: Psychometric properties of two self-report clinical competence scales for nursing students.

Abstract: Psychometric properties of two self-report clinical competence scales for nursing students.
Background: It is important to assess the clinical competence of nursing students to gauge their professional development and educational needs. This can be measured by self-assessment tools. Anema and McCoy (2010) contended that the currently available measures need further psychometric testing.
Aim: To test the psychometric properties of Nursing Competencies Questionnaire (NCQ) and Self-Efficacy in Clinical Performance (SECP) clinical competence scales.

Method: A non-randomly selected sample of n=248 2nd year nursing students completed NCQ, SECP and demographic questionnaires (June and September 2013). Mokken Scaling Analysis (MSA) was used to test the structural validity and scale properties, convergent and discriminant validity and reliability were subsequently tested.

Results: The NCQ provided evidence of a unidimensional scale which had strong scale scalability coefficients Hs =0.581; but limited evidence of item rankability HT =0.367. MSA undertaken with the SECP scale identified two potential unidimensional scales the SECP28 and SECP7, each with adequate evidence of good/reasonable scalablity psychometric properties as a summed scale but no/very limited evidence of scale rankability (SECP28: Hs = 0.55, HT=0.211; SECP7: Hs = 0.61, HT=0.049). Analysis of between cohort differences and NCQ/ SECP scale scores produced evidence of convergent and discriminant validity and good internal reliability: NCQ α = 0.93, SECP28 α = 0.96, and SECP7 α=0.89.

Discussion: The NCQ was verified to have evidence of reliability and validity; however, as the SECP findings are new, and the sample small, with reference to Straat and colleagues (2014), the SECP results should be interpreted with caution and verified on a second sample.

Conclusions: Measurement of perceived self-competence could inform the development of nursing competence and could start early in a nursing programme. Further testing of the NCQ and SECP scales with larger samples and from different years is indicated.


References:
Anema, M., G and McCoy, JK. (2010) Competency-Based Nursing Education: Guide to Achieving Outstanding Learner Outcomes. New York: Springer.
Straat, JH., van der Ark, LA and Sijtsma, K. (2014) Minimum Sample Size Requirements for Mokken Scale Analysis Educational and Psychological Measurement 74 (5), 809-822.

Planning nursing teaching: Educational purposes and clinical competence

Thinking about nursing education implies articulating this issue with the expressions of theoretical frameworks, from the perspective of a pedagogical aspect that includes both constructivism and competencies. The objective was to characterize, from a longitudinal view, the construction of care competencies that exist in the teaching plans of nursing undergraduate programs. This exploratory-descriptive study used a qualitative approach. Documentary analysis was performed on the nine teaching plans of undergraduate care subjects. The ethical-legal aspects were guaranteed, so that data was collected only after the study had been approved by the Research Ethics Committee. The data evidenced a curriculum organization centered on subjects, maintaining internal rationales that seem to resist summative organizations. Signs emerge of hardly substantial links between any previous knowledge and the strengthening of critical judgment and clinical reasoning. As proposed, the study contributed with reconsiderations for the teachinglearning process and showed the influence of constructivism on the proposal of clinical competencies.

Is family medicine meeting the HIV/AIDS challenge? A national survey of family physicians' beliefs, clinical competence, and experience regarding HIV disease

A national sample of family physicians was surveyed to (1) assess family physicians' beliefs about the human immunodeficiency virus (HIV) and individuals at risk for infection, their clinical competence regarding HIV-related issues, and their experiences with HIV disease; (2) present conclusions to the American Academy of Family Physicians (AAFP) to effect the development of an early clinical care protocol and a continuing medical education curriculum; and (3) collect base-line data for use in the evaluation of an early clinical care protocol and a continuing medical education curriculum, in the case that such programs are developed and disseminated. After considering retired or deceased respondents, of the 2,660 physicians surveyed, 1,678 (63.7%) responded. The resulting sample was representative of the active members of the AAFP. About 77% of the respondents were unable to accurately identify the universal precautions for blood and body fluids to prevent occupational transmission of HIV or hepatitis B virus (HBV). Residency trained and board certified physicians expressed fewer "external constraints," such as fear of losing patients, obviating them from providing treatment to individuals with HIV disease (p =.004 and p $<$.001, respectively). These physicians also manifested fewer "internal constraints" to the provision of HIV treatment, such as fear of becoming infected (p $<$.001 and p =.012, respectively). Residency trained physicians also expressed a greater comfort with discussing sexually-related topics with their patients than did non-residency trained physicians (p $<$.001). There were 67.1% of the physicians surveyed who reported never providing treatment to an individual with HIV disease. Residency trained and board certified physicians expressed a greater likelihood to provide treatment to HIV-infected patients (p $<$.001) than non-residency trained and non-board certified physicians.^ Among the various primary care specialties, family medicine is especially vulnerable to the current challenges of HIV/AIDS. These challenges are augmented by the epidemiologic pattern that characterizes AIDS. For the past several years, we have seen AIDS in this country assume a similar pattern to that seen in most other countries; HIV is becoming increasingly prevalent in the heterosexual population as well as in locations removed from metropolitan centers. This current phase of the epidemic generates greater pressures upon primary care physicians, particularly family physicians, to become better acquainted with the means to provide early care to HIV/AIDS patients and to prevent HIV/AIDS among their patients. Family medicine is especially appropriate for providing care to HIV patients because family medicine involves treatment to all age groups and conditions; other primary care specialties focus on limited patient populations or specific conditions. Family physicians should be armed with the expertise to confront HIV/AIDS. However, family physicians' clinical competence and experience with HIV is not known. The data collected in this survey describes their competencies, attitudes, and experiences. ^

Subject -centered and problem -based approaches to teaching clinical competence in physical therapy

Physical therapy students must apply the relevant information learned in their academic and clinical experience to problem solve in treating patients. I compared the clinical cognitive competence in patient care of second-year masters students enrolled in two different curricular programs: modified problem-based (M P-B; n = 27) and subject-centered (S-C; n = 41). Main features of S-C learning include lecture and demonstration as the major teaching strategies and no exposure to patients or problem solving learning until the sciences (knowledge) have been taught. Comparatively, main features of M P-B learning include case study in small student groups as the main teaching strategy, early and frequent exposure to patients, and knowledge and problem solving skills learned together for each specific case. Basic and clinical orthopedic knowledge was measured with a written test with open-ended items. Problem solving skills were measured with a written case study patient problem test yielding three subscores: assessment, problem identification, and treatment planning. ^ Results indicated that among the demographic and educational characteristics analyzed, there was a significant difference between groups on ethnicity, bachelor degree type, admission GPA, and current GPA, but there was no significant difference on gender, age, possession of a physical therapy assistant license, and GRE score. In addition, the M P-B group achieved a significantly higher adjusted mean score on the orthopedic knowledge test after controlling for GRE scores. The S-C group achieved a significantly higher adjusted mean total score and treatment management subscore on the case study test after controlling for orthopedic knowledge test scores. These findings did not support their respective research hypotheses. There was no significant difference between groups on the assessment and problem identification subscores of the case study test. The integrated M P-B approach promoted superior retention of basic and clinical science knowledge. The results on problem solving skills were mixed. The S-C approach facilitated superior treatment planning skills, but equivalent patient assessment and problem identification skills by emphasizing all equally and exposing the students to more patients with a wider variety of orthopedic physical therapy needs than in the M P-B approach. ^

Testing the psychometric properties of two self-report clinical competence scales for nursing students.

Background: It is important to assess the clinical competence of nursing students to gauge their educational needs. Competence can be measured by self-assessment tools; however, Anema and McCoy (2010) contend that currently available measures should be further psychometrically tested.
Aim: To test the psychometric properties of Nursing Competencies Questionnaire (NCQ) and Self-Efficacy in Clinical Performance (SECP) clinical competence scales.
Method: A non-randomly selected sample of n=248 2nd year nursing students completed NCQ, SECP and demographic questionnaires (June and September 2013). Mokken Scaling Analysis (MSA) was used to investigate structural validity and scale properties; convergent and discriminant validity and reliability were also tested for each scale.
Results: MSA analysis identified that the NCQ is a unidimensional scale with strong scale scalability coefficients Hs =0.581; but limited item rankability HT =0.367. The SECP scale MSA suggested that the scale could be potentially split into two unidimensional scales (SECP28 and SECP7), each with good/reasonable scalablity psychometric properties as summed scales but negligible/very limited scale rankability (SECP28: Hs = 0.55, HT=0.211; SECP7: Hs = 0.61, HT=0.049). Analysis of between cohort differences and NCQ/SECP scores produced evidence of discriminant and convergent validity; good internal reliability was also found: NCQ α = 0.93, SECP28 α = 0.96 and SECP7 α=0.89.

Discussion: In line with previous research further evidence of the NCQ’s reliability and validity was demonstrated. However, as the SECP findings are new and the sample small with reference to Straat and colleagues (2014), the SECP results should be interpreted with caution and verified on a second sample.
Conclusions: Measurement of perceived self-competence could start early in a nursing programme to support students’ development of clinical competence. Further testing of the SECP scale with larger nursing student samples from different programme years is indicated.

References:
Anema, M., G and McCoy, JK. (2010) Competency-Based Nursing Education: Guide to Achieving Outstanding Learner Outcomes. New York: Springer.
Straat, JH., van der Ark, LA and Sijtsma, K. (2014) Minimum Sample Size Requirements for Mokken Scale Analysis Educational and Psychological Measurement 74 (5), 809-822.

Clinical reasoning: The relative contribution of identification, interpretation and hypothesis errors to misdiagnosis

The aim of this study was to identify and describe the types of errors in clinical reasoning that contribute to poor diagnostic performance at different levels of medical training and experience. Three cohorts of subjects, second- and fourth- (final) year medical students and a group of general practitioners, completed a set of clinical reasoning problems. The responses of those whose scores fell below the 25th centile were analysed to establish the stage of the clinical reasoning process - identification of relevant information, interpretation or hypothesis generation - at which most errors occurred and whether this was dependent on problem difficulty and level of medical experience. Results indicate that hypothesis errors decrease as expertise increases but that identification and interpretation errors increase. This may be due to inappropriate use of pattern recognition or to failure of the knowledge base. Furthermore, although hypothesis errors increased in line with problem difficulty, identification and interpretation errors decreased. A possible explanation is that as problem difficulty increases, subjects at all levels of expertise are less able to differentiate between relevant and irrelevant clinical features and so give equal consideration to all information contained within a case. It is concluded that the development of clinical reasoning in medical students throughout the course of their pre-clinical and clinical education may be enhanced by both an analysis of the clinical reasoning process and a specific focus on each of the stages at which errors commonly occur.

The association between student characteristics and the development of clinical reasoning in a graduate-entry, PBL medical programme

This study sought to assess the extent to which the entry characteristics of students in a graduate-entry medical programme predict the subsequent development of clinical reasoning ability. Subjects comprised 290 students voluntarily recruited from three successive cohorts of the University of Queensland's MBBS Programme. Clinical reasoning was measured once a year over a period of three years using two methods, a set of 10 Clinical Reasoning Problems (CRPs) and the Diagnostic Thinking Inventory (DTI). Data on gender, age at entry into the programme, nature of primary degree, scores on selection criteria (written examination plus interview) and academic performance in the first two years of the programme were recorded for each student, and their association with clinical reasoning skill analysed using univariate and multivariate analysis. Univariate analysis indicated significant associations between CRP score, gender and primary degree with a significant but small association between DTI and interview score. Stage of progression through the programme was also an important predictor of performance on both indicators. Subsequent multivariate analysis suggested that female gender is a positive predictor of CRP score independently of the nature of a subject's primary degree and stage of progression through the programme, although these latter two variables are interdependent. Positive predictors of clinical reasoning skill are stage of progression through the MBBS programme, female gender and interview score. Although the nature of a student's primary degree is important in the early years of the programme, evidence suggests that by graduation differences between students' clinical reasoning skill due to this factor have been resolved.

Assessing clinical reasoning: A method to monitor its development in a PBL curriculum

The aim of this study was to develop and trial a method to monitor the evolution of clinical reasoning in a PBL curriculum that is suitable for use in a large medical school. Termed Clinical Reasoning Problems (CRPs), it is based on the notion that clinical reasoning is dependent on the identification and correct interpretation of certain critical clinical features. Each problem consists of a clinical scenario comprising presentation, history and physical examination. Based on this information, subjects are asked to nominate the two most likely diagnoses and to list the clinical features that they considered in formulating their diagnoses, indicating whether these features supported or opposed the nominated diagnoses. Students at different levels of medical training completed a set of 10 CRPs as well as the Diagnostic Thinking Inventory, a self-reporting questionnaire designed to assess reasoning style. Responses were scored against those of a reference group of general practitioners. Results indicate that the CRPs are an easily administered, reliable and valid assessment of clinical reasoning, able to successfully monitor its development throughout medical training. Consequently, they can be employed to assess clinical reasoning skill in individual students and to evaluate the success of undergraduate medical schools in providing effective tuition in clinical reasoning.

The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates

Objective: To evaluate the feasibility, reliability and acceptability of the mini clinical evaluation exercise (mini-CEX) for performance assessment among international medical graduates (IMGs). Design, setting and participants: Observational study of 209 patient encounters involving 28 IMGs and 35 examiners at three metropolitan teaching hospitals in New South Wales, Victoria and Queensland, September-December 2006. Main outcome measures: The reliability of the mini-CEX was estimated using generatisability (G) analysis, and its acceptability was evaluated by a written survey of the examiners and IMGs. Results: The G coefficient for eight encounters was 0.88, suggesting that the reliability of the mini-CEX was 0.90 for 10 encounters. Almost half of the IMGs (7/16) and most examiners (14/18) were satisfied with the mini-CEX as a learning tool. Most of the IMGs and examiners enjoyed the immediate feedback, which is a strong component of the tool. Conclusion: The mini-CEX is a reliable tool for performance assessment of IMGs, and is acceptable to and well received by both learners and supervisors.

In-training assessment - Its potential in enhancing clinical teaching

Context In-training assessment (ITA) has established its place alongside formative and summative assessment at both the undergraduate and postgraduate level. In this paper the authors aimed to identify those characteristics of ITA that could enhance clinical teaching. Methods A literature review and discussions by an expert working group at the Ninth Cambridge Conference identified the aspects of ITA that could enhance clinical teaching. Results The features of ITA identified included defining the specific benefits to the learner, teacher and institution, and highlighting the patient as the context for ITA and clinical teaching. The ‘mapping’ of a learner’s progress towards the clinical teaching objectives by using multiple assessments over time, by multiple observers in both a systematic and opportunistic way correlates with the incremental nature of reaching clinical competence. Conclusions The importance of ITA based on both direct and indirect evidence of what the learner actually does in the real clinical setting is emphasized. Particular attention is given to addressing concerns in the more controversial areas of assessor training, ratings and documentation for ITA. Areas for future research are also identified.