How standardised patients award global scores in OSCEs: a qualitative study

Standardized patients (SPs) are often asked to award global scores on the humanistic aspects of a candidate’s performance in an OSCE. However, little is known about the process by which SPs arrive at their mark.
Five focus groups of SPs, using a convenience sample, were used to collect data until saturation. Thematic analysis was carried out independently by three researchers using a grounded theory approach.

Four major themes contributed to their decision-making process: environment, relationships within the exam, preparedness for the task and expectations of the student’s performance. Environmental factors included the station itself, the rating scale and examiner fatigue. Relationship factors included first impressions, the sense of purpose derived from examining and a tendency to mirror the examiner’s reaction. Factors relating to preparedness for task involved experience as an SP and technical aspects, such as the need for calibration. Lastly, expectations of performance were related to preconceptions about what makes a ‘good’ student, including their level of studies, appearance and technical performance.

In assessing students, SPs drew on their wider attitudes and experiences. SPs did not limit their assessment to humanistic traits but often included technical performance. Thus, SPs to some extent assessed a similar construct to examiners and this may help to explain the increased reliability associated with using SP scores. SP global scores are a useful adjunct but the process by which SPs award marks is complex and provides a challenge for training and standardisation.

A comparison of Diamond Forrester and coronary calcium scores as gatekeepers for investigations of stable chest pain

To determine if calcium scores (CS) could act as a more effective gatekeeper than Diamond Forrester (DF) in the assessment of patients with suspected coronary artery disease (CAD). A sub-study of the Cardiac CT for the Assessment of Chest Pain and Plaque (CAPP) study, a randomised control trial evaluating the cost-effectiveness of cardiac CT in symptomatic patients with stable chest pain. Stable pain was defined as troponin negative pain without symptoms of unstable angina. 250 patients undergoing cardiac CT had both DF scores and CS calculated, with the accuracy of both evaluated against CT coronary angiogram. Criteria given in UK national guidelines were compared. Of the 250 patients, 4 withdrew. 140 (57 %) patients were male. The mean DF was 47.8 and mean CS 172.5. Of the 144 patients with non-anginal pain 19.4 % had significant disease (>50 % stenosis). In general the DF over estimated the presence of CAD whereas the CS reclassified patients to lower risk groups, with 91 in the high risk DF category compared to 26 in the CS. Both receiver operating curve and McNemar Bowker test analysis suggested the DF was less accurate in the prediction of CAD compared to CS [Formula: see text] Projected downstream investigations were also calculated, with the cost per number of significant stenoses identified cheaper with the CS criteria. Patients with suspected stable CAD are more accurately risk stratified by CS compared to the traditional DF. CS was more successful in the prediction of significant stenosis and appears to be more effective at targeting clinical resources to those patients that are in need of them.

Age related differences in mental health scale scores and depression diagnosis: Adult responses to the CIDI-SF and MHI-5.

Inconsistencies surrounding the prevalence levels of depression in later life suggest that the measurement of depression in older people may be problematic. The current study aimed to map responses to a depressive symptom scale, the Mental Health Index-5 (MHI-5) which is part of the Short form 36 (SF-36, Ware et al., 1993) against the diagnostic screening items of the Composite International Diagnostic Instrument-Short Form (CIDI-SF, Kessler et al., 1998) to examine disagreement rates across age groups. The study examined data from a national random sample of 10,641 participants living in Ireland, 58.8% were female and 19% were over 65 (SLÁN, 2007). CIDI-SF depression screening endorsement was lower in older groups, whereas mean MHI-5 depressive symptoms showed less change across age groups. Results showed that the odds of MHI-5 endorsers aged 18–44 endorsing CIDI-SF screening questions were 5 times and 4.5 times (dysphoria and anhedonia, respectively) greater than the odds of people aged 75 or more endorsing these items. Findings suggest that although the risk of depressive disorder may decrease with age, complex diagnostic screening questions may exaggerate lower rates of depression among older people.

Correlation of phantom-based and log file patient-specific QA with complexity scores for VMAT

The motivation for this study was to reduce physics workload relating to patient- specific quality assurance (QA). VMAT plan delivery accuracy was determined from analysis of pre- and on-treatment trajectory log files and phantom-based ionization chamber array measurements. The correlation in this combination of measurements for patient-specific QA was investigated. The relationship between delivery errors and plan complexity was investigated as a potential method to further reduce patient-specific QA workload. Thirty VMAT plans from three treatment sites - prostate only, prostate and pelvic node (PPN), and head and neck (H&N) - were retrospectively analyzed in this work. The 2D fluence delivery reconstructed from pretreatment and on-treatment trajectory log files was compared with the planned fluence using gamma analysis. Pretreatment dose delivery verification was also car- ried out using gamma analysis of ionization chamber array measurements compared with calculated doses. Pearson correlations were used to explore any relationship between trajectory log file (pretreatment and on-treatment) and ionization chamber array gamma results (pretreatment). Plan complexity was assessed using the MU/ arc and the modulation complexity score (MCS), with Pearson correlations used to examine any relationships between complexity metrics and plan delivery accu- racy. Trajectory log files were also used to further explore the accuracy of MLC and gantry positions. Pretreatment 1%/1 mm gamma passing rates for trajectory log file analysis were 99.1% (98.7%-99.2%), 99.3% (99.1%-99.5%), and 98.4% (97.3%-98.8%) (median (IQR)) for prostate, PPN, and H&N, respectively, and were significantly correlated to on-treatment trajectory log file gamma results (R = 0.989, p < 0.001). Pretreatment ionization chamber array (2%/2 mm) gamma results were also significantly correlated with on-treatment trajectory log file gamma results (R = 0.623, p < 0.001). Furthermore, all gamma results displayed a significant correlation with MCS (R > 0.57, p < 0.001), but not with MU/arc. Average MLC position and gantry angle errors were 0.001 ± 0.002 mm and 0.025° ± 0.008° over all treatment sites and were not found to affect delivery accuracy. However, vari- ability in MLC speed was found to be directly related to MLC position accuracy. The accuracy of VMAT plan delivery assessed using pretreatment trajectory log file fluence delivery and ionization chamber array measurements were strongly correlated with on-treatment trajectory log file fluence delivery. The strong corre- lation between trajectory log file and phantom-based gamma results demonstrates potential to reduce our current patient-specific QA. Additionally, insight into MLC and gantry position accuracy through trajectory log file analysis and the strong cor- relation between gamma analysis results and the MCS could also provide further methodologies to both optimize the VMAT planning and QA process. 

Min-BDeu and Max-BDeu Scores for Learning Bayesian Networks

This work presents two new score functions based on the Bayesian Dirichlet equivalent uniform (BDeu) score for learning Bayesian network structures. They consider the sensitivity of BDeu to varying parameters of the Dirichlet prior. The scores take on the most adversary and the most beneficial priors among those within a contamination set around the symmetric one. We build these scores in such way that they are decomposable and can be computed efficiently. Because of that, they can be integrated into any state-of-the-art structure learning method that explores the space of directed acyclic graphs and allows decomposable scores. Empirical results suggest that our scores outperform the standard BDeu score in terms of the likelihood of unseen data and in terms of edge discovery with respect to the true network, at least when the training sample size is small. We discuss the relation between these new scores and the accuracy of inferred models. Moreover, our new criteria can be used to identify the amount of data after which learning is saturated, that is, additional data are of little help to improve the resulting model.

Trio Performance with 'Faint' at "Sounds New Contemporary Music Festival":Graphic Scores and Free improvisation: Canterbury, Kent

The programme contained a performance by the trio FAINT (Pedro Rebelo - Piano and instru- mental parasites, Franziska Schroeder - Saxophone and Steve Davis - Drums). The performance includes short electroacoustic works based on the trio's free improvisation and a performance of Rebelo's Cipher Series graphic scores.

Modeling Linkage Disequilibrium Increases Accuracy of Polygenic Risk Scores

Polygenic risk scores have shown great promise in predicting complex disease risk and will become more accurate as training sample sizes increase. The standard approach for calculating risk scores involves linkage disequilibrium (LD)-based marker pruning and applying a p value threshold to association statistics, but this discards information and can reduce predictive accuracy. We introduce LDpred, a method that infers the posterior mean effect size of each marker by using a prior on effect sizes and LD information from an external reference panel. Theory and simulations show that LDpred outperforms the approach of pruning followed by thresholding, particularly at large sample sizes. Accordingly, predicted R(2) increased from 20.1% to 25.3% in a large schizophrenia dataset and from 9.8% to 12.0% in a large multiple sclerosis dataset. A similar relative improvement in accuracy was observed for three additional large disease datasets and for non-European schizophrenia samples. The advantage of LDpred over existing methods will grow as sample sizes increase.

Reliability and factorial validity of Adolescent Time Inventory-Time Attitude (ATI-TA) Scores in Scottish and Northern Irish adolescents

The contemporary literature investigating the construct broadly known as time perspective is replete with methodological and conceptual concerns. These concerns focus on the reliability and factorial validity of measurement tools, and the sample-specific modification of scales. These issues continue to hamper the development of this potentially useful psychological construct. An emerging body of evidence has supported the six-factor structure of scores on the Adolescent Time Inventory-Time Attitudes Scale, as well as their reliability. The present study utilized data from the first wave of a longitudinal study in the United Kingdom to examine the reliability, validity, and cross-cultural invariance of the scale. Results showed that the hypothesized six-factor model provided the best fit for the data; all alpha and omega estimates were >. .70; scores on ATI-TA factors related meaningfully to self-efficacy scores; and the factor structure was invariant across both research sites. Results are discussed in the context of the extant temporal literature.

Validation of Two US Risk Scores for Percutaneous Coronary Intervention in a Single-Center Portuguese Population of Patients with Acute Coronary Syndrome

INTRODUCTION: New scores have been developed and validated in the US for in-hospital mortality risk stratification in patients undergoing coronary angioplasty: the National Cardiovascular Data Registry (NCDR) risk score and the Mayo Clinic Risk Score (MCRS). We sought to validate these scores in a European population with acute coronary syndrome (ACS) and to compare their predictive accuracy with that of the GRACE risk score. METHODS: In a single-center ACS registry of patients undergoing coronary angioplasty, we used the area under the receiver operating characteristic curve (AUC), a graphical representation of observed vs. expected mortality, and net reclassification improvement (NRI)/integrated discrimination improvement (IDI) analysis to compare the scores. RESULTS: A total of 2148 consecutive patients were included, mean age 63 years (SD 13), 74% male and 71% with ST-segment elevation ACS. In-hospital mortality was 4.5%. The GRACE score showed the best AUC (0.94, 95% CI 0.91-0.96) compared with NCDR (0.87, 95% CI 0.83-0.91, p=0.0003) and MCRS (0.85, 95% CI 0.81-0.90, p=0.0003). In model calibration analysis, GRACE showed the best predictive power. With GRACE, patients were more often correctly classified than with MCRS (NRI 78.7, 95% CI 59.6-97.7; IDI 0.136, 95% CI 0.073-0.199) or NCDR (NRI 79.2, 95% CI 60.2-98.2; IDI 0.148, 95% CI 0.087-0.209). CONCLUSION: The NCDR and Mayo Clinic risk scores are useful for risk stratification of in-hospital mortality in a European population of patients with ACS undergoing coronary angioplasty. However, the GRACE score is still to be preferred.

The reasons behind the progression in PISA scores: An education production function approach using semi-parametric techniques

Research Masters

Scores to predict major bleeding risk during oral anticoagulation therapy: a prospective validation study.

BACKGROUND: Clinical scores may help physicians to better assess the individual risk/benefit of oral anticoagulant therapy. We aimed to externally validate and compare the prognostic performance of 7 clinical prediction scores for major bleeding events during oral anticoagulation therapy. METHODS: We followed 515 adult patients taking oral anticoagulants to measure the first major bleeding event over a 12-month follow-up period. The performance of each score to predict the risk of major bleeding and the physician's subjective assessment of bleeding risk were compared with the C statistic. RESULTS: The cumulative incidence of a first major bleeding event during follow-up was 6.8% (35/515). According to the 7 scoring systems, the proportions of major bleeding ranged from 3.0% to 5.7% for low-risk, 6.7% to 9.9% for intermediate-risk, and 7.4% to 15.4% for high-risk patients. The overall predictive accuracy of the scores was poor, with the C statistic ranging from 0.54 to 0.61 and not significantly different from each other (P=.84). Only the Anticoagulation and Risk Factors in Atrial Fibrillation score performed slightly better than would be expected by chance (C statistic, 0.61; 95% confidence interval, 0.52-0.70). The performance of the scores was not statistically better than physicians' subjective risk assessments (C statistic, 0.55; P=.94). CONCLUSION: The performance of 7 clinical scoring systems in predicting major bleeding events in patients receiving oral anticoagulation therapy was poor and not better than physicians' subjective assessments.

Estimating the risk of developing type 2 diabetes: a comparison of several risk scores: the Cohorte Lausannoise study.

To compare in the Swiss population the results of several scores estimating the risk of developing type 2 diabetes. This was a single-center, cross-sectional study conducted between 2003 and 2006 in Lausanne, Switzerland. Overall, 3,251 women and 2,937 men, aged 35-75 years, were assessed, of which 5,760 (93%) were free from diabetes and included in the current study. The risk of developing type 2 diabetes was assessed using seven different risk scores, including clinical data with or without biological data. Participants were considered to be eligible for primary prevention according to the thresholds provided for each score. The results were then extrapolated to the Swiss population of the same sex and age. The risk of developing type 2 diabetes increased with age in all scores. The prevalence of participants at high risk ranged between 1.6 and 24.9% in men and between 1.1 and 15.7% in women. Extrapolated to the Swiss population of similar age, the overall number of participants at risk, and thus susceptible to intervention, ranged between 46,708 and 636,841. In addition, scores that included the same clinical variables led to a significantly different prevalence of participants at risk (4.2% [95% CI 3.4-5.0] vs. 12.8% [11.5-14.1] in men and 2.9% [2.4-3.6] vs. 6.0% [5.2-6.9] in women). CONCLUSIONS; The prevalence of participants at risk for developing type 2 diabetes varies considerably according to the scoring system used. To adequately prevent type 2 diabetes, risk-scoring systems must be validated for each population considered.

Comparison of clinical and angiographic prognostic risk scores in elderly patients presenting with acute coronary syndrome and referred for percutaneous coronary intervention.

BACKGROUND: Multiple risk prediction models have been validated in all-age patients presenting with acute coronary syndrome (ACS) and treated with percutaneous coronary intervention (PCI); however, they have not been validated specifically in the elderly. METHODS: We calculated the GRACE (Global Registry of Acute Coronary Events) score, the logistic EuroSCORE, the AMIS (Acute Myocardial Infarction Swiss registry) score, and the SYNTAX (Synergy between Percutaneous Coronary Intervention with TAXUS and Cardiac Surgery) score in a consecutive series of 114 patients ≥75 years presenting with ACS and treated with PCI within 24 hours of hospital admission. Patients were stratified according to score tertiles and analysed retrospectively by comparing the lower/mid tertiles as an aggregate group with the higher tertile group. The primary endpoint was 30-day mortality. Secondary endpoints were the composite of death and major adverse cardiovascular events (MACE) at 30 days, and 1-year MACE-free survival. Model discrimination ability was assessed using the area under receiver operating characteristic curve (AUC). RESULTS: Thirty-day mortality was higher in the upper tertile compared with the aggregate lower/mid tertiles according to the logistic EuroSCORE (42% vs 5%; odds ratio [OR] = 14, 95% confidence interval [CI] = 4-48; p <0.001; AUC = 0.79), the GRACE score (40% vs 4%; OR = 17, 95% CI = 4-64; p <0.001; AUC = 0.80), the AMIS score (40% vs 4%; OR = 16, 95% CI = 4-63; p <0.001; AUC = 0.80), and the SYNTAX score (37% vs 5%; OR = 11, 95% CI = 3-37; p <0.001; AUC = 0.77). CONCLUSIONS: In elderly patients presenting with ACS and referred to PCI within 24 hours of admission, the GRACE score, the EuroSCORE, the AMIS score, and the SYNTAX score predicted 30 day mortality. The predictive value of clinical scores was improved by using them in combination.