Warning Against Faking: Boon or Bane? Effects on Test Score Validity

Previous research found personality test scores to be inflated on average among individuals who were motivated to present themselves in a desirable fashion in high stakes situations, such as during the employee selection process. One apparently effective way to reduce the undesirable test score inflation in such situations was to warn participants against faking. This research set out to investigate whether warning against faking would indeed affect personality test scores in the theoretically expected fashion. Contrary to expectations, the results did not support the hypothesized causal chain. Results across three studies show that while a warning may lower test scores in participants motivated to respond desirably (i.e., to fake), the effect of warning on test scores was not fully mediated by: a reduction in motivation to do well and self-reports of exaggerated responses in the personality test. Theoretical and practical implications are discussed.

Evidências de validade e propriedades psicométricas do SON-R 6-40

Dissertação (mestrado)—Universidade de Brasília, Instituto de Psicologia, 2016.

Reliability, construct and criterion validity of the KIDSCREEN-10 score: a short measure for children and adolescents' well-being and health-related quality of life

Background To assess the criterion and construct validity of the KIDSCREEN-10 well-being and health-related quality of life (HRQoL) score, a short version of the KIDSCREEN-52 and KIDSCREEN-27 instruments. Methods The child self-report and parent report versions of the KIDSCREEN-10 were tested in a sample of 22,830 European children and adolescents aged 8–18 and their parents (n = 16,237). Correlation with the KIDSCREEN-52 and associations with other generic HRQoL measures, physical and mental health, and socioeconomic status were examined. Score differences by age, gender, and country were investigated. Results Correlations between the 10-item KIDSCREEN score and KIDSCREEN-52 scales ranged from r = 0.24 to 0.72 (r = 0.27–0.72) for the self-report version (proxy-report version). Coefficients below r = 0.5 were observed for the KIDSCREEN-52 dimensions Financial Resources and Being Bullied only. Cronbach alpha was 0.82 (0.78), test–retest reliability was ICC = 0.70 (0.67) for the self- (proxy-)report version. Correlations between other children self-completed HRQoL questionnaires and KIDSCREEN-10 ranged from r = 0.43 to r = 0.63 for the KIDSCREEN children self-report and r = 0.22–0.40 for the KIDSCREEN parent proxy report. Known group differences in HRQoL between physically/mentally healthy and ill children were observed in the KIDSCREEN-10 self and proxy scores. Associations with self-reported psychosomatic complaints were r = −0.52 (−0.36) for the KIDSCREEN-10 self-report (proxy-report). Statistically significant differences in KIDSCREEN-10 self and proxy scores were found by socioeconomic status, age, and gender. Conclusions Our results indicate that the KIDSCREEN-10 provides a valid measure of a general HRQoL factor in children and adolescents, but the instrument does not represent well most of the single dimensions of the original KIDSCREEN-52. Test–retest reliability was slightly below a priori defined thresholds.

Validity of SYNTAX score II for risk stratification of percutaneous coronary interventions: A patient-level pooled analysis of 5,433 patients enrolled in contemporary coronary stent trials.

OBJECTIVES To assess the clinical profile and long-term mortality in SYNTAX score II based strata of patients who received percutaneous coronary interventions (PCI) in contemporary randomized trials. BACKGROUND The SYNTAX score II was developed in the randomized, all-comers' SYNTAX trial population and is composed by 2 anatomical and 6 clinical variables. The interaction of these variables with the treatment provides individual long-term mortality predictions if a patient undergoes coronary artery bypass grafting (CABG) or PCI. METHODS Patient-level (n=5433) data from 7 contemporary coronary drug-eluting stent (DES) trials were pooled. The mortality for CABG or PCI was estimated for every patient. The difference in mortality estimates for these two revascularization strategies was used to divide the patients into three groups of theoretical treatment recommendations: PCI, CABG or PCI/CABG (the latter means equipoise between CABG and PCI for long term mortality). RESULTS The three groups had marked differences in their baseline characteristics. According to the predicted risk differences, 5115 patients could be treated either by PCI or CABG, 271 should be treated only by PCI and, rarely, CABG (n=47) was recommended. At 3-year follow-up, according to the SYNTAX score II recommendations, patients recommended for CABG had higher mortality compared to the PCI and PCI/CABG groups (17.4%; 6.1% and 5.3%, respectively; P<0.01). CONCLUSIONS The SYNTAX score II demonstrated capability to help in stratifying PCI procedures.

The reliability and validity of the Severity of Dependence Scale for detecting cannabis dependence in psychosis.

Aims: To determine the reliability and validity of the Severity of Dependence Scale (SDS) for detecting cannabis dependence in a large sample of in-patients with a schizophrenia spectrum disorder. Design: Cross-sectional study. Participants: Participants were 153 in-patients with a schizophrenia spectrum disorder in Brisbane, Australia. Measurements: Participants were administered the SDS for cannabis dependence in the past 12 months. The presence of Diagnostic and Statistical Manual Version-IV (DSM-IV) cannabis dependence in the previous 12 months was assessed using the Comprehensive International Diagnostic Interview (CIDI). Findings: The SDS had high levels of internal consistency and strong construct and concurrent validity. Individuals with a score of ≥2 on the SDS were nearly 30 times more likely to have DSM-IV cannabis dependence. The SDS was the strongest predictor of DSM-IV cannabis dependence after controlling for other predictor variables. Conclusions: The SDS is a brief, valid and reliable screen for cannabis dependence among people with psychosis

The Validity of the Brief Michigan Alcohol Screening Test (bMAST) as a Problem Drinking Severity Measure

Objective: The Brief Michigan Alcoholism Screening Test (bMAST) is a 10-item test derived from the 25-item Michigan Alcoholism Screening Test (MAST). It is widely used in the assessment of alcohol dependence. In the absence of previous validation studies, the principal aim of this study was to assess the validity and reliability of the bMAST as a measure of the severity of problem drinking. Method: There were 6,594 patients (4,854 men, 1,740 women) who had been referred for alcohol-use disorders to a hospital alcohol and drug service who voluntarily participated in this study. Results: An exploratory factor analysis defined a two-factor solution, consisting of Perception of Current Drinking and Drinking Consequences factors. Structural equation modeling confirmed that the fit of a nine-item, two-factor model was superior to the original one-factor model. Concurrent validity was assessed through simultaneous administration of the Alcohol Use Disorders Identification Test (AUDIT) and associations with alcohol consumption and clinically assessed features of alcohol dependence. The two-factor bMAST model showed moderate correlations with the AUDIT. The two-factor bMAST and AUDIT were similarly associated with quantity of alcohol consumption and clinically assessed dependence severity features. No differences were observed between the existing weighted scoring system and the proposed simple scoring system. Conclusions: In this study, both the existing bMAST total score and the two-factor model identified were as effective as the AUDIT in assessing problem drinking severity. There are additional advantages of employing the two-factor bMAST in the assessment and treatment planning of patients seeking treatment for alcohol-use disorders. (J. Stud. Alcohol Drugs 68: 771-779,2007)

Clinical indicators of quality for Australian residential aged care facilities : establishing reliability, validity, and quality thresholds

Background: In response to the need for more comprehensive quality assessment within Australian residential aged care facilities, the Clinical Care Indicator (CCI) Tool was developed to collect outcome data as a means of making inferences about quality. A national trial of its effectiveness and a Brisbane-based trial of its use within the quality improvement context determined the CCI Tool represented a potentially valuable addition to the Australian aged care system. This document describes the next phase in the CCI Tool.s development; the aims of which were to establish validity and reliability of the CCI Tool, and to develop quality indicator thresholds (benchmarks) for use in Australia. The CCI Tool is now known as the ResCareQA (Residential Care Quality Assessment). Methods: The study aims were achieved through a combination of quantitative data analysis, and expert panel consultations using modified Delphi process. The expert panel consisted of experienced aged care clinicians, managers, and academics; they were initially consulted to determine face and content validity of the ResCareQA, and later to develop thresholds of quality. To analyse its psychometric properties, ResCareQA forms were completed for all residents (N=498) of nine aged care facilities throughout Queensland. Kappa statistics were used to assess inter-rater and test-retest reliability, and Cronbach.s alpha coefficient calculated to determine internal consistency. For concurrent validity, equivalent items on the ResCareQA and the Resident Classification Scales (RCS) were compared using Spearman.s rank order correlations, while discriminative validity was assessed using known-groups technique, comparing ResCareQA results between groups with differing care needs, as well as between male and female residents. Rank-ordered facility results for each clinical care indicator (CCI) were circulated to the panel; upper and lower thresholds for each CCI were nominated by panel members and refined through a Delphi process. These thresholds indicate excellent care at one extreme and questionable care at the other. Results: Minor modifications were made to the assessment, and it was renamed the ResCareQA. Agreement on its content was reached after two Delphi rounds; the final version contains 24 questions across four domains, enabling generation of 36 CCIs. Both test-retest and inter-rater reliability were sound with median kappa values of 0.74 (test-retest) and 0.91 (inter-rater); internal consistency was not as strong, with a Chronbach.s alpha of 0.46. Because the ResCareQA does not provide a single combined score, comparisons for concurrent validity were made with the RCS on an item by item basis, with most resultant correlations being quite low. Discriminative validity analyses, however, revealed highly significant differences in total number of CCIs between high care and low care groups (t199=10.77, p=0.000), while the differences between male and female residents were not significant (t414=0.56, p=0.58). Clinical outcomes varied both within and between facilities; agreed upper and lower thresholds were finalised after three Delphi rounds. Conclusions: The ResCareQA provides a comprehensive, easily administered means of monitoring quality in residential aged care facilities that can be reliably used on multiple occasions. The relatively modest internal consistency score was likely due to the multi-factorial nature of quality, and the absence of an aggregate result for the assessment. Measurement of concurrent validity proved difficult in the absence of a gold standard, but the sound discriminative validity results suggest that the ResCareQA has acceptable validity and could be confidently used as an indication of care quality within Australian residential aged care facilities. The thresholds, while preliminary due to small sample size, enable users to make judgements about quality within and between facilities. Thus it is recommended the ResCareQA be adopted for wider use.

Validity and reliability of nutrition screening administered by nurses

Background: Nutrition screening is usually administered by nurses. However, most studies on nutrition screening tools have not used nurses to validate the tools. The 3-Minute Nutrition Screening (3-MinNS) assesses weight loss, dietary intake and muscle wastage, with the composite score of each used to determine risk of malnutrition. The aim of the study was to determine the validity and reliability of 3-MinNS administered by nurses, who are the intended assessors. Methods: In this cross sectional study, three ward-based nurses screened 121 patients aged 21 years and over using 3-MinNS in three wards within 24 hours of admission. A dietitian then assessed the patients’ nutritional status using Subjective Global Assessment within 48 hours of admission, whilst blinded to the results of the screening. To assess the reliability of 3-MinNS, 37 patients screened by the first nurse were re-screened by a second nurse within 24 hours, who was blinded to the results of the first nurse. The sensitivity, specificity and best cutoff score for 3-MinNS were determined using the Receiver Operator Characteristics Curve. Results: The best cutoff score to identify all patients at risk of malnutrition using 3-MinNS was three, with sensitivity of 89% and specificity of 88%. This cutoff point also identified all (100%) severely malnourished patients. There was strong correlation between 3-MinNS and SGA (r=0.78, p<0.001). The agreement between two nurses conducting the 3-MinNS tool was 78.3%. Conclusion: 3-Minute Nutrition Screening is a valid and reliable tool for nurses to identify patients at risk of malnutrition.

Validity and reliability of 3-minute nutrition screening (3-MinNS)performed by nurses on oncology and surgical patients

Background Nutrition screening is usually administered by nurses. However, most studies on nutrition screening tools have not used nurses to validate the tools. The 3-Minute Nutrition Screening (3-MinNS) assesses weight loss, dietary intake and muscle wastage, with the composite score of each used to determine risk of malnutrition. The aim of the study was to determine the validity and reliability of 3-MinNS administered by nurses, who are the intended assessors. Methods In this cross sectional study, three ward-based nurses screened 121 patients aged 21 years and over using 3-MinNS in three wards within 24 hours of admission. A dietitian then assessed the patients’ nutritional status using Subjective Global Assessment within 48 hours of admission, whilst blinded to the results of the screening. To assess the reliability of 3-MinNS, 37 patients screened by the first nurse were re-screened by a second nurse within 24 hours, who was blinded to the results of the first nurse. The sensitivity, specificity and best cutoff score for 3-MinNS were determined using the Receiver Operator Characteristics Curve. Results The best cutoff score to identify all patients at risk of malnutrition using 3-MinNS was three, with sensitivity of 89% and specificity of 88%. This cutoff point also identified all (100%) severely malnourished patients. There was strong correlation between 3-MinNS and SGA (r=0.78, p<0.001). The agreement between two nurses conducting the 3-MinNS tool was 78.3%. Conclusion 3-Minute Nutrition Screening is a valid and reliable tool for nurses to identify patients at risk of malnutrition.

The validity and reliability of the Parent Fever Management Scale: A study from Palestine

Background Parental fever phobia and overuse of antipyretics to control fever is increasing. Little is known about childhood fever management among Arab parents. No scales to measure parents’ fever management practices in Palestine are available. Aims The aims of this study were to translate and examine the psychometric properties of the Arabic version of the Parent Fever Management Scale (PFMS). Methods A standard “forward–backward” procedure was used to translate PFMS into Arabic language. It was then validated on a convenience sample of 402 parents between July and October 2012. Descriptive statistics were used, and instrument reliability was assessed for internal consistency using Cronbach's alpha coefficient. Validity was confirmed using convergent and known group validation. Results Applying the recommended scoring method, the median (interquartile range) score of the PFMS was 26 (23-30). Acceptable internal consistency was found (Cronbach’s alpha = 0.733) and the test–retest reliability value was 0.92 (P < 0.001). The chi-squared (χ2) test showed a significant relationship between PFMS groups and frequent daily administration of antipyretic groups (χ2 = 52.86; P < 0.001). The PFMS sensitivity and specificity were 77.67% and 57.75%, respectively. The positive and negative predictive values were 67.89% and 32.11%, respectively. Conclusions The findings of this validation study indicate that the Arabic version of the PFMS is a reliable and valid measure which can be used as a useful tool for health professionals to identify parents’ fever management practices and thus provide targeted education to reduce the unnecessary burden of care they place on themselves when concerned for a febrile child.

The validity of personal disturbance scale (DSSI/sAD) in people with diabetes mellitus, using longitudinal data

Despite being used since 1976, Delusions-Symptoms-States-Inventory/states of Anxiety and Depression (DSSI/sAD) has not yet been validated for use among people with diabetes. The aim of this study was to examine the validity of the personal disturbance scale (DSSI/sAD) among women with diabetes using Mater-University of Queensland Study of Pregnancy (MUSP) cohort data. The DSSI subscales were compared against DSM-IV disorders, the Mental Component Score of the Short Form 36 (SF-36 MCS), and Center for Epidemiologic Studies Depression Scale (CES-D). Factor analyses, odds ratios, receiver operating characteristic (ROC) analyses and diagnostic efficiency tests were used to report findings. Exploratory factor analysis and fit indices confirmed the hypothesized two-factor model of DSSI/sAD. We found significant variations in the DSSI/sAD domain scores that could be explained by CES-D (DSSI-Anxiety: 55%, DSSI-Depression: 46%) and SF-36 MCS (DSSI-Anxiety: 66%, DSSI-Depression: 56%). The DSSI subscales predicted DSM-IV diagnosed depression and anxiety disorders. The ROC analyses show that although the DSSI symptoms and DSM-IV disorders were measured concurrently the estimates of concordance remained only moderate. The findings demonstrate that the DSSI/sAD items have similar relationships to one another in both the diabetes and non-diabetes data sets which therefore suggest that they have similar interpretations.

Predicting final disposition after stroke using the Orpington Prognostic Score

Background: Prediction of outcome after stroke is important for triage decisions, prognostic estimates for family and for appropriate resource utilization. Prognostication must be timely and simply applied. Several scales have shown good prognostic value. In Calgary, the Orpington Prognostic Score (OPS) has been used to predict outcome as an aid to rehabilitation triage. However, the OPS has not been assessed at one week for predictive capability. Methods: Among patients admitted to a sub-acute stroke unit, OPS from the first week were examined to determine if any correlation existed between final disposition after rehabilitation and first week score. The predictive validity of the OPS at one week was compared to National Institute of Health Stroke Scale (NIHSS) score at 24 hours using logistic regression and receiver operator characteristics analysis. The primary outcome was final disposition after discharge from the stroke unit if the patient went directly home, or died, or from the inpatient rehabilitation unit. Results: The first week OPS was highly predictive of final disposition. However, no major advantage in using the first week OPS was observed when compared to 24h NIHSS score. Both scales were equally predictive of final disposition of stroke patients, post rehabilitation. Conclusion: The first week OPS can be used to predict final outcome. The NIHSS at 24h provides the same prognostic information.

Development, reliability and validity of the queensland evaluation of wheelchair skills (QEWS)

- Objectives To develop and test a valid and reliable assessment of wheelchair skills for individuals with spinal cord injuries (SCI); the Queensland Evaluation of Wheelchair Skills (QEWS). - Setting Hospital, Australia. - Methods Phase 1: Four Delphi panel rounds with clinical experts were used to develop the QEWS. Phase 2: Intra-rater and inter-rater reliability of the QEWS items were examined in 100 people with SCI. Phase 3a: Concurrent validity was investigated by examining the association between QEWS total scores and physiotherapists’ global ratings of wheelchair skill performance. Phase 3b: Construct validity was tested in 20 people with recent SCI by examining change in QEWS total scores between when they first mobilised in a wheelchair and scores obtained 10 weeks later. - Results Phase 1: The QEWS was developed. Phase 2: The intra-class correlation coefficients reflecting the intra-rater reliability and the inter-rater reliability for the QEWS total score were 1.00 and 0.98, with scores being within one point of each other 96 and 91% of the time, respectively. Phase 3a: The QEWS total scores were comparable with the global rating of wheelchair skill performance (r2=0.93). Phase 3b: The QEWS scores changed by a median (interquartile range (IQR)) of 4 (1 to 6) points over the 10-week period following first wheelchair mobilisation. - Conclusion The QEWS is a valid and reliable tool for measuring wheelchair skills in individuals with SCI. The QEWS is efficient and practical to administer and does not require specialised equipment.

Tradução, adaptação cultural e validação do Nonarthritic Hip Score para o Brasil

A avaliação da qualidade de vida tem sido cada vez mais utilizada pelos profissionais da área de saúde para mensurar o impacto de doenças na vida dos pacientes, bem como para avaliar os resultados dos tratamentos realizados. O crescente interesse por protocolos de pesquisa clínica em doenças não degenerativas do quadril tem encontrado muitos obstáculos na avaliação objetiva de seus resultados, principalmente nos estudos de observação de novas intervenções terapêuticas, como a artroscopia. O Nonarthritic Hip Score (NAHS) é um instrumento de avaliação clínica, desenvolvido originalmente em inglês, cujo objetivo é avaliar a função da articulação do quadril em pacientes jovens e fisicamente ativos. O objetivo desse estudo foi traduzir esse instrumento para a língua portuguesa, adaptá-lo para a cultura brasileira e validá-lo para que possa ser utilizado na avaliação de qualidade de vida de pacientes brasileiros com dor no quadril, sem doença degenerativa. A metodologia utilizada é a sugerida por Guillemin et al. (1993) e revisado por Beaton et al. (2000), que propuseram um conjunto de instruções padronizadas para adaptação cultural de instrumentos de qualidade de vida, incluindo cinco etapas: tradução, tradução de volta, revisão pelo comitê, pré-teste e teste, com reavaliação dos pesos dos escores, se relevante. A versão de consenso foi aplicada em 30 indivíduos. As questões sobre atividades esportivas e tarefas domésticas foram modificadas, para melhor adaptação à cultura brasileira. A versão brasileira do Nonarthritic Hip Score (NAHS-Brasil) foi respondida por 64 pacientes com dor no quadril, a fim de avaliar as propriedades de medida do instrumento: reprodutibilidade, consistência interna e validade. A reprodutibilidade foi 0,9, mostrando uma forte correlação; a consistência interna mostrou correlação entre 0,8 e 0,9, considerada boa e excelente; a validade foi considerada respectivamente boa e excelente; a correlação entre NAHS-Brasil e WOMAC foi 0,9; e a correlação entre o NAHS-Brasil e Questionário Algofuncional de Lequesne foi 0,79. O Nonarthritic Hip Score foi traduzido para a língua portuguesa e adaptado à cultura brasileira, de acordo com o conjunto de instruções padronizadas para adaptação cultural de instrumentos de qualidade de vida. Sua reprodutibilidade, consistência interna e validade foram também demonstradas.

Entorhinal cortex volume in older adults: reliability and validity considerations for three published measurement protocols.

Measuring the entorhinal cortex (ERC) is challenging due to lateral border discrimination from the perirhinal cortex. From a sample of 39 nondemented older adults who completed volumetric image scans and verbal memory indices, we examined reliability and validity concerns for three ERC protocols with different lateral boundary guidelines (i.e., Goncharova, Dickerson, Stoub, & deToledo-Morrell, 2001; Honeycutt et al., 1998; Insausti et al., 1998). We used three novice raters to assess inter-rater reliability on a subset of scans (216 total ERCs), with the entire dataset measured by one rater with strong intra-rater reliability on each technique (234 total ERCs). We found moderate to strong inter-rater reliability for two techniques with consistent ERC lateral boundary endpoints (Goncharova, Honeycutt), with negligible to moderate reliability for the technique requiring consideration of collateral sulcal depth (Insausti). Left ERC and story memory associations were moderate and positive for two techniques designed to exclude the perirhinal cortex (Insausti, Goncharova), with the Insausti technique continuing to explain 10% of memory score variance after additionally controlling for depression symptom severity. Right ERC-story memory associations were nonexistent after excluding an outlier. Researchers are encouraged to consider challenges of rater training for ERC techniques and how lateral boundary endpoints may impact structure-function associations.