Propensity score matching and variations on the balancing test

Balancing tests are diagnostics designed for use with propensity score methods, a widely used non-experimental approach in the evaluation literature. Such tests provide useful information on whether plausible counterfactuals have been created. Currently, multiple balancing tests exist in the literature but it is unclear which is the most useful. This article highlights the poor size properties of commonly employed balancing tests and attempts to shed some light on the link between the results of balancing tests and bias of the evaluation estimator. The simulation results suggest that in scenarios where the conditional independence assumption holds, a permutation version of the balancing test described in Dehejia and Wahba (Rev Econ Stat 84:151–161, 2002) can be useful in applied study. The proposed test has good size properties. In addition, the test appears to have good power for detecting a misspecification in the link function and some power for detecting an omission of relevant non-linear terms involving variables that are included at a lower order.

Test score disclosure and school performance

In this paper we test whether the disclosure of test scores has direct impacts on student performance, school composition and school inputs. We take advantage of the discontinuity on the disclosure rules of The National Secondary Education Examination (ENEM) run in Brazil by the Ministry of Education: In 2006 it was established that the 2005 mean score results would be disclosed for schools with ten or more students who took the exam in the previous year. We use a regression discontinuity design to estimate the e ects of test disclosure. Our results indicate that private schools that had their average scores released in 2005 outperformed those that did not by 0.2-0.6 in 2007. We did not nd same results for public schools. Moreover, we did not nd evidence that treated schools adjusted their inputs or that there was major changes in the students composition of treated schools. These ndings allow us to interpret that the main mechanism driving the di erences in performance was the increased levels of students', teachers' and principals' e ort exerted by those in schools that had scores publicized.

Propensity score matching approach to test the association of income inequality and mortality in Sao Paulo, Brazil

Background Support for the adverse effect of high income inequality on population health has come from studies that focus on larger areas, such as the US states, while studies at smaller geographical areas (eg, neighbourhoods) have found mixed results. Methods We used propensity score matching to examine the relationship between income inequality and mortality rates across 96 neighbourhoods (distritos) of the municipality of Sao Paulo, Brazil. Results Prior to matching, higher income inequality distritos (Gini >= 0.25) had slightly lower overall mortality rates (2.23 per 10 000, 95% CI -23.92 to 19.46) compared to lower income inequality areas (Gini <0.25). After propensity score matching, higher inequality was associated with a statistically significant higher mortality rate (41.58 per 10 000, 95% CI 8.85 to 73.3). Conclusion In Sao Paulo, the more egalitarian communities are among some of the poorest, with the worst health profiles. Propensity score matching was used to avoid inappropriate comparisons between the health status of unequal (but wealthy) neighbourhoods versus equal (but poor) neighbourhoods. Our methods suggest that, with proper accounting of heterogeneity between areas, income inequality is associated with worse population health in Sao Paulo.

Six minute walk test Z score: Correlations with cystic fibrosis severity markers

Background: The six-minute-walk-test (6MWT) has been increasingly used in cystic fibrosis (CF) patients. However, few studies in children have correlated 6MWT with current parameters used to evaluate CF severity. Moreover, no study transformed the values of distance walked from meters into Z scores to avoid bias like age and gender, which are sources of 6MWT variability. Methods: A cross-sectional descriptive study was performed to analyze the correlations (Spearman) among forced expiratory volume in one second (FEV1), body mass index (BMI), chest radiography (CXR), chest tomography (CT), and 6MWT Z score (Z-6MWT). Clinically stable CF patients, aged 6-21 years, were included. Results: 34 patients, 14F/20M, mean age 12.1 +/- 4.0 years were studied. The mean Z-6MWT was -1.1 +/- 1.106. The following correlations versus Z-6MWT were found: FEV1 (r=0.59, r(2)=0.32, p=0.0002), BMI Z score (r=0.42, r(2)=0.17, p=0.013), CXR (r=0.34, r(2)=0.15, p=0.0472) and CT (r=-0.45, r(2)=0.23, p=0.0073). Conclusions: In conclusion there was a significant, but poor, correlation between the six minute walk test Z score and the cystic fibrosis severity markers currently in use. (C) 2011 European Cystic Fibrosis Society. Published by Elsevier B.V. All rights reserved.

Performance of gene-expression profiling test score variability to predict future clinical events in heart transplant recipients.

BACKGROUND A single non-invasive gene expression profiling (GEP) test (AlloMap®) is often used to discriminate if a heart transplant recipient is at a low risk of acute cellular rejection at time of testing. In a randomized trial, use of the test (a GEP score from 0-40) has been shown to be non-inferior to a routine endomyocardial biopsy for surveillance after heart transplantation in selected low-risk patients with respect to clinical outcomes. Recently, it was suggested that the within-patient variability of consecutive GEP scores may be used to independently predict future clinical events; however, future studies were recommended. Here we performed an analysis of an independent patient population to determine the prognostic utility of within-patient variability of GEP scores in predicting future clinical events. METHODS We defined the GEP score variability as the standard deviation of four GEP scores collected ≥315 days post-transplantation. Of the 737 patients from the Cardiac Allograft Rejection Gene Expression Observational (CARGO) II trial, 36 were assigned to the composite event group (death, re-transplantation or graft failure ≥315 days post-transplantation and within 3 years of the final GEP test) and 55 were assigned to the control group (non-event patients). In this case-controlled study, the performance of GEP score variability to predict future events was evaluated by the area under the receiver operator characteristics curve (AUC ROC). The negative predictive values (NPV) and positive predictive values (PPV) including 95 % confidence intervals (CI) of GEP score variability were calculated. RESULTS The estimated prevalence of events was 17 %. Events occurred at a median of 391 (inter-quartile range 376) days after the final GEP test. The GEP variability AUC ROC for the prediction of a composite event was 0.72 (95 % CI 0.6-0.8). The NPV for GEP score variability of 0.6 was 97 % (95 % CI 91.4-100.0); the PPV for GEP score variability of 1.5 was 35.4 % (95 % CI 13.5-75.8). CONCLUSION In heart transplant recipients, a GEP score variability may be used to predict the probability that a composite event will occur within 3 years after the last GEP score. TRIAL REGISTRATION Clinicaltrials.gov identifier NCT00761787.

Warning Against Faking: Boon or Bane? Effects on Test Score Validity

Previous research found personality test scores to be inflated on average among individuals who were motivated to present themselves in a desirable fashion in high stakes situations, such as during the employee selection process. One apparently effective way to reduce the undesirable test score inflation in such situations was to warn participants against faking. This research set out to investigate whether warning against faking would indeed affect personality test scores in the theoretically expected fashion. Contrary to expectations, the results did not support the hypothesized causal chain. Results across three studies show that while a warning may lower test scores in participants motivated to respond desirably (i.e., to fake), the effect of warning on test scores was not fully mediated by: a reduction in motivation to do well and self-reports of exaggerated responses in the personality test. Theoretical and practical implications are discussed.

The Validity of the Brief Michigan Alcohol Screening Test (bMAST) as a Problem Drinking Severity Measure

Objective: The Brief Michigan Alcoholism Screening Test (bMAST) is a 10-item test derived from the 25-item Michigan Alcoholism Screening Test (MAST). It is widely used in the assessment of alcohol dependence. In the absence of previous validation studies, the principal aim of this study was to assess the validity and reliability of the bMAST as a measure of the severity of problem drinking. Method: There were 6,594 patients (4,854 men, 1,740 women) who had been referred for alcohol-use disorders to a hospital alcohol and drug service who voluntarily participated in this study. Results: An exploratory factor analysis defined a two-factor solution, consisting of Perception of Current Drinking and Drinking Consequences factors. Structural equation modeling confirmed that the fit of a nine-item, two-factor model was superior to the original one-factor model. Concurrent validity was assessed through simultaneous administration of the Alcohol Use Disorders Identification Test (AUDIT) and associations with alcohol consumption and clinically assessed features of alcohol dependence. The two-factor bMAST model showed moderate correlations with the AUDIT. The two-factor bMAST and AUDIT were similarly associated with quantity of alcohol consumption and clinically assessed dependence severity features. No differences were observed between the existing weighted scoring system and the proposed simple scoring system. Conclusions: In this study, both the existing bMAST total score and the two-factor model identified were as effective as the AUDIT in assessing problem drinking severity. There are additional advantages of employing the two-factor bMAST in the assessment and treatment planning of patients seeking treatment for alcohol-use disorders. (J. Stud. Alcohol Drugs 68: 771-779,2007)

Corneal confocal microscopy : a novel noninvasive test to diagnose and stratify the severity of human diabetic neuropathy

OBJECTIVE: The accurate quantification of human diabetic neuropathy is important to define at-risk patients, anticipate deterioration, and assess new therapies. ---------- RESEARCH DESIGN AND METHODS: A total of 101 diabetic patients and 17 age-matched control subjects underwent neurological evaluation, neurophysiology tests, quantitative sensory testing, and evaluation of corneal sensation and corneal nerve morphology using corneal confocal microscopy (CCM). ---------- RESULTS: Corneal sensation decreased significantly (P = 0.0001) with increasing neuropathic severity and correlated with the neuropathy disability score (NDS) (r = 0.441, P < 0.0001). Corneal nerve fiber density (NFD) (P < 0.0001), nerve fiber length (NFL), (P < 0.0001), and nerve branch density (NBD) (P < 0.0001) decreased significantly with increasing neuropathic severity and correlated with NDS (NFD r = −0.475, P < 0.0001; NBD r = −0.511, P < 0.0001; and NFL r = −0.581, P < 0.0001). NBD and NFL demonstrated a significant and progressive reduction with worsening heat pain thresholds (P = 0.01). Receiver operating characteristic curve analysis for the diagnosis of neuropathy (NDS >3) defined an NFD of <27.8/mm2 with a sensitivity of 0.82 (95% CI 0.68–0.92) and specificity of 0.52 (0.40–0.64) and for detecting patients at risk of foot ulceration (NDS >6) defined a NFD cutoff of <20.8/mm2 with a sensitivity of 0.71 (0.42–0.92) and specificity of 0.64 (0.54–0.74). ---------- CONCLUSIONS: CCM is a noninvasive clinical technique that may be used to detect early nerve damage and stratify diabetic patients with increasing neuropathic severity. Established diabetic neuropathy leads to pain and foot ulceration. Detecting neuropathy early may allow intervention with treatments to slow or reverse this condition (1). Recent studies suggested that small unmyelinated C-fibers are damaged early in diabetic neuropathy (2–4) but can only be detected using invasive procedures such as sural nerve biopsy (4,5) or skin-punch biopsy (6–8). Our studies have shown that corneal confocal microscopy (CCM) can identify early small nerve fiber damage and accurately quantify the severity of diabetic neuropathy (9–11). We have also shown that CCM relates to intraepidermal nerve fiber loss (12) and a reduction in corneal sensitivity (13) and detects early nerve fiber regeneration after pancreas transplantation (14). Recently we have also shown that CCM detects nerve fiber damage in patients with Fabry disease (15) and idiopathic small fiber neuropathy (16) when results of electrophysiology tests and quantitative sensory testing (QST) are normal. In this study we assessed corneal sensitivity and corneal nerve morphology using CCM in diabetic patients stratified for the severity of diabetic neuropathy using neurological evaluation, electrophysiology tests, and QST. This enabled us to compare CCM and corneal esthesiometry with established tests of diabetic neuropathy and define their sensitivity and specificity to detect diabetic patients with early neuropathy and those at risk of foot ulceration.

Maintenance test section survey : data collection and analysis

A Maintenance Test Section Survey (MTSS) was conducted as part of a Peer State Review of the Texas Maintenance Program conducted October 5–7, 2010. The purpose of the MTSS was to conduct a field review of 34 highway test sections and obtain participants’ opinions about pavement, roadside, and maintenance conditions. The goal was to cross reference or benchmark TxDOT’s maintenance practices based on practices used by selected peer states. Representatives from six peer states (California, Georgia, Kansas, Missouri, North Carolina, and Washington) were invited to Austin to attend a 3-day Peer State Review of TxDOT Maintenance Practices Workshop and to participate in a field survey of a number of pre-selected one-mile roadway sections. It should be emphasized that the objective of the survey was not to evaluate and grade or score TxDOT’s road network but rather to determine whether the selected roadway sections met acceptable standards of service as perceived by Directors of Maintenance or senior maintenance managers from the peer states...

Blur, eye movements and performance on a driving visual recognition slide test

Purpose Optical blur and ageing are known to affect driving performance but their effects on drivers' eye movements are poorly understood. This study examined the effects of optical blur and age on eye movement patterns and performance on the DriveSafe slide recognition test which is purported to predict fitness to drive. Methods Twenty young (27.1 ± 4.6 years) and 20 older (73.3 ± 5.7 years) visually normal drivers performed the DriveSafe under two visual conditions: best-corrected vision and with +2.00 DS blur. The DriveSafe is a Visual Recognition Slide Test that consists of brief presentations of static, real-world driving scenes containing different road users (pedestrians, bicycles and vehicles). Participants reported the types, relative positions and direction of travel of the road users in each image; the score was the number of correctly reported items (maximum score of 128). Eye movements were recorded while participants performed the DriveSafe test using a Tobii TX300 eye tracking system. Results There was a significant main effect of blur on DriveSafe scores (best-corrected: 114.9 vs blur: 93.2; p < 0.001). There was also a significant age and blur interaction on the DriveSafe scores (p < 0.001) such that the young drivers were more negatively affected by blur than the older drivers (reductions of 22% and 13% respectively; p < 0.001): with best-corrected vision, the young drivers performed better than the older drivers (DriveSafe scores: 118.4 vs 111.5; p = 0.001), while with blur, the young drivers performed worse than the older drivers (88.6 vs 95.9; p = 0.009). For the eye movement patterns, blur significantly reduced the number of fixations on road users (best-corrected: 5.1 vs blur: 4.5; p < 0.001), fixation duration on road users (2.0 s vs 1.8 s; p < 0.001) and saccade amplitudes (7.4° vs 6.7°; p < 0.001). A main effect of age on eye movements was also found where older drivers made smaller saccades than the young drivers (6.7° vs 7.4°; p < 0.001). Conclusions Blur reduced DriveSafe scores for both age groups and this effect was greater for the young drivers. The decrease in number of fixations and fixation duration on road users, as well as the reduction in saccade amplitudes under the blurred condition, highlight the difficulty experienced in performing the task in the presence of optical blur, which suggests that uncorrected refractive errors may have a detrimental impact on aspects of driving performance.

Palliative care problem severity score: Reliability and acceptability in a national study

Background The Palliative Care Problem Severity Score is a clinician-rated tool to assess problem severity in four palliative care domains (pain, other symptoms, psychological/spiritual, family/carer problems) using a 4-point categorical scale (absent, mild, moderate, severe). Aim To test the reliability and acceptability of the Palliative Care Problem Severity Score. Design: Multi-centre, cross-sectional study involving pairs of clinicians independently rating problem severity using the tool. Setting/participants Clinicians from 10 Australian palliative care services: 9 inpatient units and 1 mixed inpatient/community-based service. Results A total of 102 clinicians participated, with almost 600 paired assessments completed for each domain, involving 420 patients. A total of 91% of paired assessments were undertaken within 2 h. Strength of agreement for three of the four domains was moderate: pain (Kappa = 0.42, 95% confidence interval = 0.36 to 0.49); psychological/spiritual (Kappa = 0.48, 95% confidence interval = 0.42 to 0.54); family/carer (Kappa = 0.45, 95% confidence interval = 0.40 to 0.52). Strength of agreement for the remaining domain (other symptoms) was fair (Kappa = 0.38, 95% confidence interval = 0.32 to 0.45). Conclusion The Palliative Care Problem Severity Score is an acceptable measure, with moderate reliability across three domains. Variability in inter-rater reliability across sites and participant feedback indicate that ongoing education is required to ensure that clinicians understand the purpose of the tool and each of its domains. Raters familiar with the patient they were assessing found it easier to assign problem severity, but this did not improve inter-rater reliability.

Development and psychometric properties of the DCDDaily: a new test for clinical assessment of capacity in activities of daily living in children with developmental coordination disorder

Objective To develop the DCDDaily, an instrument for objective and standardized clinical assessment of capacity in activities of daily living (ADL) in children with developmental coordination disorder (DCD), and to investigate its usability, reliability, and validity. Subjects Five to eight-year-old children with and without DCD. Main measures The DCDDaily was developed based on thorough review of the literature and extensive expert involvement. To investigate the usability (assessment time and feasibility), reliability (internal consistency and repeatability), and validity (concurrent and discriminant validity) of the DCDDaily, children were assessed with the DCDDaily and the Movement Assessment Battery for Children-2 Test, and their parents filled in the Movement Assessment Battery for Children-2 Checklist and Developmental Coordination Disorder Questionnaire. Results 459 children were assessed (DCD group, n = 55; normative reference group, n = 404). Assessment was possible within 30 minutes and in any clinical setting. For internal consistency, Cronbach’s α = 0.83. Intraclass correlation = 0.87 for test–retest reliability and 0.89 for inter-rater reliability. Concurrent correlations with Movement Assessment Battery for Children-2 Test and questionnaires were ρ = −0.494, 0.239, and −0.284, p < 0.001. Discriminant validity measures showed significantly worse performance in the DCD group than in the control group (mean (SD) score 33 (5.6) versus 26 (4.3), p < 0.001). The area under curve characteristic = 0.872, sensitivity and specificity were 80%. Conclusions The DCDDaily is a valid and reliable instrument for clinical assessment of capacity in ADL, that is feasible for use in clinical practice.

Tradução, adaptação cultural e validação do Nonarthritic Hip Score para o Brasil

A avaliação da qualidade de vida tem sido cada vez mais utilizada pelos profissionais da área de saúde para mensurar o impacto de doenças na vida dos pacientes, bem como para avaliar os resultados dos tratamentos realizados. O crescente interesse por protocolos de pesquisa clínica em doenças não degenerativas do quadril tem encontrado muitos obstáculos na avaliação objetiva de seus resultados, principalmente nos estudos de observação de novas intervenções terapêuticas, como a artroscopia. O Nonarthritic Hip Score (NAHS) é um instrumento de avaliação clínica, desenvolvido originalmente em inglês, cujo objetivo é avaliar a função da articulação do quadril em pacientes jovens e fisicamente ativos. O objetivo desse estudo foi traduzir esse instrumento para a língua portuguesa, adaptá-lo para a cultura brasileira e validá-lo para que possa ser utilizado na avaliação de qualidade de vida de pacientes brasileiros com dor no quadril, sem doença degenerativa. A metodologia utilizada é a sugerida por Guillemin et al. (1993) e revisado por Beaton et al. (2000), que propuseram um conjunto de instruções padronizadas para adaptação cultural de instrumentos de qualidade de vida, incluindo cinco etapas: tradução, tradução de volta, revisão pelo comitê, pré-teste e teste, com reavaliação dos pesos dos escores, se relevante. A versão de consenso foi aplicada em 30 indivíduos. As questões sobre atividades esportivas e tarefas domésticas foram modificadas, para melhor adaptação à cultura brasileira. A versão brasileira do Nonarthritic Hip Score (NAHS-Brasil) foi respondida por 64 pacientes com dor no quadril, a fim de avaliar as propriedades de medida do instrumento: reprodutibilidade, consistência interna e validade. A reprodutibilidade foi 0,9, mostrando uma forte correlação; a consistência interna mostrou correlação entre 0,8 e 0,9, considerada boa e excelente; a validade foi considerada respectivamente boa e excelente; a correlação entre NAHS-Brasil e WOMAC foi 0,9; e a correlação entre o NAHS-Brasil e Questionário Algofuncional de Lequesne foi 0,79. O Nonarthritic Hip Score foi traduzido para a língua portuguesa e adaptado à cultura brasileira, de acordo com o conjunto de instruções padronizadas para adaptação cultural de instrumentos de qualidade de vida. Sua reprodutibilidade, consistência interna e validade foram também demonstradas.

Validity of a 1 minute walk test for children with cerebral palsy.

The concurrent validity of a 1 minute walk test at a child's maximum walking speed was assessed in children with bilateral spastic cerebral palsy (BSCP). The distance covered during the 1 minute walk test was compared with the children's gross motor function as assessed by the Gross Motor Function Measure (GMFM). Twenty-four male and 10 female children with CP (mean age 11y, range 4 to 16y) participated in the study. Gross Motor Function Classification System (GMFCS) levels were; level I (n=3), level II (n=17), level III (n=10), and level IV (n=4). Participants had clinical diagnoses of symmetrical diplegia (n=19), asymmetrical diplegia (n=14), and quadriplegia (n=1). Results showed a significant correlation between GMFM score and the distance covered during the 1 minute walk (r=0.92; p<0.001). There was also a significant decrease in the distance walked with increasing GMFCS level (p<0.001). We concluded that the 1 minute walk test is a valid measure for assessing functional ability in children with ambulatory BSCP. Its cost-effectiveness and user friendliness make it a potentially useful tool in the clinical setting. Further study needs to address its reliability and ability to detect change over time.

LD Score regression distinguishes confounding from polygenicity in genome-wide association studies

Both polygenicity (many small genetic effects) and confounding biases, such as cryptic relatedness and population stratification, can yield an inflated distribution of test statistics in genome-wide association studies (GWAS). However, current methods cannot distinguish between inflation from a true polygenic signal and bias. We have developed an approach, LD Score regression, that quantifies the contribution of each by examining the relationship between test statistics and linkage disequilibrium (LD). The LD Score regression intercept can be used to estimate a more powerful and accurate correction factor than genomic control. We find strong evidence that polygenicity accounts for the majority of the inflation in test statistics in many GWAS of large sample size.