Can prospective usability evaluation predict data errors?

Increasing amounts of clinical research data are collected by manual data entry into electronic source systems and directly from research subjects. For this manual entered source data, common methods of data cleaning such as post-entry identification and resolution of discrepancies and double data entry are not feasible. However data accuracy rates achieved without these mechanisms may be higher than desired for a particular research use. We evaluated a heuristic usability method for utility as a tool to independently and prospectively identify data collection form questions associated with data errors. The method evaluated had a promising sensitivity of 64% and a specificity of 67%. The method was used as described in the literature for usability with no further adaptations or specialization for predicting data errors. We conclude that usability evaluation methodology should be further investigated for use in data quality assurance.

The utility of rapid participatory appraisal for community assessment, diagnosis, and policy development

The selection of a model to guide the understanding and resolution of community problems is an important issue relating to the foundation of public health practice: assessment, policy development, and assurance. Many assessment models produce a diagnosis of community weaknesses, but fail to promote planning and interventions. Rapid Participatory Appraisal (RPA) is a participatory action research model which regards assessment as the first step in the problem solving process, and claims to achieve assessment and policy development within limited resources of time and money. Literature documenting the fulfillment of these claims, and thereby supporting the utility of the model, is relatively sparse and difficult to obtain. Very few articles discuss the changes resulting from RPA assessments in urban areas, and those that do describe studies conducted outside the U.S.A. ^ This study examines the utility of the RPA model and its underlying theories: systems theory, grounded theory, and principles of participatory change, as illustrated by the case study of a community assessment conducted for the Texas Diabetes Institute (TDI), San Antonio, Texas, and subsequent outcomes. Diabetes has a high prevalence and is a major issue in San Antonio. Faculty and students conducted the assessment by informal collaboration between two nursing and public health assessment courses, providing practical student experiences. The study area was large, and the flexibility of the model tested by its use in contiguous sub-regions, reanalyzing aggregated results for the study area. Official TDI reports, and a mail survey of agency employees, described policy development resulting from community diagnoses revealed by the assessment. ^ The RPA model met the criteria for utility from the perspectives of merit, worth, efficiency, and effectiveness. The RPA model best met the agencies' criteria (merit), met the data needs of TDI in this particular situation (worth), provided valid results within budget, time, and personnel constraints (efficiency), and stimulated policy development by TDI (effectiveness). ^ The RPA model appears to have utility for community assessment, diagnosis, and policy development in circumstances similar to the TDI diabetes study. ^

Measuring health state preferences for hemophilia: Development of a disease-specific utility instrument

Research has shown that disease-specific health related quality of life (HRQoL) instruments are more responsive than generic instruments to particular disease conditions. However, only a few studies have used disease-specific instruments to measure HRQoL in hemophilia. The goal of this project was to develop a disease-specific utility instrument that measures patient preferences for various hemophilia health states. The visual analog scale (VAS), a ranking method, and the standard gamble (SG), a choice-based method incorporating risk, were used to measure patient preferences. Study participants (n = 128) were recruited from the UT/Gulf States Hemophilia and Thrombophilia Center and stratified by age: 0–18 years and 19+. ^ Test retest reliability was demonstrated for both VAS and SG instruments: overall within-subject correlation coefficients were 0.91 and 0.79, respectively. Results showed statistically significant differences in responses between pediatric and adult participants when using the SG (p = .045). However, no significant differences were shown between these groups when using the VAS (p = .636). When responses to VAS and SG instruments were compared, statistically significant differences in both pediatric (p < .0001) and adult (p < .0001) groups were observed. Data from this study also demonstrated that persons with hemophilia with varying severity of disease, as well as those who were HIV infected, were able to evaluate a range of health states for hemophilia. This has important implications for the study of quality of life in hemophilia and the development of disease-specific HRQoL instruments. ^ The utility measures obtained from this study can be applied in economic evaluations that analyze the cost/utility of alternative hemophilia treatments. Results derived from the SG indicate that age can influence patients' preferences regarding their state of health. This may have implications for considering treatment options based on the mean age of the population under consideration. Although both instruments independently demonstrated reliability and validity, results indicate that the two measures may not be interchangeable. ^

Pediatric asthma incidence in Texas and associations with ambient ozone levels in an urban area an analysis using Medicaid claims data

Few recent estimates of childhood asthma incidence exist in the literature, although the importance of incidence surveillance for understanding asthma risk factors has been recognized. Asthma prevalence, morbidity and mortality reports have repeatedly shown that low-income children are disproportionately impacted by the disease. The aim of this study was to demonstrate the utility of Medicaid claims data for providing statewide estimates of asthma incidence. Medicaid Analytic Extract (MAX) data for Texas children ages 0-17 enrolled in Medicaid between 2004 and 2007 were used to estimate incidence overall and by age group, gender, race and county of residence. A 13+ month period of continuous enrollment was required in order to distinguish incident from prevalent cases identified in the claims data. Age-adjusted incidence of asthma was 4.26/100 person-years during 2005-2007, higher than reported in other populations. Incidence rates decreased with age, were higher for males than females, differed by race, and tended to be higher in rural than urban areas. With this study, we were able to demonstrate the utility of MAX data for estimating asthma incidence, and create a dataset of incident cases to use in further analysis. ^ In subsequent analyses, we investigated a possible association between ambient air pollutants and incident asthma among Medicaid-enrolled children in Harris County Texas between 2005 and 2007. This population is at high risk for asthma, and living in an area with historically poor air quality. We used a time-stratified case-crossover design and conditional logistic regression to calculate odds ratios, adjusted for weather variables and aeroallergens, to assess the effect of increases in ozone, NO2 and PM2.5 concentrations on risk of developing asthma. Our results show that a 10 ppb increase in ozone was significantly associated with asthma during the warm season (May-October), with the strongest effect seen when a 6-day cumulative lag period was used to compute the exposure metric (OR=1.05, 95% CI, 1.02–1.08). Similar results were seen for NO2 and PM 2.5 (OR=1.07, 95% CI, 1.03–1.11 and OR=1.12, 95% CI, 1.03–1.22, respectively). PM2.5 also had significant effects in the cold season (November-April), 5-day cumulative lag: OR=1.11, 95% CI, 1.00–1.22. When compared with children in the lowest quartile of O3 exposure, the risk for children in the highest quartile was 20% higher. This study indicates that these pollutants are associated with newly-diagnosed childhood asthma in this low-income urban population, particularly during the summer months. ^