Can prospective usability evaluation predict data errors?

Increasing amounts of clinical research data are collected by manual data entry into electronic source systems and directly from research subjects. For this manual entered source data, common methods of data cleaning such as post-entry identification and resolution of discrepancies and double data entry are not feasible. However data accuracy rates achieved without these mechanisms may be higher than desired for a particular research use. We evaluated a heuristic usability method for utility as a tool to independently and prospectively identify data collection form questions associated with data errors. The method evaluated had a promising sensitivity of 64% and a specificity of 67%. The method was used as described in the literature for usability with no further adaptations or specialization for predicting data errors. We conclude that usability evaluation methodology should be further investigated for use in data quality assurance.

Concordance between medical records and personal interview in ascertaining breast cancer and diabetes status: Analysis of the south Texas women's health project

Accurate ascertainment of risk factors and disease status is vital in public health research for proper classification of research subjects. The two most common ways of obtaining this data is by self-report and review of medical records (MRs). South Texas Women’s Health Project was a case-control study looking at interrelationships between hormones, diet, and body size and breast cancer among Hispanic women 30-79 years of age. History of breast cancer, diabetes mellitus (DM) and use of DM medications was ascertained from a personal interview. At the time of interview, the subject identified her major health care providers and signed the medical records release form, which was sent to the designated providers. The MRs were reviewed to confirm information obtained from the interview.^ Aim of this study was to determine the sensitivity and specificity between MRs and personal interview in diagnosis of breast cancer, DM and DM treatment. We also wanted to assess how successful our low-cost approach was in obtaining pertinent MRs and what factors influenced the quality of MR or interview data. Study sample was 721 women with both self-report and MR data available by June 2007. Overall response rate for MR requests was 74.5%. MRs were 80.9% sensitive and 100% specific in confirming breast cancer status. Prevalence of DM was 22.7% from the interviews and 16% from MRs. MRs did not provide definite information about DM status of 53.6% subjects. Sensitivity and specificity of MRs for DM status was 88.9% and 90.4% respectively. Disagreement on DM status from the two sources was seen in 15.9% subjects. This discordance was more common among older subjects, those who were married and were predominantly Spanish speaking. Income and level of education did not have a statistically significantly association with this disagreement.^ Both self-report and MRs underestimate the prevalence of DM. Relying solely on MRs leads to greater misclassification than relying on self-report data. MRs have good to excellent specificity and thus serve as a good tool to confirm information obtained from self-report. Self-report and MRs should be used in a complementary manner for accurate assessment of DM and breast cancer status.^

Investigator/IRB-led community communication in genetic research: A descriptive feasibility pilot study

This pilot study, conducted in the Houston, TX, area, established a structured dialogue among a university Institutional Review Board, its researchers, and its local community members (i.e. pool of potential research participants) for the purpose of further educating all three parties about genetic research and community concerns related to such research. An IRB-designed educational presentation aimed at assisting potential subjects in making an informed decision to participate in genetic research was provided to four community groups (n=54); this presentation also included a current example of genetic research being conducted in the community as explained by the researcher, and a question-and-answer session designed to assist the IRB and the researcher in understanding the community's concerns about genetic research. Comparisons of pre- and post- presentation community questionnaires indicate that the joint presentation was effective in increasing community knowledge about genetic research, most notably related to the risks and benefits of this research to the individual, as well as the understanding that protections are in place for research participants. While researchers are optimistic about the idea of a collaborative effort with the IRB and the community, the feasibility of such a program and the benefit to the participating researchers remain unclear; additional research is necessary to establish the most effective method of communication for all groups involved, as well as to obtain statistically significant results with regard to race/ethnicity, gender, and education levels of community participants. ^