Quantifying data quality for clinical trials using electronic data capture.

BACKGROUND: Historically, only partial assessments of data quality have been performed in clinical trials, for which the most common method of measuring database error rates has been to compare the case report form (CRF) to database entries and count discrepancies. Importantly, errors arising from medical record abstraction and transcription are rarely evaluated as part of such quality assessments. Electronic Data Capture (EDC) technology has had a further impact, as paper CRFs typically leveraged for quality measurement are not used in EDC processes. METHODS AND PRINCIPAL FINDINGS: The National Institute on Drug Abuse Treatment Clinical Trials Network has developed, implemented, and evaluated methodology for holistically assessing data quality on EDC trials. We characterize the average source-to-database error rate (14.3 errors per 10,000 fields) for the first year of use of the new evaluation method. This error rate was significantly lower than the average of published error rates for source-to-database audits, and was similar to CRF-to-database error rates reported in the published literature. We attribute this largely to an absence of medical record abstraction on the trials we examined, and to an outpatient setting characterized by less acute patient conditions. CONCLUSIONS: Historically, medical record abstraction is the most significant source of error by an order of magnitude, and should be measured and managed during the course of clinical trials. Source-to-database error rates are highly dependent on the amount of structured data collection in the clinical setting and on the complexity of the medical record, dependencies that should be considered when developing data quality benchmarks.

Missing data in randomized clinical trials for weight loss: scope of the problem, state of the field, and performance of statistical methods.

BACKGROUND: Dropouts and missing data are nearly-ubiquitous in obesity randomized controlled trails, threatening validity and generalizability of conclusions. Herein, we meta-analytically evaluate the extent of missing data, the frequency with which various analytic methods are employed to accommodate dropouts, and the performance of multiple statistical methods. METHODOLOGY/PRINCIPAL FINDINGS: We searched PubMed and Cochrane databases (2000-2006) for articles published in English and manually searched bibliographic references. Articles of pharmaceutical randomized controlled trials with weight loss or weight gain prevention as major endpoints were included. Two authors independently reviewed each publication for inclusion. 121 articles met the inclusion criteria. Two authors independently extracted treatment, sample size, drop-out rates, study duration, and statistical method used to handle missing data from all articles and resolved disagreements by consensus. In the meta-analysis, drop-out rates were substantial with the survival (non-dropout) rates being approximated by an exponential decay curve (e(-lambdat)) where lambda was estimated to be .0088 (95% bootstrap confidence interval: .0076 to .0100) and t represents time in weeks. The estimated drop-out rate at 1 year was 37%. Most studies used last observation carried forward as the primary analytic method to handle missing data. We also obtained 12 raw obesity randomized controlled trial datasets for empirical analyses. Analyses of raw randomized controlled trial data suggested that both mixed models and multiple imputation performed well, but that multiple imputation may be more robust when missing data are extensive. CONCLUSION/SIGNIFICANCE: Our analysis offers an equation for predictions of dropout rates useful for future study planning. Our raw data analyses suggests that multiple imputation is better than other methods for handling missing data in obesity randomized controlled trials, followed closely by mixed models. We suggest these methods supplant last observation carried forward as the primary method of analysis.

Application description and policy model in collaborative environment for sharing of information on epidemiological and clinical research data sets.

BACKGROUND: Sharing of epidemiological and clinical data sets among researchers is poor at best, in detriment of science and community at large. The purpose of this paper is therefore to (1) describe a novel Web application designed to share information on study data sets focusing on epidemiological clinical research in a collaborative environment and (2) create a policy model placing this collaborative environment into the current scientific social context. METHODOLOGY: The Database of Databases application was developed based on feedback from epidemiologists and clinical researchers requiring a Web-based platform that would allow for sharing of information about epidemiological and clinical study data sets in a collaborative environment. This platform should ensure that researchers can modify the information. A Model-based predictions of number of publications and funding resulting from combinations of different policy implementation strategies (for metadata and data sharing) were generated using System Dynamics modeling. PRINCIPAL FINDINGS: The application allows researchers to easily upload information about clinical study data sets, which is searchable and modifiable by other users in a wiki environment. All modifications are filtered by the database principal investigator in order to maintain quality control. The application has been extensively tested and currently contains 130 clinical study data sets from the United States, Australia, China and Singapore. Model results indicated that any policy implementation would be better than the current strategy, that metadata sharing is better than data-sharing, and that combined policies achieve the best results in terms of publications. CONCLUSIONS: Based on our empirical observations and resulting model, the social network environment surrounding the application can assist epidemiologists and clinical researchers contribute and search for metadata in a collaborative environment, thus potentially facilitating collaboration efforts among research communities distributed around the globe.

Qualitative analysis of the interdisciplinary interaction between data analysis specialists and novice clinical researchers.

BACKGROUND: The inherent complexity of statistical methods and clinical phenomena compel researchers with diverse domains of expertise to work in interdisciplinary teams, where none of them have a complete knowledge in their counterpart's field. As a result, knowledge exchange may often be characterized by miscommunication leading to misinterpretation, ultimately resulting in errors in research and even clinical practice. Though communication has a central role in interdisciplinary collaboration and since miscommunication can have a negative impact on research processes, to the best of our knowledge, no study has yet explored how data analysis specialists and clinical researchers communicate over time. METHODS/PRINCIPAL FINDINGS: We conducted qualitative analysis of encounters between clinical researchers and data analysis specialists (epidemiologist, clinical epidemiologist, and data mining specialist). These encounters were recorded and systematically analyzed using a grounded theory methodology for extraction of emerging themes, followed by data triangulation and analysis of negative cases for validation. A policy analysis was then performed using a system dynamics methodology looking for potential interventions to improve this process. Four major emerging themes were found. Definitions using lay language were frequently employed as a way to bridge the language gap between the specialties. Thought experiments presented a series of "what if" situations that helped clarify how the method or information from the other field would behave, if exposed to alternative situations, ultimately aiding in explaining their main objective. Metaphors and analogies were used to translate concepts across fields, from the unfamiliar to the familiar. Prolepsis was used to anticipate study outcomes, thus helping specialists understand the current context based on an understanding of their final goal. CONCLUSION/SIGNIFICANCE: The communication between clinical researchers and data analysis specialists presents multiple challenges that can lead to errors.

What leads Indians to participate in clinical trials? A meta-analysis of qualitative studies.

BACKGROUND: With the globalization of clinical trials, large developing nations have substantially increased their participation in multi-site studies. This participation has raised ethical concerns, among them the fear that local customs, habits and culture are not respected while asking potential participants to take part in study. This knowledge gap is particularly noticeable among Indian subjects, since despite the large number of participants, little is known regarding what factors affect their willingness to participate in clinical trials. METHODS: We conducted a meta-analysis of all studies evaluating the factors and barriers, from the perspective of potential Indian participants, contributing to their participation in clinical trials. We searched both international as well as Indian-specific bibliographic databases, including Pubmed, Cochrane, Openjgate, MedInd, Scirus and Medknow, also performing hand searches and communicating with authors to obtain additional references. We enrolled studies dealing exclusively with the participation of Indians in clinical trials. Data extraction was conducted by three researchers, with disagreement being resolved by consensus. RESULTS: Six qualitative studies and one survey were found evaluating the main themes affecting the participation of Indian subjects. Themes included Personal health benefits, Altruism, Trust in physicians, Source of extra income, Detailed knowledge, Methods for motivating participants as factors favoring, while Mistrust on trial organizations, Concerns about efficacy and safety of trials, Psychological reasons, Trial burden, Loss of confidentiality, Dependency issues, Language as the barriers. CONCLUSION: We identified factors that facilitated and barriers that have negative implications on trial participation decisions in Indian subjects. Due consideration and weightage should be assigned to these factors while planning future trials in India.

Safety and immunogenicity of a replication-defective adenovirus type 5 HIV vaccine in Ad5-seronegative persons: a randomized clinical trial (HVTN 054).

BACKGROUND: Individuals without prior immunity to a vaccine vector may be more sensitive to reactions following injection, but may also show optimal immune responses to vaccine antigens. To assess safety and maximal tolerated dose of an adenoviral vaccine vector in volunteers without prior immunity, we evaluated a recombinant replication-defective adenovirus type 5 (rAd5) vaccine expressing HIV-1 Gag, Pol, and multiclade Env proteins, VRC-HIVADV014-00-VP, in a randomized, double-blind, dose-escalation, multicenter trial (HVTN study 054) in HIV-1-seronegative participants without detectable neutralizing antibodies (nAb) to the vector. As secondary outcomes, we also assessed T-cell and antibody responses. METHODOLOGY/PRINCIPAL FINDINGS: Volunteers received one dose of vaccine at either 10(10) or 10(11) adenovector particle units, or placebo. T-cell responses were measured against pools of global potential T-cell epitope peptides. HIV-1 binding and neutralizing antibodies were assessed. Systemic reactogenicity was greater at the higher dose, but the vaccine was well tolerated at both doses. Although no HIV infections occurred, commercial diagnostic assays were positive in 87% of vaccinees one year after vaccination. More than 85% of vaccinees developed HIV-1-specific T-cell responses detected by IFN-γ ELISpot and ICS assays at day 28. T-cell responses were: CD8-biased; evenly distributed across the three HIV-1 antigens; not substantially increased at the higher dose; and detected at similar frequencies one year following injection. The vaccine induced binding antibodies against at least one HIV-1 Env antigen in all recipients. CONCLUSIONS/SIGNIFICANCE: This vaccine appeared safe and was highly immunogenic following a single dose in human volunteers without prior nAb against the vector. TRIAL REGISTRATION: ClinicalTrials.gov NCT00119873.

Standardizing clinical trials workflow representation in UML for international site comparison.

BACKGROUND: With the globalization of clinical trials, a growing emphasis has been placed on the standardization of the workflow in order to ensure the reproducibility and reliability of the overall trial. Despite the importance of workflow evaluation, to our knowledge no previous studies have attempted to adapt existing modeling languages to standardize the representation of clinical trials. Unified Modeling Language (UML) is a computational language that can be used to model operational workflow, and a UML profile can be developed to standardize UML models within a given domain. This paper's objective is to develop a UML profile to extend the UML Activity Diagram schema into the clinical trials domain, defining a standard representation for clinical trial workflow diagrams in UML. METHODS: Two Brazilian clinical trial sites in rheumatology and oncology were examined to model their workflow and collect time-motion data. UML modeling was conducted in Eclipse, and a UML profile was developed to incorporate information used in discrete event simulation software. RESULTS: Ethnographic observation revealed bottlenecks in workflow: these included tasks requiring full commitment of CRCs, transferring notes from paper to computers, deviations from standard operating procedures, and conflicts between different IT systems. Time-motion analysis revealed that nurses' activities took up the most time in the workflow and contained a high frequency of shorter duration activities. Administrative assistants performed more activities near the beginning and end of the workflow. Overall, clinical trial tasks had a greater frequency than clinic routines or other general activities. CONCLUSIONS: This paper describes a method for modeling clinical trial workflow in UML and standardizing these workflow diagrams through a UML profile. In the increasingly global environment of clinical trials, the standardization of workflow modeling is a necessary precursor to conducting a comparative analysis of international clinical trials workflows.

So different, yet so similar: meta-analysis and policy modeling of willingness to participate in clinical trials among Brazilians and Indians.

BACKGROUND: With the global expansion of clinical trials and the expectations of the rise of the emerging economies known as BRICs (Brazil, Russia, India and China), the understanding of factors that affect the willingness to participate in clinical trials of patients from those countries assumes a central role in the future of health research. METHODS: We conducted a systematic review and meta-analysis (SRMA) of willingness to participate in clinical trials among Brazilian patients and then we compared it with Indian patients (with results of another SRMA previously conducted by our group) through a system dynamics model. RESULTS: Five studies were included in the SRMA of Brazilian patients. Our main findings are 1) the major motivation for Brazilian patients to participate in clinical trials is altruism, 2) monetary reimbursement is the least important factor motivating Brazilian patients, 3) the major barrier for Brazilian patients to not participate in clinical trials is the fear of side effects, and 4) Brazilian patients are more likely willing to participate in clinical trials than Indians. CONCLUSION: Our study provides important insights for investigators and sponsors for planning trials in Brazil (and India) in the future. Ignoring these results may lead to unnecessary fund/time spending. More studies are needed to validate our results and for better understanding of this poorly studied theme.

Inhibition of adaptive immune responses leads to a fatal clinical outcome in SIV-infected pigtailed macaques but not vervet African green monkeys.

African green monkeys (AGM) and other natural hosts for simian immunodeficiency virus (SIV) do not develop an AIDS-like disease following SIV infection. To evaluate differences in the role of SIV-specific adaptive immune responses between natural and nonnatural hosts, we used SIV(agmVer90) to infect vervet AGM and pigtailed macaques (PTM). This infection results in robust viral replication in both vervet AGM and pigtailed macaques (PTM) but only induces AIDS in the latter species. We delayed the development of adaptive immune responses through combined administration of anti-CD8 and anti-CD20 lymphocyte-depleting antibodies during primary infection of PTM (n = 4) and AGM (n = 4), and compared these animals to historical controls infected with the same virus. Lymphocyte depletion resulted in a 1-log increase in primary viremia and a 4-log increase in post-acute viremia in PTM. Three of the four PTM had to be euthanized within 6 weeks of inoculation due to massive CMV reactivation and disease. In contrast, all four lymphocyte-depleted AGM remained healthy. The lymphocyte-depleted AGM showed only a trend toward a prolongation in peak viremia but the groups were indistinguishable during chronic infection. These data show that adaptive immune responses are critical for controlling disease progression in pathogenic SIV infection in PTM. However, the maintenance of a disease-free course of SIV infection in AGM likely depends on a number of mechanisms including non-adaptive immune mechanisms.

A randomized clinical trial of a coping improvement group intervention for HIV-infected older adults.

This research tested if a 12-session coping improvement group intervention (n = 104) reduced depressive symptoms in HIV-infected older adults compared to an interpersonal support group intervention (n = 105) and an individual therapy upon request (ITUR) control condition (n = 86). Participants were 295 HIV-infected men and women 50-plus years of age living in New York City, Cincinnati, OH, and Columbus, OH. Using A-CASI assessment methodology, participants provided data on their depressive symptoms using the Geriatric Depression Screening Scale (GDS) at pre-intervention, post-intervention, and 4- and 8-month follow-up. Whether conducted with all participants (N = 295) or only a subset of participants diagnosed with mild, moderate, or severe depressive symptoms (N = 171), mixed models analyses of repeated measures found that both coping improvement and interpersonal support group intervention participants reported fewer depressive symptoms than ITUR controls at post-intervention, 4-month follow-up, and 8-month follow-up. The effect sizes of the differences between the two active interventions and the control group were greater when outcome analyses were limited to those participants with mild, moderate, or severe depressive symptoms. At no assessment period did coping improvement and interpersonal support group intervention participants differ in depressive symptoms.

Economic return of clinical trials performed under the pediatric exclusivity program.

CONTEXT: In 1997, Congress authorized the US Food and Drug Administration (FDA) to grant 6-month extensions of marketing rights through the Pediatric Exclusivity Program if industry sponsors complete FDA-requested pediatric trials. The program has been praised for creating incentives for studies in children and has been criticized as a "windfall" to the innovator drug industry. This critique has been a substantial part of congressional debate on the program, which is due to expire in 2007. OBJECTIVE: To quantify the economic return to industry for completing pediatric exclusivity trials. DESIGN AND SETTING: A cohort study of programs conducted for pediatric exclusivity. Nine drugs that were granted pediatric exclusivity were selected. From the final study reports submitted to the FDA (2002-2004), key elements of the clinical trial design and study operations were obtained, and the cost of performing each study was estimated and converted into estimates of after-tax cash outflows. Three-year market sales were obtained and converted into estimates of after-tax cash inflows based on 6 months of additional market protection. Net economic return (cash inflows minus outflows) and net return-to-costs ratio (net economic return divided by cash outflows) for each product were then calculated. MAIN OUTCOME MEASURES: Net economic return and net return-to-cost ratio. RESULTS: The indications studied reflect a broad representation of the program: asthma, tumors, attention-deficit/hyperactivity disorder, hypertension, depression/generalized anxiety disorder, diabetes mellitus, gastroesophageal reflux, bacterial infection, and bone mineralization. The distribution of net economic return for 6 months of exclusivity varied substantially among products (net economic return ranged from -$8.9 million to $507.9 million and net return-to-cost ratio ranged from -0.68 to 73.63). CONCLUSIONS: The economic return for pediatric exclusivity is variable. As an incentive to complete much-needed clinical trials in children, pediatric exclusivity can generate lucrative returns or produce more modest returns on investment.

Comparative effectiveness studies to improve clinical outcomes in end stage renal disease: the DEcIDE patient outcomes in end stage renal disease study.

BACKGROUND: Evidence is lacking to inform providers' and patients' decisions about many common treatment strategies for patients with end stage renal disease (ESRD). METHODS/DESIGN: The DEcIDE Patient Outcomes in ESRD Study is funded by the United States (US) Agency for Health Care Research and Quality to study the comparative effectiveness of: 1) antihypertensive therapies, 2) early versus later initiation of dialysis, and 3) intravenous iron therapies on clinical outcomes in patients with ESRD. Ongoing studies utilize four existing, nationally representative cohorts of patients with ESRD, including (1) the Choices for Healthy Outcomes in Caring for ESRD study (1041 incident dialysis patients recruited from October 1995 to June 1999 with complete outcome ascertainment through 2009), (2) the Dialysis Clinic Inc (45,124 incident dialysis patients initiating and receiving their care from 2003-2010 with complete outcome ascertainment through 2010), (3) the United States Renal Data System (333,308 incident dialysis patients from 2006-2009 with complete outcome ascertainment through 2010), and (4) the Cleveland Clinic Foundation Chronic Kidney Disease Registry (53,399 patients with chronic kidney disease with outcome ascertainment from 2005 through 2009). We ascertain patient reported outcomes (i.e., health-related quality of life), morbidity, and mortality using clinical and administrative data, and data obtained from national death indices. We use advanced statistical methods (e.g., propensity scoring and marginal structural modeling) to account for potential biases of our study designs. All data are de-identified for analyses. The conduct of studies and dissemination of findings are guided by input from Stakeholders in the ESRD community. DISCUSSION: The DEcIDE Patient Outcomes in ESRD Study will provide needed evidence regarding the effectiveness of common treatments employed for dialysis patients. Carefully planned dissemination strategies to the ESRD community will enhance studies' impact on clinical care and patients' outcomes.

Ethics Standards (HRPP) and Public Partnership (PARTAKE) to Address Clinical Research Concerns in India: Moving Toward Ethical, Responsible, Culturally Sensitive, and Community-Engaging Clinical Research.

Like other emerging economies, India's quest for independent, evidence-based, and affordable healthcare has led to robust and promising growth in the clinical research sector, with a compound annual growth rate (CAGR) of 20.4% between 2005 and 2010. However, while the fundamental drivers and strengths are still strong, the past few years witnessed a declining trend (CAGR -16.7%) amid regulatory concerns, activist protests, and sponsor departure. And although India accounts for 17.5% of the world's population, it currently conducts only 1% of clinical trials. Indian and international experts and public stakeholders gathered for a 2-day conference in June 2013 in New Delhi to discuss the challenges facing clinical research in India and to explore solutions. The main themes discussed were ethical standards, regulatory oversight, and partnerships with public stakeholders. The meeting was a collaboration of AAHRPP (Association for the Accreditation of Human Research Protection Programs)-aimed at establishing responsible and ethical clinical research standards-and PARTAKE (Public Awareness of Research for Therapeutic Advancements through Knowledge and Empowerment)-aimed at informing and engaging the public in clinical research. The present article covers recent clinical research developments in India as well as associated expectations, challenges, and suggestions for future directions. AAHRPP and PARTAKE provide etiologically based solutions to protect, inform, and engage the public and medical research sponsors.