Using technology to teach technology: design and evaluation of bilingual online physician education about electronic medical records.

The "EMR Tutorial" is designed to be a bilingual online physician education environment about electronic medical records. After iterative assessment and redesign, the tutorial was tested in two groups: U.S. physicians and Mexican medical students. Split-plot ANOVA revealed significantly different pre-test scores in the two groups, significant cognitive gains for the two groups overall, and no significant difference in the gains made by the two groups. Users rated the module positively on a satisfaction questionnaire.

Comparison of clinical knowledge management capabilities of commercially-available and leading internally-developed electronic health records.

BACKGROUND: We have carried out an extensive qualitative research program focused on the barriers and facilitators to successful adoption and use of various features of advanced, state-of-the-art electronic health records (EHRs) within large, academic, teaching facilities with long-standing EHR research and development programs. We have recently begun investigating smaller, community hospitals and out-patient clinics that rely on commercially-available EHRs. We sought to assess whether the current generation of commercially-available EHRs are capable of providing the clinical knowledge management features, functions, tools, and techniques required to deliver and maintain the clinical decision support (CDS) interventions required to support the recently defined "meaningful use" criteria. METHODS: We developed and fielded a 17-question survey to representatives from nine commercially available EHR vendors and four leading internally developed EHRs. The first part of the survey asked basic questions about the vendor's EHR. The second part asked specifically about the CDS-related system tools and capabilities that each vendor provides. The final section asked about clinical content. RESULTS: All of the vendors and institutions have multiple modules capable of providing clinical decision support interventions to clinicians. The majority of the systems were capable of performing almost all of the key knowledge management functions we identified. CONCLUSION: If these well-designed commercially-available systems are coupled with the other key socio-technical concepts required for safe and effective EHR implementation and use, and organizations have access to implementable clinical knowledge, we expect that the transformation of the healthcare enterprise that so many have predicted, is achievable using commercially-available, state-of-the-art EHRs.

Improving follow-up of abnormal cancer screens using electronic health records: trust but verify test result communication.

BACKGROUND: Early detection of colorectal cancer through timely follow-up of positive Fecal Occult Blood Tests (FOBTs) remains a challenge. In our previous work, we found 40% of positive FOBT results eligible for colonoscopy had no documented response by a treating clinician at two weeks despite procedures for electronic result notification. We determined if technical and/or workflow-related aspects of automated communication in the electronic health record could lead to the lack of response. METHODS: Using both qualitative and quantitative methods, we evaluated positive FOBT communication in the electronic health record of a large, urban facility between May 2008 and March 2009. We identified the source of test result communication breakdown, and developed an intervention to fix the problem. Explicit medical record reviews measured timely follow-up (defined as response within 30 days of positive FOBT) pre- and post-intervention. RESULTS: Data from 11 interviews and tracking information from 490 FOBT alerts revealed that the software intended to alert primary care practitioners (PCPs) of positive FOBT results was not configured correctly and over a third of positive FOBTs were not transmitted to PCPs. Upon correction of the technical problem, lack of timely follow-up decreased immediately from 29.9% to 5.4% (p<0.01) and was sustained at month 4 following the intervention. CONCLUSION: Electronic communication of positive FOBT results should be monitored to avoid limiting colorectal cancer screening benefits. Robust quality assurance and oversight systems are needed to achieve this. Our methods may be useful for others seeking to improve follow-up of FOBTs in their systems.

Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?

BACKGROUND: Given the fragmentation of outpatient care, timely follow-up of abnormal diagnostic imaging results remains a challenge. We hypothesized that an electronic medical record (EMR) that facilitates the transmission and availability of critical imaging results through either automated notification (alerting) or direct access to the primary report would eliminate this problem. METHODS: We studied critical imaging alert notifications in the outpatient setting of a tertiary care Department of Veterans Affairs facility from November 2007 to June 2008. Tracking software determined whether the alert was acknowledged (ie, health care practitioner/provider [HCP] opened the message for viewing) within 2 weeks of transmission; acknowledged alerts were considered read. We reviewed medical records and contacted HCPs to determine timely follow-up actions (eg, ordering a follow-up test or consultation) within 4 weeks of transmission. Multivariable logistic regression models accounting for clustering effect by HCPs analyzed predictors for 2 outcomes: lack of acknowledgment and lack of timely follow-up. RESULTS: Of 123 638 studies (including radiographs, computed tomographic scans, ultrasonograms, magnetic resonance images, and mammograms), 1196 images (0.97%) generated alerts; 217 (18.1%) of these were unacknowledged. Alerts had a higher risk of being unacknowledged when the ordering HCPs were trainees (odds ratio [OR], 5.58; 95% confidence interval [CI], 2.86-10.89) and when dual-alert (>1 HCP alerted) as opposed to single-alert communication was used (OR, 2.02; 95% CI, 1.22-3.36). Timely follow-up was lacking in 92 (7.7% of all alerts) and was similar for acknowledged and unacknowledged alerts (7.3% vs 9.7%; P = .22). Risk for lack of timely follow-up was higher with dual-alert communication (OR, 1.99; 95% CI, 1.06-3.48) but lower when additional verbal communication was used by the radiologist (OR, 0.12; 95% CI, 0.04-0.38). Nearly all abnormal results lacking timely follow-up at 4 weeks were eventually found to have measurable clinical impact in terms of further diagnostic testing or treatment. CONCLUSIONS: Critical imaging results may not receive timely follow-up actions even when HCPs receive and read results in an advanced, integrated electronic medical record system. A multidisciplinary approach is needed to improve patient safety in this area.

Estimating the mean and standard deviation of concentration distributions using censored samples

Environmental data sets of pollutant concentrations in air, water, and soil frequently include unquantified sample values reported only as being below the analytical method detection limit. These values, referred to as censored values, should be considered in the estimation of distribution parameters as each represents some value of pollutant concentration between zero and the detection limit. Most of the currently accepted methods for estimating the population parameters of environmental data sets containing censored values rely upon the assumption of an underlying normal (or transformed normal) distribution. This assumption can result in unacceptable levels of error in parameter estimation due to the unbounded left tail of the normal distribution. With the beta distribution, which is bounded by the same range of a distribution of concentrations, $\rm\lbrack0\le x\le1\rbrack,$ parameter estimation errors resulting from improper distribution bounds are avoided. This work developed a method that uses the beta distribution to estimate population parameters from censored environmental data sets and evaluated its performance in comparison to currently accepted methods that rely upon an underlying normal (or transformed normal) distribution. Data sets were generated assuming typical values encountered in environmental pollutant evaluation for mean, standard deviation, and number of variates. For each set of model values, data sets were generated assuming that the data was distributed either normally, lognormally, or according to a beta distribution. For varying levels of censoring, two established methods of parameter estimation, regression on normal ordered statistics, and regression on lognormal ordered statistics, were used to estimate the known mean and standard deviation of each data set. The method developed for this study, employing a beta distribution assumption, was also used to estimate parameters and the relative accuracy of all three methods were compared. For data sets of all three distribution types, and for censoring levels up to 50%, the performance of the new method equaled, if not exceeded, the performance of the two established methods. Because of its robustness in parameter estimation regardless of distribution type or censoring level, the method employing the beta distribution should be considered for full development in estimating parameters for censored environmental data sets. ^

Concordance between medical records and personal interview in ascertaining breast cancer and diabetes status: Analysis of the south Texas women's health project

Accurate ascertainment of risk factors and disease status is vital in public health research for proper classification of research subjects. The two most common ways of obtaining this data is by self-report and review of medical records (MRs). South Texas Women’s Health Project was a case-control study looking at interrelationships between hormones, diet, and body size and breast cancer among Hispanic women 30-79 years of age. History of breast cancer, diabetes mellitus (DM) and use of DM medications was ascertained from a personal interview. At the time of interview, the subject identified her major health care providers and signed the medical records release form, which was sent to the designated providers. The MRs were reviewed to confirm information obtained from the interview.^ Aim of this study was to determine the sensitivity and specificity between MRs and personal interview in diagnosis of breast cancer, DM and DM treatment. We also wanted to assess how successful our low-cost approach was in obtaining pertinent MRs and what factors influenced the quality of MR or interview data. Study sample was 721 women with both self-report and MR data available by June 2007. Overall response rate for MR requests was 74.5%. MRs were 80.9% sensitive and 100% specific in confirming breast cancer status. Prevalence of DM was 22.7% from the interviews and 16% from MRs. MRs did not provide definite information about DM status of 53.6% subjects. Sensitivity and specificity of MRs for DM status was 88.9% and 90.4% respectively. Disagreement on DM status from the two sources was seen in 15.9% subjects. This discordance was more common among older subjects, those who were married and were predominantly Spanish speaking. Income and level of education did not have a statistically significantly association with this disagreement.^ Both self-report and MRs underestimate the prevalence of DM. Relying solely on MRs leads to greater misclassification than relying on self-report data. MRs have good to excellent specificity and thus serve as a good tool to confirm information obtained from self-report. Self-report and MRs should be used in a complementary manner for accurate assessment of DM and breast cancer status.^

A SIMULATION STUDY OF THE SIZE AND POWER OF SOME TWO-SAMPLE TESTS FOR UNEQUALLY CENSORED SURVIVAL DATA (RANK TESTS, LOGRANK, PROPORTIONAL HAZARDS, WILCOXON TESTS, EXPONENTIAL DISTRIBUTION)

Sizes and power of selected two-sample tests of the equality of survival distributions are compared by simulation for small samples from unequally, randomly-censored exponential distributions. The tests investigated include parametric tests (F, Score, Likelihood, Asymptotic), logrank tests (Mantel, Peto-Peto), and Wilcoxon-Type tests (Gehan, Prentice). Equal sized samples, n = 18, 16, 32 with 1000 (size) and 500 (power) simulation trials, are compared for 16 combinations of the censoring proportions 0%, 20%, 40%, and 60%. For n = 8 and 16, the Asymptotic, Peto-Peto, and Wilcoxon tests perform at nominal 5% size expectations, but the F, Score and Mantel tests exceeded 5% size confidence limits for 1/3 of the censoring combinations. For n = 32, all tests showed proper size, with the Peto-Peto test most conservative in the presence of unequal censoring. Powers of all tests are compared for exponential hazard ratios of 1.4 and 2.0. There is little difference in power characteristics of the tests within the classes of tests considered. The Mantel test showed 90% to 95% power efficiency relative to parametric tests. Wilcoxon-type tests have the lowest relative power but are robust to differential censoring patterns. A modified Peto-Peto test shows power comparable to the Mantel test. For n = 32, a specific Weibull-exponential comparison of crossing survival curves suggests that the relative powers of logrank and Wilcoxon-type tests are dependent on the scale parameter of the Weibull distribution. Wilcoxon-type tests appear more powerful than logrank tests in the case of late-crossing and less powerful for early-crossing survival curves. Guidelines for the appropriate selection of two-sample tests are given. ^

COMPREHENSIVE CALCULATION-BASED IMRT QA USING R&V DATA, TREATMENT RECORDS, AND A SECOND TREATMENT PLANNING SYSTEM

Purpose: Traditional patient-specific IMRT QA measurements are labor intensive and consume machine time. Calculation-based IMRT QA methods typically are not comprehensive. We have developed a comprehensive calculation-based IMRT QA method to detect uncertainties introduced by the initial dose calculation, the data transfer through the Record-and-Verify (R&V) system, and various aspects of the physical delivery. Methods: We recomputed the treatment plans in the patient geometry for 48 cases using data from the R&V, and from the delivery unit to calculate the “as-transferred” and “as-delivered” doses respectively. These data were sent to the original TPS to verify transfer and delivery or to a second TPS to verify the original calculation. For each dataset we examined the dose computed from the R&V record (RV) and from the delivery records (Tx), and the dose computed with a second verification TPS (vTPS). Each verification dose was compared to the clinical dose distribution using 3D gamma analysis and by comparison of mean dose and ROI-specific dose levels to target volumes. Plans were also compared to IMRT QA absolute and relative dose measurements. Results: The average 3D gamma passing percentages using 3%-3mm, 2%-2mm, and 1%-1mm criteria for the RV plan were 100.0 (σ=0.0), 100.0 (σ=0.0), and 100.0 (σ=0.1); for the Tx plan they were 100.0 (σ=0.0), 100.0 (σ=0.0), and 99.0 (σ=1.4); and for the vTPS plan they were 99.3 (σ=0.6), 97.2 (σ=1.5), and 79.0 (σ=8.6). When comparing target volume doses in the RV, Tx, and vTPS plans to the clinical plans, the average ratios of ROI mean doses were 0.999 (σ=0.001), 1.001 (σ=0.002), and 0.990 (σ=0.009) and ROI-specific dose levels were 0.999 (σ=0.001), 1.001 (σ=0.002), and 0.980 (σ=0.043), respectively. Comparing the clinical, RV, TR, and vTPS calculated doses to the IMRT QA measurements for all 48 patients, the average ratios for absolute doses were 0.999 (σ=0.013), 0.998 (σ=0.013), 0.999 σ=0.015), and 0.990 (σ=0.012), respectively, and the average 2D gamma(5%-3mm) passing percentages for relative doses for 9 patients was were 99.36 (σ=0.68), 99.50 (σ=0.49), 99.13 (σ=0.84), and 98.76 (σ=1.66), respectively. Conclusions: Together with mechanical and dosimetric QA, our calculation-based IMRT QA method promises to minimize the need for patient-specific QA measurements by identifying outliers in need of further review.

Analyzing left-truncated right-censored data with uncertain onset time with parametric models

Prevalent sampling is an efficient and focused approach to the study of the natural history of disease. Right-censored time-to-event data observed from prospective prevalent cohort studies are often subject to left-truncated sampling. Left-truncated samples are not randomly selected from the population of interest and have a selection bias. Extensive studies have focused on estimating the unbiased distribution given left-truncated samples. However, in many applications, the exact date of disease onset was not observed. For example, in an HIV infection study, the exact HIV infection time is not observable. However, it is known that the HIV infection date occurred between two observable dates. Meeting these challenges motivated our study. We propose parametric models to estimate the unbiased distribution of left-truncated, right-censored time-to-event data with uncertain onset times. We first consider data from a length-biased sampling, a specific case in left-truncated samplings. Then we extend the proposed method to general left-truncated sampling. With a parametric model, we construct the full likelihood, given a biased sample with unobservable onset of disease. The parameters are estimated through the maximization of the constructed likelihood by adjusting the selection bias and unobservable exact onset. Simulations are conducted to evaluate the finite sample performance of the proposed methods. We apply the proposed method to an HIV infection study, estimating the unbiased survival function and covariance coefficients. ^

Description and analysis of variation in dairy product and concomitant nutrient purchases between Hispanic and non -Hispanic households using a panel of shoppers' purchase records

The purpose of this research was development of a method of estimating nutrient availability in populations as approximated by supermarket purchase records. Demographic information describing 12,516 panel households was obtained from a marketing and advertising program operated by H. E. Butt Grocery Company of San Antonio, Texas. A non-probability sample of 2,161 households meeting expenditure criteria was selected and all purchases of dairy products for this sample of households were organized into a database constructed to facilitate the retrieval, aggregation, and analysis of dairy product purchases and their nutrient contents. Two hypotheses were tested: (1) no difference would be found between Hispanic and non-Hispanic purchases of dairy product categories during the study period and (2) no difference would be found between Hispanic and non-Hispanic purchases of nutrients contained in those dairy products during the thirteen-week study period.^ Food purchase records were used to estimate nutrient exposure on a weekly, per capita basis for Hispanic and non-Hispanic households by linking some 40,000 dairy purchase Universal Product code (UPC) numbers with food composition values contained in USDA Handbook 8-1. Results of this study suggest Hispanic sample households consistently purchased fewer dairy products than did non-Hispanic sample households and consequently had fewer nutrients available from dairy purchases. While weekly expenditures for dairy products among the sample households remained relatively constant during the study period, shifts in the types of dairy products purchased were observed. The effect of ethnicity on dairy product and nutrient purchases was significant over the thirteen-week period. A database consisting of customer, household, and purchase information can be developed to successfully associate food item UPC numbers with a standard reference of food composition to estimate nutrient availability in a population over extended periods of time. ^

Left, right and interval bivariate censored data: Evaluating screening mammography in the presence of lead -time bias, length bias and over-detection

Of the large clinical trials evaluating screening mammography efficacy, none included women ages 75 and older. Recommendations on an upper age limit at which to discontinue screening are based on indirect evidence and are not consistent. Screening mammography is evaluated using observational data from the SEER-Medicare linked database. Measuring the benefit of screening mammography is difficult due to the impact of lead-time bias, length bias and over-detection. The underlying conceptual model divides the disease into two stages: pre-clinical (T0) and symptomatic (T1) breast cancer. Treating the time in these phases as a pair of dependent bivariate observations, (t0,t1), estimates are derived to describe the distribution of this random vector. To quantify the effect of screening mammography, statistical inference is made about the mammography parameters that correspond to the marginal distribution of the symptomatic phase duration (T1). This shows the hazard ratio of death from breast cancer comparing women with screen-detected tumors to those detected at their symptom onset is 0.36 (0.30, 0.42), indicating a benefit among the screen-detected cases. ^

Analysis of interval -censored events in hypertension studies: Evaluation of treatment of coronary heart disease

Many statistical studies feature data with both exact-time and interval-censored events. While a number of methods currently exist to handle interval-censored events and multivariate exact-time events separately, few techniques exist to deal with their combination. This thesis develops a theoretical framework for analyzing a multivariate endpoint comprised of a single interval-censored event plus an arbitrary number of exact-time events. The approach fuses the exact-time events, modeled using the marginal method of Wei, Lin, and Weissfeld, with a piecewise-exponential interval-censored component. The resulting model incorporates more of the information in the data and also removes some of the biases associated with the exclusion of interval-censored events. A simulation study demonstrates that our approach produces reliable estimates for the model parameters and their variance-covariance matrix. As a real-world data example, we apply this technique to the Systolic Hypertension in the Elderly Program (SHEP) clinical trial, which features three correlated events: clinical non-fatal myocardial infarction, fatal myocardial infarction (two exact-time events), and silent myocardial infarction (one interval-censored event). ^