DEVELOPING A METHOD FOR IDENTIFYING AND REDUCING FUNCTIONAL DISCREPANCIES OF INFORMATION SYSTEMS

Currently more than half of Electronic Health Record (EHR) projects fail. Most of these failures are not due to flawed technology, but rather due to the lack of systematic considerations of human issues. Among the barriers for EHR adoption, function mismatching among users, activities, and systems is a major area that has not been systematically addressed from a human-centered perspective. A theoretical framework called Functional Framework was developed for identifying and reducing functional discrepancies among users, activities, and systems. The Functional Framework is composed of three models – the User Model, the Designer Model, and the Activity Model. The User Model was developed by conducting a survey (N = 32) that identified the functions needed and desired from the user’s perspective. The Designer Model was developed by conducting a systemic review of an Electronic Dental Record (EDR) and its functions. The Activity Model was developed using an ethnographic method called shadowing where EDR users (5 dentists, 5 dental assistants, 5 administrative personnel) were followed quietly and observed for their activities. These three models were combined to form a unified model. From the unified model the work domain ontology was developed by asking users to rate the functions (a total of 190 functions) in the unified model along the dimensions of frequency and criticality in a survey. The functional discrepancies, as indicated by the regions of the Venn diagrams formed by the three models, were consistent with the survey results, especially with user satisfaction. The survey for the Functional Framework indicated the preference of one system over the other (R=0.895). The results of this project showed that the Functional Framework provides a systematic method for identifying, evaluating, and reducing functional discrepancies among users, systems, and activities. Limitations and generalizability of the Functional Framework were discussed.

A Time-Motion Study of Registered Nurses’ Workflow in Intensive Care Unit Remote Monitoring

Utilizing advanced information technology, Intensive Care Unit (ICU) remote monitoring allows highly trained specialists to oversee a large number of patients at multiple sites on a continuous basis. In the current research, we conducted a time-motion study of registered nurses’ work in an ICU remote monitoring facility. Data were collected on seven nurses through 40 hours of observation. The results showed that nurses’ essential tasks were centered on three themes: monitoring patients, maintaining patients’ health records, and managing technology use. In monitoring patients, nurses spent 52% of the time assimilating information embedded in a clinical information system and 15% on monitoring live vitals. System-generated alerts frequently interrupted nurses in their task performance and redirected them to manage suddenly appearing events. These findings provide insight into nurses’ workflow in a new, technology-driven critical care setting and have important implications for system design, work engineering, and personnel selection and training.

Using technology to teach technology: design and evaluation of bilingual online physician education about electronic medical records.

The "EMR Tutorial" is designed to be a bilingual online physician education environment about electronic medical records. After iterative assessment and redesign, the tutorial was tested in two groups: U.S. physicians and Mexican medical students. Split-plot ANOVA revealed significantly different pre-test scores in the two groups, significant cognitive gains for the two groups overall, and no significant difference in the gains made by the two groups. Users rated the module positively on a satisfaction questionnaire.

Comparison of clinical knowledge management capabilities of commercially-available and leading internally-developed electronic health records.

BACKGROUND: We have carried out an extensive qualitative research program focused on the barriers and facilitators to successful adoption and use of various features of advanced, state-of-the-art electronic health records (EHRs) within large, academic, teaching facilities with long-standing EHR research and development programs. We have recently begun investigating smaller, community hospitals and out-patient clinics that rely on commercially-available EHRs. We sought to assess whether the current generation of commercially-available EHRs are capable of providing the clinical knowledge management features, functions, tools, and techniques required to deliver and maintain the clinical decision support (CDS) interventions required to support the recently defined "meaningful use" criteria. METHODS: We developed and fielded a 17-question survey to representatives from nine commercially available EHR vendors and four leading internally developed EHRs. The first part of the survey asked basic questions about the vendor's EHR. The second part asked specifically about the CDS-related system tools and capabilities that each vendor provides. The final section asked about clinical content. RESULTS: All of the vendors and institutions have multiple modules capable of providing clinical decision support interventions to clinicians. The majority of the systems were capable of performing almost all of the key knowledge management functions we identified. CONCLUSION: If these well-designed commercially-available systems are coupled with the other key socio-technical concepts required for safe and effective EHR implementation and use, and organizations have access to implementable clinical knowledge, we expect that the transformation of the healthcare enterprise that so many have predicted, is achievable using commercially-available, state-of-the-art EHRs.

Improving follow-up of abnormal cancer screens using electronic health records: trust but verify test result communication.

BACKGROUND: Early detection of colorectal cancer through timely follow-up of positive Fecal Occult Blood Tests (FOBTs) remains a challenge. In our previous work, we found 40% of positive FOBT results eligible for colonoscopy had no documented response by a treating clinician at two weeks despite procedures for electronic result notification. We determined if technical and/or workflow-related aspects of automated communication in the electronic health record could lead to the lack of response. METHODS: Using both qualitative and quantitative methods, we evaluated positive FOBT communication in the electronic health record of a large, urban facility between May 2008 and March 2009. We identified the source of test result communication breakdown, and developed an intervention to fix the problem. Explicit medical record reviews measured timely follow-up (defined as response within 30 days of positive FOBT) pre- and post-intervention. RESULTS: Data from 11 interviews and tracking information from 490 FOBT alerts revealed that the software intended to alert primary care practitioners (PCPs) of positive FOBT results was not configured correctly and over a third of positive FOBTs were not transmitted to PCPs. Upon correction of the technical problem, lack of timely follow-up decreased immediately from 29.9% to 5.4% (p<0.01) and was sustained at month 4 following the intervention. CONCLUSION: Electronic communication of positive FOBT results should be monitored to avoid limiting colorectal cancer screening benefits. Robust quality assurance and oversight systems are needed to achieve this. Our methods may be useful for others seeking to improve follow-up of FOBTs in their systems.

Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?

BACKGROUND: Given the fragmentation of outpatient care, timely follow-up of abnormal diagnostic imaging results remains a challenge. We hypothesized that an electronic medical record (EMR) that facilitates the transmission and availability of critical imaging results through either automated notification (alerting) or direct access to the primary report would eliminate this problem. METHODS: We studied critical imaging alert notifications in the outpatient setting of a tertiary care Department of Veterans Affairs facility from November 2007 to June 2008. Tracking software determined whether the alert was acknowledged (ie, health care practitioner/provider [HCP] opened the message for viewing) within 2 weeks of transmission; acknowledged alerts were considered read. We reviewed medical records and contacted HCPs to determine timely follow-up actions (eg, ordering a follow-up test or consultation) within 4 weeks of transmission. Multivariable logistic regression models accounting for clustering effect by HCPs analyzed predictors for 2 outcomes: lack of acknowledgment and lack of timely follow-up. RESULTS: Of 123 638 studies (including radiographs, computed tomographic scans, ultrasonograms, magnetic resonance images, and mammograms), 1196 images (0.97%) generated alerts; 217 (18.1%) of these were unacknowledged. Alerts had a higher risk of being unacknowledged when the ordering HCPs were trainees (odds ratio [OR], 5.58; 95% confidence interval [CI], 2.86-10.89) and when dual-alert (>1 HCP alerted) as opposed to single-alert communication was used (OR, 2.02; 95% CI, 1.22-3.36). Timely follow-up was lacking in 92 (7.7% of all alerts) and was similar for acknowledged and unacknowledged alerts (7.3% vs 9.7%; P = .22). Risk for lack of timely follow-up was higher with dual-alert communication (OR, 1.99; 95% CI, 1.06-3.48) but lower when additional verbal communication was used by the radiologist (OR, 0.12; 95% CI, 0.04-0.38). Nearly all abnormal results lacking timely follow-up at 4 weeks were eventually found to have measurable clinical impact in terms of further diagnostic testing or treatment. CONCLUSIONS: Critical imaging results may not receive timely follow-up actions even when HCPs receive and read results in an advanced, integrated electronic medical record system. A multidisciplinary approach is needed to improve patient safety in this area.

Survey of shift and work tolerance of City of Houston emergency medical service personnel

This study of ambulance workers for the emergency medical services of the City of Houston studied the factors related to shiftwork tolerance and intolerance. The EMS personnel work a 24-hour shift with rotating days of the week. Workers are assigned to A, B, C, D shift, each of which rotate 24-hours on, 24-hours off, 24-hours on and 4 days off. One-hundred and seventy-six male EMTs, paramedics and chauffeurs from stations of varying levels of activity were surveyed. The sample group ranged in age from 20 to 45. The average tenure on the job was 8.2 years. Over 68% of the workers held a second job, the majority of which worked over 20 hours a week at the second position.^ The survey instrument was a 20-page questionnaire modeled after the Folkard Standardized Shiftwork Index. In addition to demographic data, the survey tool provided measurements of general job satisfaction, sleep quality, general health complaints, morningness/eveningness, cognitive and somatic anxiety, depression, and circadian types. The survey questionnaire included an EMS-specific scaler of stress.^ A conceptual model of Shiftwork Tolerance was presented to identify the key factors examined in the study. An extensive list of 265 variables was reduced to 36 key variables that related to: (1) shift schedule and demographic/lifestyle factors, (2) individual differences related to traits and characteristics, and (3) tolerance/intolerance effects. Using the general job satisfaction scaler as the key measurement of shift tolerance/intolerance, it was shown that a significant relationship existed between this dependent variable and stress, number of years working a 24-hour shift, sleep quality, languidness/vigorousness. The usual amount of sleep received during the shift, general health complaints and flexibility/rigidity (R$\sp2$ =.5073).^ The sample consisted of a majority of morningness-types or extreme-morningness types, few evening-types and no extreme-evening types, duplicating the findings of Motohashi's previous study of ambulance workers. The level of activity by station was not significant on any of the dependent variables examined. However, the shift worked had a relationship with sleep quality, despite the fact that all shifts work the same hours and participate in the same rotation schedule. ^

Concordance between medical records and personal interview in ascertaining breast cancer and diabetes status: Analysis of the south Texas women's health project

Accurate ascertainment of risk factors and disease status is vital in public health research for proper classification of research subjects. The two most common ways of obtaining this data is by self-report and review of medical records (MRs). South Texas Women’s Health Project was a case-control study looking at interrelationships between hormones, diet, and body size and breast cancer among Hispanic women 30-79 years of age. History of breast cancer, diabetes mellitus (DM) and use of DM medications was ascertained from a personal interview. At the time of interview, the subject identified her major health care providers and signed the medical records release form, which was sent to the designated providers. The MRs were reviewed to confirm information obtained from the interview.^ Aim of this study was to determine the sensitivity and specificity between MRs and personal interview in diagnosis of breast cancer, DM and DM treatment. We also wanted to assess how successful our low-cost approach was in obtaining pertinent MRs and what factors influenced the quality of MR or interview data. Study sample was 721 women with both self-report and MR data available by June 2007. Overall response rate for MR requests was 74.5%. MRs were 80.9% sensitive and 100% specific in confirming breast cancer status. Prevalence of DM was 22.7% from the interviews and 16% from MRs. MRs did not provide definite information about DM status of 53.6% subjects. Sensitivity and specificity of MRs for DM status was 88.9% and 90.4% respectively. Disagreement on DM status from the two sources was seen in 15.9% subjects. This discordance was more common among older subjects, those who were married and were predominantly Spanish speaking. Income and level of education did not have a statistically significantly association with this disagreement.^ Both self-report and MRs underestimate the prevalence of DM. Relying solely on MRs leads to greater misclassification than relying on self-report data. MRs have good to excellent specificity and thus serve as a good tool to confirm information obtained from self-report. Self-report and MRs should be used in a complementary manner for accurate assessment of DM and breast cancer status.^

Introduction to disaster epidemiology for public health personnel

There has been a renewed interest in disaster epidemiology after the World Trade Center and Pentagon terrorist attacks of 2001, the devastation of Hurricane Katrina and the overwhelming loss of life that resulted from the tsunami that originated in the Indian Ocean and struck Indonesia and other adjacent countries on December 26, 2004. Institutions that have accepted the challenge of training the next generation of public health professionals as well as to continue the education of the dedicated professionals already serving in public health fields have a responsibility to train practitioners in the basic principles of disaster epidemiology as well as in practical applications of these principles. This culminating experience project involved developing an on-line course complete with the background information as well as relevant case studies that can be used as a curriculum for an introductory course in disaster epidemiology. ^

A nested case-control study of radiation exposure and brain cancer incidence in the United States Air Force

A nested case-control study design was used to investigate the relationship between radiation exposure and brain cancer risk in the United States Air Force (USAF). The cohort consisted of approximately 880,000 men with at least 1 year of service between 1970 and 1989. Two hundred and thirty cases were identified from hospital discharge records with a diagnosis of primary malignant brain tumor (International Classification of Diseases, 9th revision, code 191). Four controls were exactly matched with each case on year of age and race using incidence density sampling. Potential career summary extremely low frequency (ELF) and microwave-radiofrequency (MWRF) radiation exposures were based upon the duration in each occupation and an intensity score assigned by an expert panel. Ionizing radiation (IR) exposures were obtained from personal dosimetry records.^ Relative to the unexposed, the overall age-race adjusted odds ratio (OR) for ELF exposure was 1.39, 95 percent confidence interval (CI) 1.03-1.88. A dose-response was not evident. The same was true for MWRF, although the OR = 1.59, with 95 percent CI 1.18-2.16. Excess risk was not found for IR exposure (OR = 0.66, 45 percent CI 0.26-1.72).^ Increasing socioeconomic status (SES), as identified by military pay grade, was associated with elevated brain tumor risk (officer vs. enlisted personnel age-race adjusted OR = 2.11, 95 percent CI 1.98-3.01, and senior officers vs. all others age-race adjusted OR = 3.30, 95 percent CI 2.0-5.46). SES proved to be an important confounder of the brain tumor risk associated with ELF and MWRF exposure. For ELF, the age-race-SES adjusted OR = 1.28, 95 percent CI 0.94-1.74, and for MWRF, the age-race-SES adjusted OR = 1.39, 95 percent CI 1.01-1.90.^ These results indicate that employment in Air Force occupations with potential electromagnetic field exposures is weakly, though not significantly, associated with increased risk for brain tumors. SES appeared to be the most consistent brain tumor risk factor in the USAF cohort. Other investigators have suggested that an association between brain tumor risk and SES may arise from differential access to medical care. However, in the USAF cohort health care is universally available. This study suggests that some factor other than access to medical care must underlie the association between SES and brain tumor risk. ^

EXPOSURE OF PHARMACY PERSONNEL TO ANTINEOPLASTIC DRUGS

This dissertation consists of two parts: (1) Exposure of pharmacy personnel to antineoplastic drugs. The Salmonella reversion test was used to measure the mutagenic activities of urine concentrates from individuals preparing antineoplastic drugs for intravenous administration. Longitudinal studies were performed in which the total urine produced in 24-hour periods was collected, starting on a Sunday at 7 P.M. after a duty-free weekend and extending over an eight-day period. There was no detectable increase in mutagenic activity in the urine concentrates of three pharmacy administrators who had no contact with these drugs. All six individuals admixing drugs in open-faced, horizontal laminar flow hoods displayed a two-fold increase in mutagenesis by the fourth day with peak values of 2.7 to 24-fold occurring on days five and six, reduced values by day seven with a return to the spontaneous level by day eight. When four of the six positive individuals in the preceding experiment admixed comparable amounts of antineoplastic drugs in a closed-faced, vertical laminar flow hood, no increase in mutagenic activity was detected in their urine concentrates over the eight-day period. (2) Estimate of potential carcinogenic risks of antineoplastic drugs. Excision repair is the major repair system that is involved with the elimination of chemically induced DNA (deoxyribonucleic acid) lesions. This DNA excision repair capability increases in mammalian species with longer life span such as humans. In this study, the effect of functional DNA excision repair on the mutagenesis invoked by 17 antineoplastic drugs was determined by using a Salmonella/Microsome assay which was expanded to include some uvr('+) counterparts of the excisionless (uvrB) tester strains routinely employed. Although extrapolation cannot be made from bacteria to humans, one should be able to make a qualitative comparison as to which antineoplastic drugs are more potentially carcinogenic to humans based on the effects of excision repair on their mutagenesis in bacteria. The tested antineoplastic drugs were divided into three classes: those requiring excision repair for mutagenesis; those producing nonrepairable genetic damage; and those producing mostly repairable premutational DNA lesions. ^

COMPREHENSIVE CALCULATION-BASED IMRT QA USING R&V DATA, TREATMENT RECORDS, AND A SECOND TREATMENT PLANNING SYSTEM

Purpose: Traditional patient-specific IMRT QA measurements are labor intensive and consume machine time. Calculation-based IMRT QA methods typically are not comprehensive. We have developed a comprehensive calculation-based IMRT QA method to detect uncertainties introduced by the initial dose calculation, the data transfer through the Record-and-Verify (R&V) system, and various aspects of the physical delivery. Methods: We recomputed the treatment plans in the patient geometry for 48 cases using data from the R&V, and from the delivery unit to calculate the “as-transferred” and “as-delivered” doses respectively. These data were sent to the original TPS to verify transfer and delivery or to a second TPS to verify the original calculation. For each dataset we examined the dose computed from the R&V record (RV) and from the delivery records (Tx), and the dose computed with a second verification TPS (vTPS). Each verification dose was compared to the clinical dose distribution using 3D gamma analysis and by comparison of mean dose and ROI-specific dose levels to target volumes. Plans were also compared to IMRT QA absolute and relative dose measurements. Results: The average 3D gamma passing percentages using 3%-3mm, 2%-2mm, and 1%-1mm criteria for the RV plan were 100.0 (σ=0.0), 100.0 (σ=0.0), and 100.0 (σ=0.1); for the Tx plan they were 100.0 (σ=0.0), 100.0 (σ=0.0), and 99.0 (σ=1.4); and for the vTPS plan they were 99.3 (σ=0.6), 97.2 (σ=1.5), and 79.0 (σ=8.6). When comparing target volume doses in the RV, Tx, and vTPS plans to the clinical plans, the average ratios of ROI mean doses were 0.999 (σ=0.001), 1.001 (σ=0.002), and 0.990 (σ=0.009) and ROI-specific dose levels were 0.999 (σ=0.001), 1.001 (σ=0.002), and 0.980 (σ=0.043), respectively. Comparing the clinical, RV, TR, and vTPS calculated doses to the IMRT QA measurements for all 48 patients, the average ratios for absolute doses were 0.999 (σ=0.013), 0.998 (σ=0.013), 0.999 σ=0.015), and 0.990 (σ=0.012), respectively, and the average 2D gamma(5%-3mm) passing percentages for relative doses for 9 patients was were 99.36 (σ=0.68), 99.50 (σ=0.49), 99.13 (σ=0.84), and 98.76 (σ=1.66), respectively. Conclusions: Together with mechanical and dosimetric QA, our calculation-based IMRT QA method promises to minimize the need for patient-specific QA measurements by identifying outliers in need of further review.

Indeterminate QunatiFERON-TB Gold In-Tube assay results in association with procedural specimen collection in children

BACKGROUND. The development of interferon-gamma release assays (IGRA) has introduced powerful tools in diagnosing latent tuberculosis infection (LTBI) and may play a critical role in the future of tuberculosis diagnosis. However, there have been reports of high indeterminate results in young patient populations (0-18 years). This study investigated results of the QunatiFERON-TB Gold In-Tube (QFT-GIT) IGRA in a population of children (0-18 years) at Texas Children's Hospital in association with specimen collection procedures using surrogate variables. ^ METHODS. A retrospective case-control study design was used for this investigation. Cases were defined as having QFT-GIT indeterminate results. Controls were defined as having either positive or negative results (determinates). Patients' admission status, staff performing specimen collection, and specific nurse performing specimen collection were used as surrogates to measure specimen collection procedures. ^ To minimize potential confounding, abstraction of patients' electronic medical records was performed. Abstracted data included patients' medications and evaluation at the time of QFT-GIT specimen collection in addition to their medical history. QFT-GIT related data was also abstracted. Cases and controls were characterized using chi-squared tests or Fisher's exact tests across categorical variables. Continuous variables were analyzed using one-way ANOVA and t-tests for continuous variables. A multivariate model was constructed by backward stepwise removal of statistically significant variables from univariate analysis. ^ RESULTS. Patient data was abstracted from 182 individuals aged 0-18 years from July 2010 to August 2011 at Texas Children's Hospital. 56 cases (indeterminates) and 126 controls (determinates) were enrolled. Cancer was found to be an effect modifier with subsequent stratification resulting in a cancer patient population too small to analyze (n=13). Subsequent analyses excluded these patients. ^ The exclusion of cancer patients resulted in a population of 169 patients with 49 indeterminates (28.99%) and 120 determinates (71.01%), with mean ages of 9.73 (95% CI: 8.03, 11.43) years and 11.66 (95% CI: 10.75, 12.56) years (p = 0.033), respectively. Median age of patients who were indeterminates and determinates were 12.37 and 12.87 years, respectively. Lack of data for our specific nurse surrogate (QFTNurse) resulted in its exclusion from analysis. The final model included only our remaining surrogate variables (QFTStaff and QFTInpatientOutpatient). The staff collecting surrogate (QFTStaff) was found to be modestly associated with indeterminates when nurses collected the specimen (OR = 1.54, 95% CI: 0.51, 4.64, p = 0.439) in the final model. Inpatients were found to have a strong and statistically significant association with indeterminates (OR = 11.65, 95% CI: 3.89, 34.9, p < 0.001) in the final model. ^ CONCLUSION. Inpatient status was used as a surrogate for indication of nurse drawn blood specimens. Nurses have had little to no training regarding shaking of tubes versus phlebotomists regarding QFT-GIT testing procedures. This was also measured by two other surrogates; specifically a medical note stating whether a nurse or phlebotomist collected the specimen (QFTStaff) and the name and title of the specific nurse if collection was performed by a nurse (QFTNurse). Results indicated that inpatient status was a strong and statistically significant factor for indeterminates, however, nurse collected specimens and indeterminate results had no statistically significant association in non-cancer patients. The lack of data denoting the specific nurse performing specimen collection excluded the QFTNurse surrogate in our analysis. ^ Findings suggests training of staff personnel in specimen procedures may have little effect on the number of indeterminates while inpatient status and thus possibly illness severity may be the most important factor for indeterminate results in this population. The lack of congruence between our surrogate measures may imply that our inpatient surrogate gauged illness severity rather than collection procedures as intended. ^ Despite the lack of clear findings, our analysis indicated that more than half of indeterminates were found in specimens drawn by nurses and as such staff training may be explored. Future studies may explore methods in measuring modifiable variables during pre-analytical QFT-GIT procedures that can be discerned and controlled. Identification of such measures may provide insight into ways to lowering indeterminate QFT-GIT rates in children.^