Content validation for level of use of feature rich systems: a Delphi study of electronic medical records systems

Introduction. Feature usage is a pre-requisite to realising the benefits of investments in feature rich systems. We propose that conceptualising the dependent variable 'system use' as 'level of use' and specifying it as a formative construct has greater value for measuring the post-adoption use of feature rich systems. We then validate the content of the construct as a first step in developing a research instrument to measure it. The context of our study is the post-adoption use of electronic medical records (EMR) by primary care physicians. Method. Initially, a literature review of the empirical context defines the scope based on prior studies. Having identified core features from the literature, they are further refined with the help of experts in a consensus seeking process that follows the Delphi technique. Results.The methodology was successfully applied to EMRs, which were selected as an example of feature rich systems. A review of EMR usage and regulatory standards provided the feature input for the first round of the Delphi process. A panel of experts then reached consensus after four rounds, identifying ten task-based features that would be indicators of level of use. Conclusions. To study why some users deploy more advanced features than others, theories of post-adoption require a rich formative dependent variable that measures level of use. We have demonstrated that a context sensitive literature review followed by refinement through a consensus seeking process is a suitable methodology to validate the content of this dependent variable. This is the first step of instrument development prior to statistical confirmation with a larger sample.

Using technology to teach technology: design and evaluation of bilingual online physician education about electronic medical records.

The "EMR Tutorial" is designed to be a bilingual online physician education environment about electronic medical records. After iterative assessment and redesign, the tutorial was tested in two groups: U.S. physicians and Mexican medical students. Split-plot ANOVA revealed significantly different pre-test scores in the two groups, significant cognitive gains for the two groups overall, and no significant difference in the gains made by the two groups. Users rated the module positively on a satisfaction questionnaire.

Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?

BACKGROUND: Given the fragmentation of outpatient care, timely follow-up of abnormal diagnostic imaging results remains a challenge. We hypothesized that an electronic medical record (EMR) that facilitates the transmission and availability of critical imaging results through either automated notification (alerting) or direct access to the primary report would eliminate this problem. METHODS: We studied critical imaging alert notifications in the outpatient setting of a tertiary care Department of Veterans Affairs facility from November 2007 to June 2008. Tracking software determined whether the alert was acknowledged (ie, health care practitioner/provider [HCP] opened the message for viewing) within 2 weeks of transmission; acknowledged alerts were considered read. We reviewed medical records and contacted HCPs to determine timely follow-up actions (eg, ordering a follow-up test or consultation) within 4 weeks of transmission. Multivariable logistic regression models accounting for clustering effect by HCPs analyzed predictors for 2 outcomes: lack of acknowledgment and lack of timely follow-up. RESULTS: Of 123 638 studies (including radiographs, computed tomographic scans, ultrasonograms, magnetic resonance images, and mammograms), 1196 images (0.97%) generated alerts; 217 (18.1%) of these were unacknowledged. Alerts had a higher risk of being unacknowledged when the ordering HCPs were trainees (odds ratio [OR], 5.58; 95% confidence interval [CI], 2.86-10.89) and when dual-alert (>1 HCP alerted) as opposed to single-alert communication was used (OR, 2.02; 95% CI, 1.22-3.36). Timely follow-up was lacking in 92 (7.7% of all alerts) and was similar for acknowledged and unacknowledged alerts (7.3% vs 9.7%; P = .22). Risk for lack of timely follow-up was higher with dual-alert communication (OR, 1.99; 95% CI, 1.06-3.48) but lower when additional verbal communication was used by the radiologist (OR, 0.12; 95% CI, 0.04-0.38). Nearly all abnormal results lacking timely follow-up at 4 weeks were eventually found to have measurable clinical impact in terms of further diagnostic testing or treatment. CONCLUSIONS: Critical imaging results may not receive timely follow-up actions even when HCPs receive and read results in an advanced, integrated electronic medical record system. A multidisciplinary approach is needed to improve patient safety in this area.

National Immunization Program: Computerized System as a tool for new challenges

The scope and coverage of the Brazilian Immunization Program can be compared with those in developed countries because it provides a large number of vaccines and has a considerable coverage. The increasing complexity of the program brings challenges regarding its development, high coverage levels, access equality, and safety. The Immunization Information System, with nominal data, is an innovative tool that can more accurately monitor these indicators and allows the evaluation of the impact of new vaccination strategies. The main difficulties for such a system are in its implementation process, training of professionals, mastering its use, its constant maintenance needs and ensuring the information contained remain confidential. Therefore, encouraging the development of this tool should be part of public health policies and should also be involved in the three spheres of government as well as the public and private vaccination services.

DEVELOPING A METHOD FOR IDENTIFYING AND REDUCING FUNCTIONAL DISCREPANCIES OF INFORMATION SYSTEMS

Currently more than half of Electronic Health Record (EHR) projects fail. Most of these failures are not due to flawed technology, but rather due to the lack of systematic considerations of human issues. Among the barriers for EHR adoption, function mismatching among users, activities, and systems is a major area that has not been systematically addressed from a human-centered perspective. A theoretical framework called Functional Framework was developed for identifying and reducing functional discrepancies among users, activities, and systems. The Functional Framework is composed of three models – the User Model, the Designer Model, and the Activity Model. The User Model was developed by conducting a survey (N = 32) that identified the functions needed and desired from the user’s perspective. The Designer Model was developed by conducting a systemic review of an Electronic Dental Record (EDR) and its functions. The Activity Model was developed using an ethnographic method called shadowing where EDR users (5 dentists, 5 dental assistants, 5 administrative personnel) were followed quietly and observed for their activities. These three models were combined to form a unified model. From the unified model the work domain ontology was developed by asking users to rate the functions (a total of 190 functions) in the unified model along the dimensions of frequency and criticality in a survey. The functional discrepancies, as indicated by the regions of the Venn diagrams formed by the three models, were consistent with the survey results, especially with user satisfaction. The survey for the Functional Framework indicated the preference of one system over the other (R=0.895). The results of this project showed that the Functional Framework provides a systematic method for identifying, evaluating, and reducing functional discrepancies among users, systems, and activities. Limitations and generalizability of the Functional Framework were discussed.

Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain?

BACKGROUND: Follow-up of abnormal outpatient laboratory test results is a major patient safety concern. Electronic medical records can potentially address this concern through automated notification. We examined whether automated notifications of abnormal laboratory results (alerts) in an integrated electronic medical record resulted in timely follow-up actions. METHODS: We studied 4 alerts: hemoglobin A1c > or =15%, positive hepatitis C antibody, prostate-specific antigen > or =15 ng/mL, and thyroid-stimulating hormone > or =15 mIU/L. An alert tracking system determined whether the alert was acknowledged (ie, provider clicked on and opened the message) within 2 weeks of transmission; acknowledged alerts were considered read. Within 30 days of result transmission, record review and provider contact determined follow-up actions (eg, patient contact, treatment). Multivariable logistic regression models analyzed predictors for lack of timely follow-up. RESULTS: Between May and December 2008, 78,158 tests (hemoglobin A1c, hepatitis C antibody, thyroid-stimulating hormone, and prostate-specific antigen) were performed, of which 1163 (1.48%) were transmitted as alerts; 10.2% of these (119/1163) were unacknowledged. Timely follow-up was lacking in 79 (6.8%), and was statistically not different for acknowledged and unacknowledged alerts (6.4% vs 10.1%; P =.13). Of 1163 alerts, 202 (17.4%) arose from unnecessarily ordered (redundant) tests. Alerts for a new versus known diagnosis were more likely to lack timely follow-up (odds ratio 7.35; 95% confidence interval, 4.16-12.97), whereas alerts related to redundant tests were less likely to lack timely follow-up (odds ratio 0.24; 95% confidence interval, 0.07-0.84). CONCLUSIONS: Safety concerns related to timely patient follow-up remain despite automated notification of non-life-threatening abnormal laboratory results in the outpatient setting.

Matching prescription claims with medication data for nursing home residents: Implications for prescriber feedback, drug utilisation studies and selection of prescription claims database

Medication data retrieved from Australian Repatriation Pharmaceutical Benefits Scheme (RPBS) claims for 44 veterans residing in nursing homes and Pharmaceutical Benefits Scheme (PBS) claims for 898 nursing home residents were compared with medication data from nursing home records to determine the optimal time interval for retrieving claims data and its validity. Optimal matching was achieved using 12 weeks of RPBS claims data, with 60% of medications in the RPBS claims located in nursing home administration records, and 78% of medications administered to nursing home residents identified in RPBS claims. In comparison, 48% of medications administered to nursing home residents could be found in 12 weeks of PBS data, and 56% of medications present in PBS claims could be matched with nursing home administration records. RPBS claims data was superior to PBS, due to the larger number of scheduled items available to veterans and the veteran's file number, which acts as a unique identifier. These findings should be taken into account when using prescription claims data for medication histories, prescriber feedback, drug utilisation, intervention or epidemiological studies. (C) 2001 Elsevier Science Inc. All rights reserved.

Improving integrated care in chronic kidney failure patients with a standard-based interoperability framework.

This paper introduces the evaluation report after fostering a Standard-based Interoperability Framework (SIF) between the Virgen del Rocío University Hospital (VRUH) Haemodialysis (HD) Unit and 5 outsourced HD centres in order to improve integrated care by automatically sharing patients' Electronic Health Record (EHR) and lab test reports. A pre-post study was conducted during fourteen months. The number of lab test reports of both emergency and routine nature regarding to 379 outpatients was computed before and after the integration of the SIF. Before fostering SIF, 19.38 lab tests per patient were shared between VRUH and HD centres, 5.52 of them were of emergency nature while 13.85 were routine. After integrating SIF, 17.98 lab tests per patient were shared, 3.82 of them were of emergency nature while 14.16 were routine. The inclusion of a SIF in the HD Integrated Care Process has led to an average reduction of 1.39 (p=0.775) lab test requests per patient, including a reduction of 1.70 (p=0.084) in those of emergency nature, whereas an increase of 0.31 (p=0.062) was observed in routine lab tests. Fostering this strategy has led to the reduction in emergency lab test requests, which implies a potential improvement of the integrated care.

A Clinical Decision Support using a Terminology Server to improve Patient Safety.

Clinical Decision Support Systems (CDSS) are software applications that support clinicians in making healthcare decisions providing relevant information for individual patients about their specific conditions. The lack of integration between CDSS and Electronic Health Record (EHR) has been identified as a significant barrier to CDSS development and adoption. Andalusia Healthcare Public System (AHPS) provides an interoperable health information infrastructure based on a Service Oriented Architecture (SOA) that eases CDSS implementation. This paper details the deployment of a CDSS jointly with the deployment of a Terminology Server (TS) within the AHPS infrastructure. It also explains a case study about the application of decision support to thromboembolism patients and its potential impact on improving patient safety. We will apply the inSPECt tool proposal to evaluate the appropriateness of alerts in this scenario.

Current variables, definitions and endpoints of the European cardiovascular magnetic resonance registry.

BACKGROUND: Cardiovascular magnetic resonance (CMR) is increasingly used in daily clinical practice. However, little is known about its clinical utility such as image quality, safety and impact on patient management. In addition, there is limited information about the potential of CMR to acquire prognostic information. METHODS: The European Cardiovascular Magnetic Resonance Registry (EuroCMR Registry) will consist of two parts: 1) Multicenter registry with consecutive enrolment of patients scanned in all participating European CMR centres using web based online case record forms. 2) Prospective clinical follow up of patients with suspected coronary artery disease (CAD) and hypertrophic cardiomyopathy (HCM) every 12 months after enrolment to assess prognostic data. CONCLUSION: The EuroCMR Registry offers an opportunity to provide information about the clinical utility of routine CMR in a large number of cases and a diverse population. Furthermore it has the potential to gather information about the prognostic value of CMR in specific patient populations.

Measuring potentially avoidable hospital readmissions.

The objectives of this study were to develop a computerized method to screen for potentially avoidable hospital readmissions using routinely collected data and a prediction model to adjust rates for case mix. We studied hospital information system data of a random sample of 3,474 inpatients discharged alive in 1997 from a university hospital and medical records of those (1,115) readmitted within 1 year. The gold standard was set on the basis of the hospital data and medical records: all readmissions were classified as foreseen readmissions, unforeseen readmissions for a new affection, or unforeseen readmissions for a previously known affection. The latter category was submitted to a systematic medical record review to identify the main cause of readmission. Potentially avoidable readmissions were defined as a subgroup of unforeseen readmissions for a previously known affection occurring within an appropriate interval, set to maximize the chance of detecting avoidable readmissions. The computerized screening algorithm was strictly based on routine statistics: diagnosis and procedures coding and admission mode. The prediction was based on a Poisson regression model. There were 454 (13.1%) unforeseen readmissions for a previously known affection within 1 year. Fifty-nine readmissions (1.7%) were judged avoidable, most of them occurring within 1 month, which was the interval used to define potentially avoidable readmissions (n = 174, 5.0%). The intra-sample sensitivity and specificity of the screening algorithm both reached approximately 96%. Higher risk for potentially avoidable readmission was associated with previous hospitalizations, high comorbidity index, and long length of stay; lower risk was associated with surgery and delivery. The model offers satisfactory predictive performance and a good medical plausibility. The proposed measure could be used as an indicator of inpatient care outcome. However, the instrument should be validated using other sets of data from various hospitals.

Análise de prontuários e da legalidade da criação do banco de imagens dos pacientes portadores de câncer atendidos na Faculdade de Odontologia de Araçatuba – UNESP

Pós-graduação em Odontologia - FOA

Extending feature usage: a study of the post-adoption of electronic medical records

Technology Acceptance Model (TAM) posits that Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) influence the ‘intention to use’. The Post-Acceptance Model (PAM) posits that continued use is influenced by prior experience. In order to study the factors that influence how professionals use complex systems, we create a tentative research model that builds on PAM and TAM. Specifically we include PEOU and the construct ‘Professional Association Guidance’. We postulate that feature usage is enhanced when professional associations influence PU by highlighting additional benefits. We explore the theory in the context of post-adoption use of Electronic Medical Records (EMRs) by primary care physicians in Ontario. The methodology can be extended to other professional environments and we suggest directions for future research.

Electronic medical records may be inadequate for improving population health status through general practice: cervical smears as a case study

Objective: To determine whether routine electronic records are an accurate source of population health data in general practice through reviewing cervical smears rates in four South Australian practices. Methods: The cervical screening rate in a purposive sample of four general practices (three rural and one urban) was obtained using an audit of medical records and a telephone follow-up. Results: The cervical screening rate using only immediately available electronic medical records indicated an overall low rate for the participating practices (44.9%). However, telephone follow-up and adjustments to the denominator indicated the real rate to be 85.7%. The offer of appointments during the telephone follow-up further improved this rate for eligible women (93.8%). Conclusions and implications: Electronic medical records may be inadequate in preventive screening in general practice, without ensuring their accuracy. Updating records by telephone or personal follow-up produces a much more accurate denominator.