Content validation for level of use of feature rich systems: a Delphi study of electronic medical records systems

Introduction. Feature usage is a pre-requisite to realising the benefits of investments in feature rich systems. We propose that conceptualising the dependent variable 'system use' as 'level of use' and specifying it as a formative construct has greater value for measuring the post-adoption use of feature rich systems. We then validate the content of the construct as a first step in developing a research instrument to measure it. The context of our study is the post-adoption use of electronic medical records (EMR) by primary care physicians. Method. Initially, a literature review of the empirical context defines the scope based on prior studies. Having identified core features from the literature, they are further refined with the help of experts in a consensus seeking process that follows the Delphi technique. Results.The methodology was successfully applied to EMRs, which were selected as an example of feature rich systems. A review of EMR usage and regulatory standards provided the feature input for the first round of the Delphi process. A panel of experts then reached consensus after four rounds, identifying ten task-based features that would be indicators of level of use. Conclusions. To study why some users deploy more advanced features than others, theories of post-adoption require a rich formative dependent variable that measures level of use. We have demonstrated that a context sensitive literature review followed by refinement through a consensus seeking process is a suitable methodology to validate the content of this dependent variable. This is the first step of instrument development prior to statistical confirmation with a larger sample.

Using technology to teach technology: design and evaluation of bilingual online physician education about electronic medical records.

The "EMR Tutorial" is designed to be a bilingual online physician education environment about electronic medical records. After iterative assessment and redesign, the tutorial was tested in two groups: U.S. physicians and Mexican medical students. Split-plot ANOVA revealed significantly different pre-test scores in the two groups, significant cognitive gains for the two groups overall, and no significant difference in the gains made by the two groups. Users rated the module positively on a satisfaction questionnaire.

Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?

BACKGROUND: Given the fragmentation of outpatient care, timely follow-up of abnormal diagnostic imaging results remains a challenge. We hypothesized that an electronic medical record (EMR) that facilitates the transmission and availability of critical imaging results through either automated notification (alerting) or direct access to the primary report would eliminate this problem. METHODS: We studied critical imaging alert notifications in the outpatient setting of a tertiary care Department of Veterans Affairs facility from November 2007 to June 2008. Tracking software determined whether the alert was acknowledged (ie, health care practitioner/provider [HCP] opened the message for viewing) within 2 weeks of transmission; acknowledged alerts were considered read. We reviewed medical records and contacted HCPs to determine timely follow-up actions (eg, ordering a follow-up test or consultation) within 4 weeks of transmission. Multivariable logistic regression models accounting for clustering effect by HCPs analyzed predictors for 2 outcomes: lack of acknowledgment and lack of timely follow-up. RESULTS: Of 123 638 studies (including radiographs, computed tomographic scans, ultrasonograms, magnetic resonance images, and mammograms), 1196 images (0.97%) generated alerts; 217 (18.1%) of these were unacknowledged. Alerts had a higher risk of being unacknowledged when the ordering HCPs were trainees (odds ratio [OR], 5.58; 95% confidence interval [CI], 2.86-10.89) and when dual-alert (>1 HCP alerted) as opposed to single-alert communication was used (OR, 2.02; 95% CI, 1.22-3.36). Timely follow-up was lacking in 92 (7.7% of all alerts) and was similar for acknowledged and unacknowledged alerts (7.3% vs 9.7%; P = .22). Risk for lack of timely follow-up was higher with dual-alert communication (OR, 1.99; 95% CI, 1.06-3.48) but lower when additional verbal communication was used by the radiologist (OR, 0.12; 95% CI, 0.04-0.38). Nearly all abnormal results lacking timely follow-up at 4 weeks were eventually found to have measurable clinical impact in terms of further diagnostic testing or treatment. CONCLUSIONS: Critical imaging results may not receive timely follow-up actions even when HCPs receive and read results in an advanced, integrated electronic medical record system. A multidisciplinary approach is needed to improve patient safety in this area.

Electronic medical records may be inadequate for improving population health status through general practice: cervical smears as a case study

Objective: To determine whether routine electronic records are an accurate source of population health data in general practice through reviewing cervical smears rates in four South Australian practices. Methods: The cervical screening rate in a purposive sample of four general practices (three rural and one urban) was obtained using an audit of medical records and a telephone follow-up. Results: The cervical screening rate using only immediately available electronic medical records indicated an overall low rate for the participating practices (44.9%). However, telephone follow-up and adjustments to the denominator indicated the real rate to be 85.7%. The offer of appointments during the telephone follow-up further improved this rate for eligible women (93.8%). Conclusions and implications: Electronic medical records may be inadequate in preventive screening in general practice, without ensuring their accuracy. Updating records by telephone or personal follow-up produces a much more accurate denominator.

Understanding Continuance Intentions of Physicians with Electronic Medical Records (EMRS): An Expectancy-Confirmation Perspective

This thesis examines physicians’ satisfaction with electronic medical records (EMRs) in the post-adoption phase. More specifically, the study examines how physicians’ satisfaction with EMRs impacts on their intention to continue using as well as extend their adoption of additional functions of EMRs. Expectation-confirmation theory is used with the incorporation of perceived risk as the theoretical framework. The extended theoretical model is used to formulate eight hypotheses to aid in the understanding of physicians’ continuance intentions. A field survey of 135 Canadian physicians that utilize EMRs was performed to test the model empirically. The study found that physicians are willing to continue using and adopting additional components of EMRs. In addition, the empirical results suggest that physicians’ perceived usefulness and perceived risk impacts satisfaction, which in turn influences physicians’ continuance intentions. As well, perceived risk has an influence on physicians’ continuance intentions directly.

Extending feature usage: a study of the post-adoption of electronic medical records

Technology Acceptance Model (TAM) posits that Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) influence the ‘intention to use’. The Post-Acceptance Model (PAM) posits that continued use is influenced by prior experience. In order to study the factors that influence how professionals use complex systems, we create a tentative research model that builds on PAM and TAM. Specifically we include PEOU and the construct ‘Professional Association Guidance’. We postulate that feature usage is enhanced when professional associations influence PU by highlighting additional benefits. We explore the theory in the context of post-adoption use of Electronic Medical Records (EMRs) by primary care physicians in Ontario. The methodology can be extended to other professional environments and we suggest directions for future research.

Spatial-temporal clustering of companion animal enteric syndrome: detection and investigation through the use of electronic medical records from participating private practices

SUMMARY There is interest in the potential of companion animal surveillance to provide data to improve pet health and to provide early warning of environmental hazards to people. We implemented a companion animal surveillance system in Calgary, Alberta and the surrounding communities. Informatics technologies automatically extracted electronic medical records from participating veterinary practices and identified cases of enteric syndrome in the warehoused records. The data were analysed using time-series analyses and a retrospective space-time permutation scan statistic. We identified a seasonal pattern of reports of occurrences of enteric syndromes in companion animals and four statistically significant clusters of enteric syndrome cases. The cases within each cluster were examined and information about the animals involved (species, age, sex), their vaccination history, possible exposure or risk behaviour history, information about disease severity, and the aetiological diagnosis was collected. We then assessed whether the cases within the cluster were unusual and if they represented an animal or public health threat. There was often insufficient information recorded in the medical record to characterize the clusters by aetiology or exposures. Space-time analysis of companion animal enteric syndrome cases found evidence of clustering. Collection of more epidemiologically relevant data would enhance the utility of practice-based companion animal surveillance.

Mining free-text medical records for companion animal enteric syndrome surveillance.

Large amounts of animal health care data are present in veterinary electronic medical records (EMR) and they present an opportunity for companion animal disease surveillance. Veterinary patient records are largely in free-text without clinical coding or fixed vocabulary. Text-mining, a computer and information technology application, is needed to identify cases of interest and to add structure to the otherwise unstructured data. In this study EMR's were extracted from veterinary management programs of 12 participating veterinary practices and stored in a data warehouse. Using commercially available text-mining software (WordStat™), we developed a categorization dictionary that could be used to automatically classify and extract enteric syndrome cases from the warehoused electronic medical records. The diagnostic accuracy of the text-miner for retrieving cases of enteric syndrome was measured against human reviewers who independently categorized a random sample of 2500 cases as enteric syndrome positive or negative. Compared to the reviewers, the text-miner retrieved cases with enteric signs with a sensitivity of 87.6% (95%CI, 80.4-92.9%) and a specificity of 99.3% (95%CI, 98.9-99.6%). Automatic and accurate detection of enteric syndrome cases provides an opportunity for community surveillance of enteric pathogens in companion animals.

Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain?

BACKGROUND: Follow-up of abnormal outpatient laboratory test results is a major patient safety concern. Electronic medical records can potentially address this concern through automated notification. We examined whether automated notifications of abnormal laboratory results (alerts) in an integrated electronic medical record resulted in timely follow-up actions. METHODS: We studied 4 alerts: hemoglobin A1c > or =15%, positive hepatitis C antibody, prostate-specific antigen > or =15 ng/mL, and thyroid-stimulating hormone > or =15 mIU/L. An alert tracking system determined whether the alert was acknowledged (ie, provider clicked on and opened the message) within 2 weeks of transmission; acknowledged alerts were considered read. Within 30 days of result transmission, record review and provider contact determined follow-up actions (eg, patient contact, treatment). Multivariable logistic regression models analyzed predictors for lack of timely follow-up. RESULTS: Between May and December 2008, 78,158 tests (hemoglobin A1c, hepatitis C antibody, thyroid-stimulating hormone, and prostate-specific antigen) were performed, of which 1163 (1.48%) were transmitted as alerts; 10.2% of these (119/1163) were unacknowledged. Timely follow-up was lacking in 79 (6.8%), and was statistically not different for acknowledged and unacknowledged alerts (6.4% vs 10.1%; P =.13). Of 1163 alerts, 202 (17.4%) arose from unnecessarily ordered (redundant) tests. Alerts for a new versus known diagnosis were more likely to lack timely follow-up (odds ratio 7.35; 95% confidence interval, 4.16-12.97), whereas alerts related to redundant tests were less likely to lack timely follow-up (odds ratio 0.24; 95% confidence interval, 0.07-0.84). CONCLUSIONS: Safety concerns related to timely patient follow-up remain despite automated notification of non-life-threatening abnormal laboratory results in the outpatient setting.

The application of medical informatics to the veterinary management programs at companion animal practices in Alberta, Canada: a case study.

Companion animals closely share their domestic environment with people and have the potential to, act as sources of zoonotic diseases. They also have the potential to be sentinels of infectious and noninfectious, diseases. With the exception of rabies, there has been minimal ongoing surveillance of, companion animals in Canada. We developed customized data extraction software, the University of, Calgary Data Extraction Program (UCDEP), to automatically extract and warehouse the electronic, medical records (EMR) from participating private veterinary practices to make them available for, disease surveillance and knowledge creation for evidence-based practice. It was not possible to build, generic data extraction software; the UCDEP required customization to meet the specific software, capabilities of the veterinary practices. The UCDEP, tailored to the participating veterinary practices', management software, was capable of extracting data from the EMR with greater than 99%, completeness and accuracy. The experiences of the people developing and using the UCDEP and the, quality of the extracted data were evaluated. The electronic medical record data stored in the data, warehouse may be a valuable resource for surveillance and evidence-based medical research.

Using Informatics and the Electronic Medical Record to Describe Antimicrobial Use in the Clinical Management of Diarrhea Cases at 12 Companion Animal Practices

Antimicrobial drugs may be used to treat diarrheal illness in companion animals. It is important to monitor antimicrobial use to better understand trends and patterns in antimicrobial resistance. There is no monitoring of antimicrobial use in companion animals in Canada. To explore how the use of electronic medical records could contribute to the ongoing, systematic collection of antimicrobial use data in companion animals, anonymized electronic medical records were extracted from 12 participating companion animal practices and warehoused at the University of Calgary. We used the pre-diagnostic, clinical features of diarrhea as the case definition in this study. Using text-mining technologies, cases of diarrhea were described by each of the following variables: diagnostic laboratory tests performed, the etiological diagnosis and antimicrobial therapies. The ability of the text miner to accurately describe the cases for each of the variables was evaluated. It could not reliably classify cases in terms of diagnostic tests or etiological diagnosis; a manual review of a random sample of 500 diarrhea cases determined that 88/500 (17.6%) of the target cases underwent diagnostic testing of which 36/88 (40.9%) had an etiological diagnosis. Text mining, compared to a human reviewer, could accurately identify cases that had been treated with antimicrobials with high sensitivity (92%, 95% confidence interval, 88.1%-95.4%) and specificity (85%, 95% confidence interval, 80.2%-89.1%). Overall, 7400/15,928 (46.5%) of pets presenting with diarrhea were treated with antimicrobials. Some temporal trends and patterns of the antimicrobial use are described. The results from this study suggest that informatics and the electronic medical records could be useful for monitoring trends in antimicrobial use.

Developing a point-of-care electronic medical record system for TB/HIV co-infected patients: experiences from Lighthouse Trust, Lilongwe, Malawi.

BACKGROUND Implementation of user-friendly, real-time, electronic medical records for patient management may lead to improved adherence to clinical guidelines and improved quality of patient care. We detail the systematic, iterative process that implementation partners, Lighthouse clinic and Baobab Health Trust, employed to develop and implement a point-of-care electronic medical records system in an integrated, public clinic in Malawi that serves HIV-infected and tuberculosis (TB) patients. METHODS Baobab Health Trust, the system developers, conducted a series of technical and clinical meetings with Lighthouse and Ministry of Health to determine specifications. Multiple pre-testing sessions assessed patient flow, question clarity, information sequencing, and verified compliance to national guidelines. Final components of the TB/HIV electronic medical records system include: patient demographics; anthropometric measurements; laboratory samples and results; HIV testing; WHO clinical staging; TB diagnosis; family planning; clinical review; and drug dispensing. RESULTS Our experience suggests that an electronic medical records system can improve patient management, enhance integration of TB/HIV services, and improve provider decision-making. However, despite sufficient funding and motivation, several challenges delayed system launch including: expansion of system components to include of HIV testing and counseling services; changes in the national antiretroviral treatment guidelines that required system revision; and low confidence to use the system among new healthcare workers. To ensure a more robust and agile system that met all stakeholder and user needs, our electronic medical records launch was delayed more than a year. Open communication with stakeholders, careful consideration of ongoing provider input, and a well-functioning, backup, paper-based TB registry helped ensure successful implementation and sustainability of the system. Additional, on-site, technical support provided reassurance and swift problem-solving during the extended launch period. CONCLUSION Even when system users are closely involved in the design and development of an electronic medical record system, it is critical to allow sufficient time for software development, solicitation of detailed feedback from both users and stakeholders, and iterative system revisions to successfully transition from paper to point-of-care electronic medical records. For those in low-resource settings, electronic medical records for integrated care is a possible and positive innovation.

Facteurs liés à l’adoption du dossier médical électronique (DME): une étude de cas sur le processus d'implantation d’un DME dans un groupe de médecine de famille.

Dans l’espoir d'améliorer l'efficacité, l’efficience, la qualité et la sécurité des soins de santé, la plupart des pays investissent dans l’informatisation de leur système de santé. Malgré l’octroi de ressources substantielles, les projets d'implantation d’un Dossier médical électronique (DME) font parfois l’objet d’une résistance importante de la part des utilisateurs au moment de leur implantation sur le terrain. Pour expliquer l’adoption d’un DME par les professionnels de la santé, plusieurs modèles théoriques ont été développés et appliqués. Une diversité de facteurs agissant à différents niveaux (individuel, organisationnel et liés à la technologie elle-même) a ainsi été identifiée. L’objectif de cette recherche est d’approfondir les connaissances empiriques quant aux facteurs influençant l’utilisation du DME chez les professionnels de la santé. Le devis de recherche repose sur une étude de cas unique avec douze entrevues et une observation non participante réalisées un an suite au lancement du processus d’implantation d’un DME auprès d’un groupe de médecine famille (GMF) au sein du Centre de santé et de services sociaux du Sud-Ouest-Verdun (Montréal, Canada). Dans le cadre de cette étude, l’analyse a permis l’identification de facilitateurs et de barrières influençant l’adoption du DME. Les facilitateurs étaient l’utilité perçue, la décision du GMF d’implanter le DME, le support de la direction ainsi que la présence de champions et de superutilisateurs. Les barrières les plus importantes étaient l’impact négatif sur le travail clinique, la fragmentation de l’information dans le DME ainsi que les problèmes liés à l’infrastructure technique. Cette connaissance permettra d’alimenter des stratégies visant à mieux répondre aux défis suscités par l’implantation du dossier médical électronique.