Can prospective usability evaluation predict data errors?

Increasing amounts of clinical research data are collected by manual data entry into electronic source systems and directly from research subjects. For this manual entered source data, common methods of data cleaning such as post-entry identification and resolution of discrepancies and double data entry are not feasible. However data accuracy rates achieved without these mechanisms may be higher than desired for a particular research use. We evaluated a heuristic usability method for utility as a tool to independently and prospectively identify data collection form questions associated with data errors. The method evaluated had a promising sensitivity of 64% and a specificity of 67%. The method was used as described in the literature for usability with no further adaptations or specialization for predicting data errors. We conclude that usability evaluation methodology should be further investigated for use in data quality assurance.

A comparative review of water disinfection methods appropriate for developing countries and their efficacy, cost-efficiency, and usability

Developing countries are heavily burdened by limited access to safe drinking water and subsequent water-related diseases. Numerous water treatment interventions combat this public health crisis, encompassing both traditional and less-common methods. Of these, water disinfection serves as an important means to provide safe drinking water. Existing literature discusses a wide range of traditional treatment options and encourages the use of multi-barrier approaches including coagulation-flocculation, filtration, and disinfection. Most sources do not delve into approaches specifically appropriate for developing countries, nor do they exclusively examine water disinfection methods.^ The objective of this review is to focus on an extensive range of chemical, physio-chemical, and physical water disinfection techniques to provide a compilation, description and evaluation of options available. Such an objective provides further understanding and knowledge to better inform water treatment interventions and explores alternate means of water disinfection appropriate for developing countries. Appropriateness for developing countries corresponds to the effectiveness of an available, easy to use disinfection technique at providing safe drinking water at a low cost.^ Among chemical disinfectants, SWS sodium hypochlorite solution is preferred over sodium hypochlorite bleach due to consistent concentrations. Tablet forms are highly recommended chemical disinfectants because they are effective and very easy to use, but also because they are stable. Examples include sodium dichloroisocyanurate, calcium hypochlorite, and chlorine dioxide, which vary in cost depending on location and availability. Among physio-chemical disinfection options, electrolysis which produces mixed oxidants (MIOX) provides a highly effective disinfection option with a higher upfront cost but very low cost over the long term. Among physical disinfection options, solar disinfection (SODIS) applications are effective, but they treat only a fixed volume of water at a time. They come with higher initial costs but very low on-going costs. Additional effective disinfection techniques may be suitable depending on the location, availability and cost.^

Encouraging adolescents toward mental health careers via website biographies.

This project designed, developed, implemented and is currently evaluating the effectiveness of an interactive, multi-media website designed to encourage adolescents to consider careers in mental health. This Web-based learning environment features biographies of mental health scientists. Evaluation is conducted in a systematic, structured way using cognitive achievement, usability (ease of use), and affective scales (e.g., fun to use) as outcome measures

An Extended Hierarchical Task Analysis for Error Prediction in Medical Devices

This paper introduces an extended hierarchical task analysis (HTA) methodology devised to evaluate and compare user interfaces on volumetric infusion pumps. The pumps were studied along the dimensions of overall usability and propensity for generating human error. With HTA as our framework, we analyzed six pumps on a variety of common tasks using Norman’s Action theory. The introduced method of evaluation divides the problem space between the external world of the device interface and the user’s internal cognitive world, allowing for predictions of potential user errors at the human-device level. In this paper, one detailed analysis is provided as an example, comparing two different pumps on two separate tasks. The results demonstrate the inherent variation, often the cause of usage errors, found with infusion pumps being used in hospitals today. The reported methodology is a useful tool for evaluating human performance and predicting potential user errors with infusion pumps and other simple medical devices.

Strategies to improve healthcare websites.

Healthcare websites that are influential in healthcare decision-making must be evaluated for accuracy, readability and understandability by the average population. Most existing frameworks for designing and evaluating interactive websites focus on the utility and usability of the site. Although these are significant to the design of the basic site, they are not sufficient. We have developed an iterative framework that considers additional attributes.

An innovative approach to impacting student academic achievement and attitudes: Pilot study of the HEADS UP Virtual Molecular Biology Lab

Objective. To evaluate the HEADS UP Virtual Molecular Biology Lab, a computer-based simulated laboratory designed to teach advanced high school biology students how to create a mouse model. ^ Design. A randomized clinical control design of forty-four students from two science magnet high schools in Mercedes, Texas was utilized to assess knowledge and skills of molecular laboratory procedures, attitudes towards science and computers as a learning tool, and usability of the program. ^ Measurements. Data was collected using five paper-and-pencil formatted questionnaires and an internal "lab notebook." ^ Results. The Virtual Lab was found to significantly increase student knowledge over time (p<0.005) and with each use (p<0.001) as well as positively increase attitudes towards computers (p<0.001) and skills (p<0.005). No significant differences were seen in science attitude scores.^ Conclusion. These results provide evidence that the HEADS UP Virtual Molecular Biology Lab is a potentially effective educational tool for high school molecular biology education.^

Assessing fun in serious games: Formative evaluation of games from two diabetes prevention interventions

Purpose. To determine the usability of two video games to prevent type 2 diabetes and obesity among youth through analysis of data collected during alpha-testing. ^ Subjects. Ten children aged 9 to 12 were selected for three 2-hour alpha testing sessions.^ Methods. "Escape from Diab" and "Nanoswarm" were designed to change dietary and physical inactivity behaviors, based on a theoretical framework of mediating variables obtained from social cognitive theory, self-determination theory, elaboration likelihood model, and behavioral inoculation theory. Thirteen mini-games developed by the software company were divided into 3 groups based on completion date. Children tested 4-5 mini-games in each of three sessions. Observed game play was followed by a scripted interview. Results from observation forms and interview transcripts were tabulated and coded to determine usability. Suggestions for game modifications were delivered to the software design firm, and a follow-up table reports rationale for inclusion or exclusion of such modifications.^ Results. Participants were 50% frequent video game players and 20% non game-players. Most (60%) were female. The mean grade (indicating likeability as a subset of usability) across all games given by children was significantly greater than a neutral grade of 80% (89%, p < 0.01), indicating a positive likeability score. The games on average also received positive ratings for fun, helpfulness of instructions and length compared to neutral values (midpoint on likert scales) (all p < 0.01). Observation notes indicated that participants paid attention to the instructions, did not appear to have much difficulty with the games, and were "not frustrated", "not bored", "very engaged", "not fidgety" and "very calm" (all p < 0.01). The primary issues noted in observations and interviews were unclear instructions and unclear purpose of some games. Player suggestions primarily involved ways to make on screen cues more visible or noticeable, instructions more clear, and games more elaborate or difficult.^ Conclusions. The present study highlights the importance of alpha testing video game components for usability prior to completion to enhance usability and likeability. Results indicate that creating clear instructions, making peripheral screen cues more eye-catching or noticeable, and vigorously stating the purpose of the game to improve understandability are important elements. However, future interventions will each present unique materials and user-interfaces and should therefore also be thoroughly alpha-tested. ^

The SanaViz: Human Centered Geovisualization to facilitate visual exploration of Public Health data

These three manuscripts are presented as a PhD dissertation for the study of using GeoVis application to evaluate telehealth programs. The primary reason of this research was to understand how the GeoVis applications can be designed and developed using combined approaches of HC approach and cognitive fit theory and in terms utilized to evaluate telehealth program in Brazil. First manuscript The first manuscript in this dissertation presented a background about the use of GeoVisualization to facilitate visual exploration of public health data. The manuscript covered the existing challenges that were associated with an adoption of existing GeoVis applications. The manuscript combines the principles of Human Centered approach and Cognitive Fit Theory and a framework using a combination of these approaches is developed that lays the foundation of this research. The framework is then utilized to propose the design, development and evaluation of “the SanaViz” to evaluate telehealth data in Brazil, as a proof of concept. Second manuscript The second manuscript is a methods paper that describes the approaches that can be employed to design and develop “the SanaViz” based on the proposed framework. By defining the various elements of the HC approach and CFT, a mixed methods approach is utilized for the card sorting and sketching techniques. A representative sample of 20 study participants currently involved in the telehealth program at the NUTES telehealth center at UFPE, Recife, Brazil was enrolled. The findings of this manuscript helped us understand the needs of the diverse group of telehealth users, the tasks that they perform and helped us determine the essential features that might be necessary to be included in the proposed GeoVis application “the SanaViz”. Third manuscript The third manuscript involved mix- methods approach to compare the effectiveness and usefulness of the HC GeoVis application “the SanaViz” against a conventional GeoVis application “Instant Atlas”. The same group of 20 study participants who had earlier participated during Aim 2 was enrolled and a combination of quantitative and qualitative assessments was done. Effectiveness was gauged by the time that the participants took to complete the tasks using both the GeoVis applications, the ease with which they completed the tasks and the number of attempts that were taken to complete each task. Usefulness was assessed by System Usability Scale (SUS), a validated questionnaire tested in prior studies. In-depth interviews were conducted to gather opinions about both the GeoVis applications. This manuscript helped us in the demonstration of the usefulness and effectiveness of HC GeoVis applications to facilitate visual exploration of telehealth data, as a proof of concept. Together, these three manuscripts represent challenges of combining principles of Human Centered approach, Cognitive Fit Theory to design and develop GeoVis applications as a method to evaluate Telehealth data. To our knowledge, this is the first study to explore the usefulness and effectiveness of GeoVis to facilitate visual exploration of telehealth data. The results of the research enabled us to develop a framework for the design and development of GeoVis applications related to the areas of public health and especially telehealth. The results of our study showed that the varied users were involved with the telehealth program and the tasks that they performed. Further it enabled us to identify the components that might be essential to be included in these GeoVis applications. The results of our research answered the following questions; (a) Telehealth users vary in their level of understanding about GeoVis (b) Interaction features such as zooming, sorting, and linking and multiple views and representation features such as bar chart and choropleth maps were considered the most essential features of the GeoVis applications. (c) Comparing and sorting were two important tasks that the telehealth users would perform for exploratory data analysis. (d) A HC GeoVis prototype application is more effective and useful for exploration of telehealth data than a conventional GeoVis application. Future studies should be done to incorporate the proposed HC GeoVis framework to enable comprehensive assessment of the users and the tasks they perform to identify the features that might be necessary to be a part of the GeoVis applications. The results of this study demonstrate a novel approach to comprehensively and systematically enhance the evaluation of telehealth programs using the proposed GeoVis Framework.