Controlled Software Evaluation Methodology: Case Simulation and Cognitive User Analysis

Software for use with patient records is challenging to design and difficult to evaluate because of the tremendous variability of patient circumstances. A method was devised by the authors to overcome a number of difficulties. The method evaluates and compares objectively various software products for use in emergency departments and compares software to conventional methods like dictation and templated chart forms. The technique utilizes oral case simulation and video recording for analysis. The methodology and experiences of executing a study using this case simulation are discussed in this presentation.

Application of Cognitive Engineering Principles to the Redesign of a Dichotomous Identification Key for Parasitology

Dichotomous identification keys are used throughout biology for identification of plants, insects, and parasites. However, correct use of identification keys can be difficult as they are not usually intended for novice users who may not be familiar with the terminology used or with the morphology of the organism being identified. Therefore, we applied cognitive engineering principles to redesign a parasitology identification key for the Internet. We addressed issues of visual clutter and spatial distance by displaying a single question couplet at a time and by switching to the appropriate next couplet after the user made a choice. Our analysis of the original paper-based key versus the Web-based approach found that of 26 applicable cognitive engineering principles, the paper key did not meet 4 (15%) and partially met 11 (42%). In contrast, the redesigned key met 100% of 32 applicable cognitive engineering principles.

COGNITIVE IMPACT OF INTERACTIVE MULTIMEDIA

Technology has been gradually introduced in heath education. One of the most attractive features of this technology-based education is the use of multimedia. In this article we explore the research evidence about the role that multimedia is playing in education. From that analysis we describe the most relevant features of this technology to prepare a common ground of discussion about the evaluation of its impact on educational outcomes. As part of this analysis, we organize current research evidence on the use of technology in medical education, distinguishing diverse variables involved in the process, like knowledge (declarative, procedural), learner characteristics, curricular scenario, etc. This article presents an overview of the Distributed Representations theory and its relationship with research on educational outcomes and multimedia. Next we discuss the relationship between media and diverse learning theories, proposing a theory based taxonomy for educational multimedia.

An Extended Hierarchical Task Analysis for Error Prediction in Medical Devices

This paper introduces an extended hierarchical task analysis (HTA) methodology devised to evaluate and compare user interfaces on volumetric infusion pumps. The pumps were studied along the dimensions of overall usability and propensity for generating human error. With HTA as our framework, we analyzed six pumps on a variety of common tasks using Norman’s Action theory. The introduced method of evaluation divides the problem space between the external world of the device interface and the user’s internal cognitive world, allowing for predictions of potential user errors at the human-device level. In this paper, one detailed analysis is provided as an example, comparing two different pumps on two separate tasks. The results demonstrate the inherent variation, often the cause of usage errors, found with infusion pumps being used in hospitals today. The reported methodology is a useful tool for evaluating human performance and predicting potential user errors with infusion pumps and other simple medical devices.

Generalized resistance resources of informal caregivers: A study of demands for care, competing demands, perception of burden and cognitive stress

This study examines the relationship among psychological resources (generalized resistance resources), care demands (demands for care, competing demands, perception of burden) and cognitive stress in a selected population of primary family caregivers. The study utilizes Antonovsky's Salutogenic Model of Health, specifically the concept of generalized resistance resources (GRRs), to analyze the relative effect of these resources on mediating cognitive stress, controlling for other care demands. The study is based on a sample of 784 eligible caregivers who (1) were relatives, (2) had the main responsibility for care, defined as a primary caregiver, and (3) provided a scaled stress score for the amount of overall care given to the care recipient (family member). The sample was drawn from the 1982 National Long-Term Care Survey (NLTCS) of individuals who assisted a given NLTCS sample person with ADL limitations.^ The study tests the following hypotheses: (a) There will be a negative relationship between generalized resistance resources (GRRs) and cognitive stress controlling for care demands (demands for care, competing demands, and perceptions of burden); (b) of the specific GRRs (material, cognitive, social, cultural-environmental) the social domain will represent the most significant factor predicting a decrease in cognitive stress; and (c) the social domain will be more significant for the female than the male primary family caregiver in decreasing cognitive stress.^ The study found that GRRs had a statistically significant mediating effect on cognitive stress, but the GRRs were a less significant predictor of stress than perception of burden and demands for care. Thus, although the analysis supported the underlying hypothesis, the specific hypothesis regarding GRRs' greater significance in buffering cognitive stress was not supported. Second, the results did not demonstrate the statistical significance or differences among the GRR domains. The hypothesis that the social GRR domain was most significant in mediating stress of family caregivers was not supported. Finally, the results confirmed that there are differences in the importance of social support help in mediating stress based on gender. It was found that gender and social support help were related to cognitive stress and gender had a statistically significant interaction effect with social support help. Implications for clinical practice, public health policy, and research are discussed. ^

The Texas driver responsibility program: A preliminary analysis of the impact on impaired driving and trauma system funding

Objective. In 2003, the State of Texas instituted the Driver Responsibility Program (TDRP), a program consisting of a driving infraction point system coupled with a series of graded fines and annual surcharges for specific traffic violations such as driving while intoxicated (DWI). Approximately half of the revenues generated are earmarked to be disbursed to the state's trauma system to cover uncompensated trauma care costs. This study examined initial program implementation, the impact of trauma system funding, and initial impact on impaired driving knowledge, attitudes and behaviors. A model for targeted media campaigns to improve the program's deterrence effects was developed. ^ Methods. Data from two independent driver survey samples (conducted in 1999 and 2005), department of public safety records, state health department data and a state auditor's report were used to evaluate the program's initial implementation, impact and outcome with respect to drivers' impaired driving knowledge, attitudes and behavior (based on constructs of social cognitive theory) and hospital uncompensated trauma care funding. Survey results were used to develop a regression model of high risk drivers who should be targeted to improve program outcome with respect to deterring impaired driving. ^ Results. Low driver compliance with fee payment (28%) and program implementation problems were associated with lower surcharge revenues in the first two years ($59.5 million versus $525 million predicted). Program revenue distribution to trauma hospitals was associated with a 16% increase in designated trauma centers. Survey data demonstrated that only 28% of drivers are aware of the TDRP and that there has been no initial impact on impaired driving behavior. Logistical regression modeling suggested that target media campaigns highlighting the likelihood of DWI detection by law enforcement and the increased surcharges associated with the TDRP are required to deter impaired driving. ^ Conclusions. Although the TDRP raised nearly $60 million in surcharge revenue for the Texas trauma system over the first two years, this study did not find evidence of a change in impaired driving knowledge, attitudes or behaviors from 1999 to 2005. Further research is required to measure whether the program is associated with decreased alcohol-related traffic fatalities. ^

Use of cognitive interviewing in the development of a Spanish language genetic knowledge instrument

Study purpose. Genetic advances are significantly impacting healthcare, yet recent studies of ethnic group participation in genetic services demonstrate low utilization rates by Latinos. Limited genetic knowledge is a major barrier. The purpose of this study was to field test items in a Spanish-language instrument that will be used to measure genetic knowledge relevant to type 2 diabetes among members of the ethnically heterogeneous U.S. Latino community. Accurate genetic knowledge measurement can provide the foundation for interventions to enhance genetic service utilization. ^ Design. Three waves of cognitive interviews were conducted in Spanish to field test 44 instrument items Thirty-six Latinos, with 12 persons representative of Mexican, Central and South American, and Cuban heritage participated, including 7 males and 29 females between 22 and 60 years of age; 17 participants had 12 years or less of education. ^ Methods. Text narratives from transcriptions of audiotaped interviews were qualitatively analyzed using a coding strategy to indicate potential sources of response error. Through an iterative process of instrument refinement, codes that emerged from the data were used to guide item revisions at the conclusion of each phase; revised items were examined in subsequent interview waves. ^ Results. Inter-cultural and cross-cultural themes associated with difficulties in interpretation and grammatical structuring of items were identified; difficulties associated with comprehension reflected variations in educational level. Of the original 44 items, 32 were retained, 89% of which were revised. Six additional items reflective of cultural knowledge were constructed, resulting in a 38-item instrument. ^ Conclusions. Use of cognitive interviewing provided a valuable tool for detecting both potential sources of response error and cultural variations in these sources. Analysis of interview data guided successive instrument revisions leading to improved item interpretability and comprehension. Although testing in a larger sample will be essential to test validity and reliability, the outcome of field testing suggests initial content validity of a Spanish-language instrument to measure genetic knowledge relative to type 2 diabetes. ^ Keywords. Latinos, genetic knowledge, instrument development, cognitive interviewing ^

Social cognitive theory and factors associated with Mexican-American parents' beliefs, attitudes and practices regarding communication with their adolescent children about sex

Latinos have the highest teen birth rate nationally. Cameron County, Texas is primarily Latino (Mexican-American). This mixed-method study (n=43) examines Mexican-American parents of adolescents' beliefs, attitudes and practices regarding communication with their adolescent children about sex. Social Cognitive Theory (SCT) constructs self-efficacy, behavioral determinism, environment, outcome expectations and reciprocal determinism can be influences on frequency and quality of parent-adolescent sex communication.^ This study describes Mexican-American parents' of adolescents recollections of their own experiences associated with learning about sexuality. It also examines the attitudes and practices regarding communication about sex and the self-efficacy and behavioral capability of participants to teach their adolescent children about sex and sexually transmitted infections. ^ Negative childhood experiences (shame, lies and trauma) of the parents in this study played a key role in terms of their desire to communicate more comprehensively about sexuality with their own children than did their parents. While participants' reported low self-efficacy and behavioral capability to communicate with their adolescent children about sex, they reported relatively high frequency and quality of communication, with 75% of participants receiving a high quality score and over 44% reporting frequent communication with their adolescent children about sex. A Chi square analysis and Fisher's Exact Score revealed no association between acculturation status, gender or having a child who has mothered/fathered a baby and the frequency or quality of communication about sex with adolescent children. Study participants also gave specific recommendations for method, content and setting of sex education for their children and themselves. Promotora delivery of information and education in a comfortable, culturally appropriate neighborhood setting, as well as parent –child learning sessions were identified as possible approaches to address improve self-efficacy and behavioral capability of parents communicating with their adolescent children about sex.^ The results of this analysis provide public health practitioners and interested community entities data to identify and develop interventions that use a theoretical, evidence-based framework for culturally appropriate interventions to encourage and equip Mexican-American parents to effectively communicate with their adolescent children about sexuality, and ultimately to address the high rates of teen pregnancy in this U.S.-Mexico border community. ^

The feasibility of self-report advance directives among elderly patients with cognitive impairment

The central objective of this dissertation was to determine the feasibility of self-completed advance directives (AD) in older persons suffering from mild and moderate stages of dementia. This was accomplished by identifying differences in ability to complete AD among elderly subjects with increasing degrees of dementia and cognitive incompetence. Secondary objectives were to describe and compare advance directives completed by elders and identified proxy decision makers. Secondary objectives were accomplished by measuring the agreement between advance directives completed by proxy and elder, and comparing that agreement across groups defined by the elder's cognitive status. This cross-sectional study employed a structured interview to elicit AD, followed by a similar interview with a proxy decision maker identified by the elder. A stratified sampling scheme recruited elders with normal cognition, mild, and moderate forms of dementia using the Mini Mental-State Exam (MMSE). The Hopkins Competency Assessment Test (HCAT) was used for evaluation of competency to make medical decisions. Analysis was conducted on "between group" (non-demented $\leftrightarrow$ mild dementia $\leftrightarrow$ moderate dementia, and competent $\leftrightarrow$ incompetent) and "within group" (elder $\leftrightarrow$ family member) variation.^ The 118 elderly subjects interviewed were generally male, Caucasian, and of low socioeconomic status. Mean age was 77. Overall, elders preferred a "trial of therapy" regarding AD rather than to "always receive the therapy". No intervention was refused outright more often than it was accepted. A test-retest of elders' AD revealed stable responses. Eleven logic checks measured appropriateness of AD responses independent of preference. No difference was found in logic error rates between elders grouped by MMSE or HCAT. Agreement between proxy and elder responses showed significant dissimilarity, indicating that proxies were not making the same medical decisions as the elders.^ Conclusions based on these data are: (1) Self reporting AD is feasible among elders showing signs of cognitive impairment and they should be given all opportunities to complete advance directives, (2) variation in preferences for advance directives in cognitively impaired elders should not be assumed to be the effects of their impairment alone, (3) proxies do not appear to forego life-prolonging interventions in the face of increasing impairment in their ward, however, their advance directives choices are frequently not those of the elder they represent. ^

UNDERSTANDING NURSE CREATED COGNITIVE ARTIFACTS: PERSONALLY-CREATED-COGNITIVE-ARTIFACTS AS EXTERNAL REPRESENTATIONS OF DISTRIBUTED COGNITION

Manuscript 1: “Conceptual Analysis: Externalizing Nursing Knowledge” We use concept analysis to establish that the report tool nurses prepare, carry, reference, amend, and use as a temporary data repository are examples of cognitive artifacts. This tool, integrally woven throughout the work and practice of nurses, is important to cognition and clinical decision-making. Establishing the tool as a cognitive artifact will support new dimensions of study. Such studies can characterize how this report tool supports cognition, internal representation of knowledge and skills, and external representation of knowledge of the nurse. Manuscript 2: “Research Methods: Exploring Cognitive Work” The purpose of this paper is to describe a complex, cross-sectional, multi-method approach to study of personal cognitive artifacts in the clinical environment. The complex data arrays present in these cognitive artifacts warrant the use of multiple methods of data collection. Use of a less robust research design may result in an incomplete understanding of the meaning, value, content, and relationships between personal cognitive artifacts in the clinical environment and the cognitive work of the user. Manuscript 3: “Making the Cognitive Work of Registered Nurses Visible” Purpose: Knowledge representations and structures are created and used by registered nurses to guide patient care. Understanding is limited regarding how these knowledge representations, or cognitive artifacts, contribute to working memory, prioritization, organization, cognition, and decision-making. The purpose of this study was to identify and characterize the role a specific cognitive artifact knowledge representation and structure as it contributed to the cognitive work of the registered nurse. Methods: Data collection was completed, using qualitative research methods, by shadowing and interviewing 25 registered nurses. Data analysis employed triangulation and iterative analytic processes. Results: Nurse cognitive artifacts support recall, data evaluation, decision-making, organization, and prioritization. These cognitive artifacts demonstrated spatial, longitudinal, chronologic, visual, and personal cues to support the cognitive work of nurses. Conclusions: Nurse cognitive artifacts are an important adjunct to the cognitive work of nurses, and directly support patient care. Nurses need to be able to configure their cognitive artifact in ways that are meaningful and support their internal knowledge representations.