Improving Chat in an Online Graduate Class

Introduction: Foundations of Health Information Sciences I is the first class many students take to introduce them to the field of health informatics. It is completely online, and uses optional weekly text-only chats to provide real time interaction between faculty and students. Chat sessions were very disorganized and difficult to follow, both real time and on the transcript. Research suggests that the disorganization contributes to cognitive load. [See PDF for complete abstract]

Efficacy of quality criteria to identify potentially harmful information: a cross-sectional survey of complementary and alternative medicine web sites.

BACKGROUND: Many users search the Internet for answers to health questions. Complementary and alternative medicine (CAM) is a particularly common search topic. Because many CAM therapies do not require a clinician's prescription, false or misleading CAM information may be more dangerous than information about traditional therapies. Many quality criteria have been suggested to filter out potentially harmful online health information. However, assessing the accuracy of CAM information is uniquely challenging since CAM is generally not supported by conventional literature. OBJECTIVE: The purpose of this study is to determine whether domain-independent technical quality criteria can identify potentially harmful online CAM content. METHODS: We analyzed 150 Web sites retrieved from a search for the three most popular herbs: ginseng, ginkgo and St. John's wort and their purported uses on the ten most commonly used search engines. The presence of technical quality criteria as well as potentially harmful statements (commissions) and vital information that should have been mentioned (omissions) was recorded. RESULTS: Thirty-eight sites (25%) contained statements that could lead to direct physical harm if acted upon. One hundred forty five sites (97%) had omitted information. We found no relationship between technical quality criteria and potentially harmful information. CONCLUSIONS: Current technical quality criteria do not identify potentially harmful CAM information online. Consumers should be warned to use other means of validation or to trust only known sites. Quality criteria that consider the uniqueness of CAM must be developed and validated.

A day in the life of PubMed: analysis of a typical day's query log.

OBJECTIVE: To characterize PubMed usage over a typical day and compare it to previous studies of user behavior on Web search engines. DESIGN: We performed a lexical and semantic analysis of 2,689,166 queries issued on PubMed over 24 consecutive hours on a typical day. MEASUREMENTS: We measured the number of queries, number of distinct users, queries per user, terms per query, common terms, Boolean operator use, common phrases, result set size, MeSH categories, used semantic measurements to group queries into sessions, and studied the addition and removal of terms from consecutive queries to gauge search strategies. RESULTS: The size of the result sets from a sample of queries showed a bimodal distribution, with peaks at approximately 3 and 100 results, suggesting that a large group of queries was tightly focused and another was broad. Like Web search engine sessions, most PubMed sessions consisted of a single query. However, PubMed queries contained more terms. CONCLUSION: PubMed's usage profile should be considered when educating users, building user interfaces, and developing future biomedical information retrieval systems.

Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites.

OBJECTIVES: To determine the characteristics of popular breast cancer related websites and whether more popular sites are of higher quality. DESIGN: The search engine Google was used to generate a list of websites about breast cancer. Google ranks search results by measures of link popularity---the number of links to a site from other sites. The top 200 sites returned in response to the query "breast cancer" were divided into "more popular" and "less popular" subgroups by three different measures of link popularity: Google rank and number of links reported independently by Google and by AltaVista (another search engine). MAIN OUTCOME MEASURES: Type and quality of content. RESULTS: More popular sites according to Google rank were more likely than less popular ones to contain information on ongoing clinical trials (27% v 12%, P=0.01 ), results of trials (12% v 3%, P=0.02), and opportunities for psychosocial adjustment (48% v 23%, P<0.01). These characteristics were also associated with higher number of links as reported by Google and AltaVista. More popular sites by number of linking sites were also more likely to provide updates on other breast cancer research, information on legislation and advocacy, and a message board service. Measures of quality such as display of authorship, attribution or references, currency of information, and disclosure did not differ between groups. CONCLUSIONS: Popularity of websites is associated with type rather than quality of content. Sites that include content correlated with popularity may best meet the public's desire for information about breast cancer.

Preliminary Evaluation of a Guideline Classification System

In a previous paper, we presented a proposed expansion of the National Guideline Clearing-house (NGC) classification1. We performed a preliminary evaluation of the classification based on 100 guidelines randomly selected from the NGC collection. We found that 89 of the 100 guidelines could be assigned to a single guideline category. To test inter-observer agreement, twenty guidelines were also categorized by a second investigator. Agreement was found to be 40-90% depending on the axis, which compares favorably with agreement among MeSH indexers (30-60%)2. We conclude that categorization is feasible. Further research is needed to clarify axes with poor inter-observer agreement.

Ontology driven integration platform for clinical and translational research

Semantic Web technologies offer a promising framework for integration of disparate biomedical data. In this paper we present the semantic information integration platform under development at the Center for Clinical and Translational Sciences (CCTS) at the University of Texas Health Science Center at Houston (UTHSC-H) as part of our Clinical and Translational Science Award (CTSA) program. We utilize the Semantic Web technologies not only for integrating, repurposing and classification of multi-source clinical data, but also to construct a distributed environment for information sharing, and collaboration online. Service Oriented Architecture (SOA) is used to modularize and distribute reusable services in a dynamic and distributed environment. Components of the semantic solution and its overall architecture are described.

Accuracy and self correction of information received from an internet breast cancer list: content analysis.

OBJECTIVES: To determine the prevalence of false or misleading statements in messages posted by internet cancer support groups and whether these statements were identified as false or misleading and corrected by other participants in subsequent postings. DESIGN: Analysis of content of postings. SETTING: Internet cancer support group Breast Cancer Mailing List. MAIN OUTCOME MEASURES: Number of false or misleading statements posted from 1 January to 23 April 2005 and whether these were identified and corrected by participants in subsequent postings. RESULTS: 10 of 4600 postings (0.22%) were found to be false or misleading. Of these, seven were identified as false or misleading by other participants and corrected within an average of four hours and 33 minutes (maximum, nine hours and nine minutes). CONCLUSIONS: Most posted information on breast cancer was accurate. Most false or misleading statements were rapidly corrected by participants in subsequent postings.

Quality of weight loss advice on internet forums.

BACKGROUND: Adults use the Internet for weight loss information, sometimes by participating in discussion forums. Our purpose was to analyze the quality of advice exchanged on these forums. METHODS: This was a retrospective analysis of messages posted to 18 Internet weight loss forums during 1 month in 2006. Advice was evaluated for congruence with clinical guidelines; potential for causing harm; and subsequent correction when it was contradictory to guidelines (erroneous) or potentially harmful. Message- and forum-specific characteristics were evaluated as predictors of advice quality and self-correction. RESULTS: Of 3368 initial messages, 266 (7.9%) were requests for advice. Of 654 provisions of advice, 56 (8.6%) were erroneous and 19 of these 56 (34%) were subsequently corrected. Forty-three (6.6%) provisions of advice were harmful, and 12 of these 43 (28%) were subsequently corrected. Messages from low-activity forums (fewer messages) were more likely than those from high-activity forums to be erroneous (10.6% vs 2.4%, P < .001) or harmful (8.4% vs 1.2%, P < .001). In high-activity forums, 2 of 4 (50%) erroneous provisions of advice and 2 of 2 (100%) potentially harmful provisions of advice were corrected by subsequent postings. Compared with general weight loss advice, medication-related advice was more likely to be erroneous (P = .02) or harmful (P = .01). CONCLUSIONS: Most advice posted on highly active Internet weight loss forums is not erroneous or harmful. However, clinical and research strategies are needed to address the quality of medication-related advice.

Using citation data to improve retrieval from MEDLINE.

OBJECTIVE: To determine whether algorithms developed for the World Wide Web can be applied to the biomedical literature in order to identify articles that are important as well as relevant. DESIGN AND MEASUREMENTS A direct comparison of eight algorithms: simple PubMed queries, clinical queries (sensitive and specific versions), vector cosine comparison, citation count, journal impact factor, PageRank, and machine learning based on polynomial support vector machines. The objective was to prioritize important articles, defined as being included in a pre-existing bibliography of important literature in surgical oncology. RESULTS Citation-based algorithms were more effective than noncitation-based algorithms at identifying important articles. The most effective strategies were simple citation count and PageRank, which on average identified over six important articles in the first 100 results compared to 0.85 for the best noncitation-based algorithm (p < 0.001). The authors saw similar differences between citation-based and noncitation-based algorithms at 10, 20, 50, 200, 500, and 1,000 results (p < 0.001). Citation lag affects performance of PageRank more than simple citation count. However, in spite of citation lag, citation-based algorithms remain more effective than noncitation-based algorithms. CONCLUSION Algorithms that have proved successful on the World Wide Web can be applied to biomedical information retrieval. Citation-based algorithms can help identify important articles within large sets of relevant results. Further studies are needed to determine whether citation-based algorithms can effectively meet actual user information needs.

Persuasive email messages for patient communication.

To improve health and reduce costs, we need to encourage patients to make better health care decisions. Since email is widely available, it may be useful for patient-directed interventions. However, we know little about how the contents of an email message can influence a health-related decision. We propose a model to understand how patients may process persuasive email messages.

Using incomplete citation data for MEDLINE results ranking.

Information overload is a significant problem for modern medicine. Searching MEDLINE for common topics often retrieves more relevant documents than users can review. Therefore, we must identify documents that are not only relevant, but also important. Our system ranks articles using citation counts and the PageRank algorithm, incorporating data from the Science Citation Index. However, citation data is usually incomplete. Therefore, we explore the relationship between the quantity of citation information available to the system and the quality of the result ranking. Specifically, we test the ability of citation count and PageRank to identify "important articles" as defined by experts from large result sets with decreasing citation information. We found that PageRank performs better than simple citation counts, but both algorithms are surprisingly robust to information loss. We conclude that even an incomplete citation database is likely to be effective for importance ranking.

Evaluating the prevalence, content and readability of complementary and alternative medicine (CAM) web pages on the internet.

Complementary and alternative medicine (CAM) use is growing rapidly. As CAM is relatively unregulated, it is important to evaluate the type and availability of CAM information. The goal of this study is to deter-mine the prevalence, content and readability of online CAM information based on searches for arthritis, diabetes and fibromyalgia using four common search engines. Fifty-eight of 599 web pages retrieved by a "condition search" (9.6%) were CAM-oriented. Of 216 CAM pages found by the "condition" and "condition + herbs" searches, 78% were authored by commercial organizations, whose pur-pose involved commerce 69% of the time and 52.3% had no references. Although 98% of the CAM information was intended for consumers, the mean read-ability was at grade level 11. We conclude that consumers searching the web for health information are likely to encounter consumer-oriented CAM advertising, which is difficult to read and is not supported by the conventional literature.

Using incomplete citation data for MEDLINE results ranking.

Information overload is a significant problem for modern medicine. Searching MEDLINE for common topics often retrieves more relevant documents than users can review. Therefore, we must identify documents that are not only relevant, but also important. Our system ranks articles using citation counts and the PageRank algorithm, incorporating data from the Science Citation Index. However, citation data is usually incomplete. Therefore, we explore the relationship between the quantity of citation information available to the system and the quality of the result ranking. Specifically, we test the ability of citation count and PageRank to identify "important articles" as defined by experts from large result sets with decreasing citation information. We found that PageRank performs better than simple citation counts, but both algorithms are surprisingly robust to information loss. We conclude that even an incomplete citation database is likely to be effective for importance ranking.

Comparing communication technology on Chinese, English, and Spanish diabetes web sites.

Technological and cultural factors influence access to health information on the web in multifarious ways. We evaluated structural differences and availability of communication services on the web in three diverse language and cultural groups: Chinese, English, and Spanish. A total of 382 web sites were analyzed: 144 were English language sites (38%), 129 were Chinese language sites (34%), and 108 were Spanish language sites (28%). We did not find technical differences in the number of outgoing links per domain or the total availability of communication services between the three groups. There were differences in the distribution of available services between Chinese and English sites. In the Chinese sites, there were more communication services between consumers and health experts. Our results suggest that the health-related web presence of these three cultural groups is technologically comparable, but reflects differences that may be attributable to cultural factors.

Explanations for unsuccessful weight loss among bariatric surgery candidates.

BACKGROUND: Our objective was to analyze subjective explanations for unsuccessful weight loss among bariatric surgery candidates. METHODS: This was a retrospective analysis of 909 bariatric surgery candidates (78.2% female, average body mass index [BMI] 47.3) at a university center from 2001 to April 2007 who answered an open-ended question about why they were unable to lose weight. We generated a coding scheme for answers to the question and established inter-rater reliability of the coding process. Associations with demographic parameters and initial BMI were tested. RESULTS: The most common categories of answers were nonspecific explanations related to diet (25.3%), physical activity (21.0%), or motivation (19.7%), followed by diet-related motivation (12.7%) and medical conditions or medications affecting physical activity (12.7%). Categories related to time, financial cost, social support, physical environment, and knowledge occurred in less than 4% each. Men were more likely than women to cite a medical condition or medication affecting physical activity (19.2% vs 10.8%, P = 0.002, odds ratio [OR] = 1.96, 95% confidence interval [CI] = 1.28-2.99) but less likely to cite diet-related motivation (7.1% vs 14.2%, P = 0.008, OR = 0.46, 95% CI = 0.26-0.82). CONCLUSIONS: Our findings suggest that addressing diet, physical activity, and motivation in a comprehensive approach would meet the stated needs of obese patients. Raising patient awareness of under-recognized barriers to weight loss, such as the physical environment and lack of social support, should also be considered. Lastly, anticipating gender-specific attributions may facilitate tailoring of interventions.