Limitations of the Odds Ratio in Gauging the Performance of a Diagnostic or Prognostic Marker

A marker that is strongly associated with outcome (or disease) is often assumed to be effective for classifying individuals according to their current or future outcome. However, for this to be true, the associated odds ratio must be of a magnitude rarely seen in epidemiological studies. An illustration of the relationship between odds ratios and receiver operating characteristic (ROC) curves shows, for example, that a marker with an odds ratio as high as 3 is in fact a very poor classification tool. If a marker identifies 10 percent of controls as positive (false positives) and has an odds ratio of 3, then it will only correctly identify 25 percent of cases as positive (true positives). Moreover, the authors illustrate that a single measure of association such as an odds ratio does not meaningfully describe a marker’s ability to classify subjects. Appropriate statistical methods for assessing and reporting the classification power of a marker are described. The serious pitfalls of using more traditional methods based on parameters in logistic regression models are illustrated.

A Diagnostic Test for the Mixing Distribution in a Generalised Linear Mixed Model

We introduce a diagnostic test for the mixing distribution in a generalised linear mixed model. The test is based on the difference between the marginal maximum likelihood and conditional maximum likelihood estimates of a subset of the fixed effects in the model. We derive the asymptotic variance of this difference, and propose a test statistic that has a limiting chi-square distribution under the null hypothesis that the mixing distribution is correctly specified. For the important special case of the logistic regression model with random intercepts, we evaluate via simulation the power of the test in finite samples under several alternative distributional forms for the mixing distribution. We illustrate the method by applying it to data from a clinical trial investigating the effects of hormonal contraceptives in women.

On the Behaviour of Marginal and Conditional Akaike Information Criteria in Linear Mixed Models

In linear mixed models, model selection frequently includes the selection of random effects. Two versions of the Akaike information criterion (AIC) have been used, based either on the marginal or on the conditional distribution. We show that the marginal AIC is no longer an asymptotically unbiased estimator of the Akaike information, and in fact favours smaller models without random effects. For the conditional AIC, we show that ignoring estimation uncertainty in the random effects covariance matrix, as is common practice, induces a bias that leads to the selection of any random effect not predicted to be exactly zero. We derive an analytic representation of a corrected version of the conditional AIC, which avoids the high computational cost and imprecision of available numerical approximations. An implementation in an R package is provided. All theoretical results are illustrated in simulation studies, and their impact in practice is investigated in an analysis of childhood malnutrition in Zambia.

RANDOM EFFECTS MODELS IN A META-ANALYSIS OF THE ACCURACY OF DIAGNOSTIC TESTS WITHIN A GOLD STANDARD IN THE PRESENCE OF MISSING DATA

In evaluating the accuracy of diagnosis tests, it is common to apply two imperfect tests jointly or sequentially to a study population. In a recent meta-analysis of the accuracy of microsatellite instability testing (MSI) and traditional mutation analysis (MUT) in predicting germline mutations of the mismatch repair (MMR) genes, a Bayesian approach (Chen, Watson, and Parmigiani 2005) was proposed to handle missing data resulting from partial testing and the lack of a gold standard. In this paper, we demonstrate an improved estimation of the sensitivities and specificities of MSI and MUT by using a nonlinear mixed model and a Bayesian hierarchical model, both of which account for the heterogeneity across studies through study-specific random effects. The methods can be used to estimate the accuracy of two imperfect diagnostic tests in other meta-analyses when the prevalence of disease, the sensitivities and/or the specificities of diagnostic tests are heterogeneous among studies. Furthermore, simulation studies have demonstrated the importance of carefully selecting appropriate random effects on the estimation of diagnostic accuracy measurements in this scenario.

Quality of analgesic treatment in patients with advanced prostate cancer: do we do a better job now? The Swiss Group for Clinical Cancer Research (SAKK) experience

GOALS OF WORK: The aim of this study was to evaluate pain intensity and the application of the WHO guidelines for cancer pain treatment in patients with prostate cancer treated at Swiss cancer centers. MATERIALS AND METHODS: We analyzed a series of five multicenter phase II clinical trials which examined the palliative effect of different chemotherapies in patients with advanced hormone-refractory prostate carcinoma. Of 170 patients, 1,018 visits were evaluable for our purpose, including ratings of pain intensity by patients and prescribed analgesics. MAIN RESULTS: No or mild pain was indicated by patients in 36 to 55% of the visits, more than mild pain in 30 to 46%. In 21% of the visits, the WHO pain treatment criteria (treatment according to one of the three steps; oral, rectal or transdermal application of the main dose; administration on a regular schedule) were fulfilled, and the Cleeland index was positive according to all recommendations. In 6% of the visits, neither the WHO criteria were fulfilled nor was the Cleeland index positive. This indicates insufficient pain treatment not following the WHO guidelines and that the prescribed analgesics were not sufficiently potent for the rated pain intensity. CONCLUSIONS: In this selective Swiss sample, the standard of analgesic treatment is high. However, there is still scope for improvement. This cannot solely be solved by improving the knowledge of the physicians. Programs to change the patients' attitude towards cancer pain, training to improve the physicians' communication skills, and institutional changes may be promising strategies.

A review identifies and classifies reasons for ordering diagnostic tests

OBJECTIVE: To consider the reasons and context for test ordering by doctors when faced with an undiagnosed complaint in primary or secondary care. STUDY DESIGN AND SETTING: We reviewed any study of any design that discussed factors that may affect a doctor's decision to order a test. Articles were located through searches of electronic databases, authors' files on diagnostic methodology, and reference lists of relevant studies. We extracted data on: study design, type of analysis, setting, topic area, and any factors reported to influence test ordering. RESULTS: We included 37 studies. We carried out a thematic analysis to synthesize data. Five key groupings arose from this process: diagnostic factors, therapeutic and prognostic factors, patient-related factors, doctor-related factors, and policy and organization-related factors. To illustrate how the various factors identified may influence test ordering we considered the symptom low back pain and the diagnosis multiple sclerosis as examples. CONCLUSIONS: A wide variety of factors influence a doctor's decision to order a test. These are integral to understanding diagnosis in clinical practice. Traditional diagnostic accuracy studies should be supplemented with research into the broader context in which doctors perform their work.

Rosette-forming glioneuronal tumor of the fourth ventricle: Report of two cases with a differential diagnostic overview

We report on clinicopathological findings in two cases of rosette-forming glioneuronal tumor of the fourth ventricle (RGNT) occurring in females aged 16 years (Case 1) and 30 years (Case 2). Symptoms included vertigo, nausea, cerebellar ataxia, as well as headaches, and had been present for 4-months and 1 week, respectively. Magnetic resonance imaging (MRI) indicated a cerebellar-based tumor of 1.8cm (Case 1) and 5cm (Case 2) diameter each, bulging into the fourth ventricle. Case 2 involved a cyst-mural-nodule configuration. In both instances, the solid component appeared isointense on T(1) sequences, hyperintense in the T(2) mode, and enhanced moderately. Gross total resection was achieved via suboccipital craniotomy. However, functional recovery was disappointing in Case 1. On microscopy, both tumors comprised an admixture of low-grade astrocytoma interspersed with circular aggregates of synaptophysin-expressing round cells harboring oligodendrocyte-like nuclei. The astrocytic moiety in Case 1 was nondescript, and overtly pilocytic in Case 2. The architecture of neuronal elements variously consisted of neurocytic rosettes, of pseudorosettes centered on a capillary core, as well as of concentric ribbons along irregular lumina. Gangliocytic maturation, especially "floating neurons", or a corresponding immunoreactivity for neurofilament protein was absent. Neither of these populations exhibited atypia, mitotic activity, or a significant labeling for MIB-1. Cerebellar parenchyma included in the surgical specimen did not reveal any preexisting malformative anomaly. Despite sharing some overlapping histologic traits with dysembryoplastic neuroepithelial tumor (DNT), the presentation of RGNT with respect to both patient age and location is consistent enough for this lesion to be singled out as an autonomous entity.

Interactional behaviour as a marker for screening patients with environment-related complaints

BACKGROUND: Adequate assessment of symptoms of patients suffering from environmental illnesses requires appropriate procedures such as psychological and psychiatric diagnostics, medical screening and a thorough analysis of noxious environmental factors. The Basel pilot research project established a multi-methodological assessment procedure that meets these criteria. However, an exhaustive three-fold analysis is very costly in terms of both equipment and personnel, and hence the need for a heuristic approach and pre-screening persists. METHOD: The three-fold diagnostic approach was preceded by a structured psychodynamic interview; the findings were used to construct a new profile of the patient's interactional behaviour (IB) in conjunction with the interviewer's countertransference. The extent to which this new profile could predict the results of the multi-method assessment was then assessed. RESULTS: A low level of IB on the part of the patient significantly predicted the degree of stress and the extent of the psychiatric diagnosis, including personality disorders. A negative IB was associated with negative personality traits. Furthermore, a high level of IB implied more medical, but not more environmental, findings which could plausibly be related to the patient's complaints. CONCLUSIONS: Assessment of patients' IB in conjunction with one's own countertransference is very helpful as a preliminary heuristic approach and may lead to consequences for treatment and therapy. Therefore, the training provided for experts who deal with patients suffering from environment-related complaints should place more specific emphasis on assessing patients' behaviour and on incorporating information gathered from countertransference. Nevertheless, an interdisciplinary assessment including medical, psychological/psychiatric, and environmental expertise remains mandatory for adequate and satisfactory diagnosis of patients with environment-related complaints.

Design, construction, and testing of a research quality electric machine testbed

With energy demands and costs growing every day, the need for improving energy efficiency in electrical devices has become very important. Research into various methods of improving efficiency for all electrical components will be a key to meet future energy needs. This report documents the design, construction, and testing of a research quality electric machine dynamometer and test bed. This test cell system can be used for research in several areas including: electric drives systems, electric vehicle propulsion systems, power electronic converters, load/source element in an AC Microgrid, as well as many others. The test cell design criteria, and decisions, will be discussed in reference to user functionality and flexibility. The individual power components will be discussed in detail to how they relate to the project, highlighting any feature used in operation of the test cell. A project timeline will be discussed, clearly stating the work done by the different individuals involved in the project. In addition, the system will be parameterized and benchmark data will be used to provide the functional operation of the system. With energy demands and costs growing every day, the need for improving energy efficiency in electrical devices has become very important. Research into various methods of improving efficiency for all electrical components will be a key to meet future energy needs. This report documents the design, construction, and testing of a research quality electric machine dynamometer and test bed. This test cell system can be used for research in several areas including: electric drives systems, electric vehicle propulsion systems, power electronic converters, load/source element in an AC Microgrid, as well as many others. The test cell design criteria, and decisions, will be discussed in reference to user functionality and flexibility. The individual power components will be discussed in detail to how they relate to the project, highlighting any feature used in operation of the test cell. A project timeline will be discussed, clearly stating the work done by the different individuals involved in the project. In addition, the system will be parameterized and benchmark data will be used to provide the functional operation of the system.

Community managed rural water supply systems in the Dominican Republic : assessment of sustainability of systems built by the National Institute of Potable Water and Peace Corps, Dominican Republic

In the Dominican Republic economic growth in the past twenty years has not yielded sufficient improvement in access to drinking water services, especially in rural areas where 1.5 million people do not have access to an improved water source (WHO, 2006). Worldwide, strategic development planning in the rural water sector has focused on participatory processes and the use of demand filters to ensure that service levels match community commitment to post-project operation and maintenance. However studies have concluded that an alarmingly high percentage of drinking water systems (20-50%) do not provide service at the design levels and/or fail altogether (up to 90%): BNWP (2009), Annis (2006), and Reents (2003). World Bank, USAID, NGOs, and private consultants have invested significant resources in an effort to determine what components make up an “enabling environment” for sustainable community management of rural water systems (RWS). Research has identified an array of critical factors, internal and external to the community, which affect long term sustainability of water services. Different frameworks have been proposed in order to better understand the linkages between individual factors and sustainability of service. This research proposes a Sustainability Analysis Tool to evaluate the sustainability of RWS, adapted from previous relevant work in the field to reflect the realities in the Dominican Republic. It can be used as a diagnostic tool for government entities and development organizations to characterize the needs of specific communities and identify weaknesses in existing training regimes or support mechanisms. The framework utilizes eight indicators in three categories (Organization/Management, Financial Administration, and Technical Service). Nineteen independent variables are measured resulting in a score of sustainability likely (SL), possible (SP), or unlikely (SU) for each of the eight indicators. Thresholds are based upon benchmarks from the DR and around the world, primary data collected during the research, and the author’s 32 months of field experience. A final sustainability score is calculated using weighting factors for each indicator, derived from Lockwood (2003). The framework was tested using a statistically representative geographically stratified random sample of 61 water systems built in the DR by initiatives of the National Institute of Potable Water (INAPA) and Peace Corps. The results concluded that 23% of sample systems are likely to be sustainable in the long term, 59% are possibly sustainable, and for 18% it is unlikely that the community will be able to overcome any significant challenge. Communities that were scored as unlikely sustainable perform poorly in participation, financial durability, and governance while the highest scores were for system function and repair service. The Sustainability Analysis Tool results are verified by INAPA and PC reports, evaluations, and database information, as well as, field observations and primary data collected during the surveys. Future research will analyze the nature and magnitude of relationships between key factors and the sustainability score defined by the tool. Factors include: gender participation, legal status of water committees, plumber/operator remuneration, demand responsiveness, post construction support methodologies, and project design criteria.

Graphical presentation of diagnostic information

BACKGROUND: Graphical displays of results allow researchers to summarise and communicate the key findings of their study. Diagnostic information should be presented in an easily interpretable way, which conveys both test characteristics (diagnostic accuracy) and the potential for use in clinical practice (predictive value). METHODS: We discuss the types of graphical display commonly encountered in primary diagnostic accuracy studies and systematic reviews of such studies, and systematically review the use of graphical displays in recent diagnostic primary studies and systematic reviews. RESULTS: We identified 57 primary studies and 49 systematic reviews. Fifty-six percent of primary studies and 53% of systematic reviews used graphical displays to present results. Dot-plot or box-and- whisker plots were the most commonly used graph in primary studies and were included in 22 (39%) studies. ROC plots were the most common type of plot included in systematic reviews and were included in 22 (45%) reviews. One primary study and five systematic reviews included a probability-modifying plot. CONCLUSION: Graphical displays are currently underused in primary diagnostic accuracy studies and systematic reviews of such studies. Diagnostic accuracy studies need to include multiple types of graphic in order to provide both a detailed overview of the results (diagnostic accuracy) and to communicate information that can be used to inform clinical practice (predictive value). Work is required to improve graphical displays, to better communicate the utility of a test in clinical practice and the implications of test results for individual patients.

Strategies to advance translational research into brain barriers

There is a paucity of therapies for most neurological disorders--from rare lysosomal storage diseases to major public health concerns such as stroke and Alzheimer's disease. Advances in the targeting of drugs to the CNS are essential for the future success of neurotherapeutics; however, the delivery of many potentially therapeutic and diagnostic compounds to specific areas of the brain is restricted by the blood-brain barrier, the blood-CSF barrier, or other specialised CNS barriers. These brain barriers are now recognised as a major obstacle to the treatment of most brain disorders. The challenge to deliver therapies to the CNS is formidable, and the solution will require concerted international efforts among academia, government, and industry. At a recent meeting of expert panels, essential and high-priority recommendations to propel brain barrier research forward in six topical areas were developed and these recommendations are presented here.

Individual patient data meta-analysis of diagnostic and prognostic studies in obstetrics, gynaecology and reproductive medicine

BACKGROUND: In clinical practice a diagnosis is based on a combination of clinical history, physical examination and additional diagnostic tests. At present, studies on diagnostic research often report the accuracy of tests without taking into account the information already known from history and examination. Due to this lack of information, together with variations in design and quality of studies, conventional meta-analyses based on these studies will not show the accuracy of the tests in real practice. By using individual patient data (IPD) to perform meta-analyses, the accuracy of tests can be assessed in relation to other patient characteristics and allows the development or evaluation of diagnostic algorithms for individual patients. In this study we will examine these potential benefits in four clinical diagnostic problems in the field of gynaecology, obstetrics and reproductive medicine. METHODS/DESIGN: Based on earlier systematic reviews for each of the four clinical problems, studies are considered for inclusion. The first authors of the included studies will be invited to participate and share their original data. After assessment of validity and completeness the acquired datasets are merged. Based on these data, a series of analyses will be performed, including a systematic comparison of the results of the IPD meta-analysis with those of a conventional meta-analysis, development of multivariable models for clinical history alone and for the combination of history, physical examination and relevant diagnostic tests and development of clinical prediction rules for the individual patients. These will be made accessible for clinicians. DISCUSSION: The use of IPD meta-analysis will allow evaluating accuracy of diagnostic tests in relation to other relevant information. Ultimately, this could increase the efficiency of the diagnostic work-up, e.g. by reducing the need for invasive tests and/or improving the accuracy of the diagnostic workup. This study will assess whether these benefits of IPD meta-analysis over conventional meta-analysis can be exploited and will provide a framework for future IPD meta-analyses in diagnostic and prognostic research.

The role of psychological stress in inflammatory bowel disease: quality assessment of methods of 18 prospective studies and suggestions for future research

BACKGROUND: Enquiries among patients on the one hand and experimental and observational studies on the other suggest an influence of stress on inflammatory bowel diseases (IBD). However, since this influence remains hypothetical, further research is essential. We aimed to devise recommendations for future investigations in IBD by means of scrutinizing previously applied methodology. METHODS: We critically reviewed prospective clinical studies on the effect of psychological stress on IBD. Eligible studies were searched by means of the PubMed electronic library and through checking the bibliographies of located sources. RESULTS: We identified 20 publications resulting from 18 different studies. Sample sizes ranged between 10 and 155 participants. Study designs in terms of patient assessment, control variables, and applied psychometric instruments varied substantially across studies. Methodological strengths and weaknesses were irregularly dispersed. Thirteen studies reported significant relationships between stress and adverse outcomes. CONCLUSIONS: Study designs, including accuracy of outcome assessment and repeated sampling of outcomes (i.e. symptoms, clinical, and endoscopic), depended upon conditions like sample size, participants' compliance, and available resources. Meeting additional criteria of sound methodology, like taking into account covariates of the disease and its course, is strongly recommended to possibly improve study designs in future IBD research.