Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

OBJECTIVE: To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. DESIGN: Observer agreement study. DATA SOURCES: A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. DATA EXTRACTION: Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. RESULTS: The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. CONCLUSIONS: Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.

International validation of a behavioral scale in Parkinson's disease without dementia.

The "Ardouin Scale of Behavior in Parkinson's Disease" is a new instrument specifically designed for assessing mood and behavior with a view to quantifying changes related to Parkinson's disease, to dopaminergic medication, and to non-motor fluctuations. This study was aimed at analyzing the psychometric attributes of this scale in patients with Parkinson's disease without dementia. In addition to this scale, the following measures were applied: the Unified Parkinson's Disease Rating Scale, the Montgomery and Asberg Depression Rating Scale, the Lille Apathy Rating Scale, the Bech and Rafaelsen Mania Scale, the Positive and Negative Syndrome Scale, the MacElroy Criteria, the Patrick Carnes criteria, the Hospital Anxiety and Depression Scale, and the Mini-International Neuropsychiatric Interview. Patients (n = 260) were recruited at 13 centers across four countries (France, Spain, United Kingdom, and United States). Cronbach's alpha coefficient for domains ranged from 0.69 to 0.78. Regarding test-retest reliability, the kappa coefficient for items was higher than 0.4. For inter-rater reliability, the kappa values were 0.29 to 0.81. Furthermore, most of the items from the Ardouin Scale of Behavior in Parkinson's Disease correlated with the corresponding items of the other scales, depressed mood with the Montgomery and Asberg Depression Rating Scale (ρ = 0.82); anxiety with the Hospital Anxiety and Depression Scale-anxiety (ρ = 0.56); apathy with the Lille Apathy Rating Scale (ρ = 0.60). The Ardouin Scale of Behavior in Parkinson's disease is an acceptable, reproducible, valid, and precise assessment for evaluating changes in behavior in patients with Parkinson's disease without dementia. © 2015 International Parkinson and Movement Disorder Society.

Utility of the Social Responsiveness Scale- Parent Report (SRS-Parent) as a Diagnostic Tool for Autism Identification

Thesis (Ph.D.)--University of Washington, 2016-06

Evaluations of treatment efficacy of depression from perspective of both patients' symptoms and general sense of mental health and wellbeing: a large scale, multi-centered, longitudinal study in China

Relying on the absence, presence of level of symptomatology may not provide an adequate indication of the effects of treatment for depression, nor sufficient information for the development of treatment plans that meet patients' needs. Using a prospective, multi-centered, and observational design, the present study surveyed a large sample of outpatients with depression in China (n=9855). The 17-item Hamilton Rating Scale for Depression (HRSD-17) and the Remission Evaluation and Mood Inventory Tool (REMIT) were administered at baseline, two weeks later and 4 weeks, to assess patients' self-reported symptoms and general sense of mental health and wellbeing. Of 9855 outpatients, 91.3% were diagnosed as experiencing moderate to severe depression. The patients reported significant improvement over time on both depressive symptoms and general sense after 4-week treatment. The effect sizes of change in general sense were lower than those in symptoms at both two week and four week follow-up. Treatment effects on both general sense and depressive symptomatology were associated with demographic and clinical factors. The findings indicate that a focus on both general sense of mental health and wellbeing in addition to depressive symptomatology will provide clinicians, researchers and patients themselves with a broader perspective of the status of patients.

The relationship between closeness to significant others and self-esteem

One hundred and forty-three children (61 boys and 82 girls) between 8 and 12 years of age participated in a study which focused on closeness to significant others and its relationship with self-esteem. A closeness rating scale was developed to determine quantitatively how close children feel to their mother, father, two closest peers, and current teacher. Marsh's (1990) eight general self-concept items from the Self-Description Questionnaire 1 (SDQI), together with eight items from Burnett's (1994) Self-scale, were administered to the children, who ranged in age from 8 to 12 years, to measure their self-esteem. Closeness to mother was found to correlate most significantly with a child's self-esteem, while closeness to teacher was related more strongly to self-esteem for girls than for boys.

Learning to drive with cognitive impairment : the experience of young ADHD diagnosed drivers and their parent supervisors

A subset of novice drivers exhibit executive function impairments which may adversely impact on the learn-to-drive period and subsequent driving experience, potentially explaining their overrepresentation in traffic offences and crashes. This paper presents the results of a qualitative analysis of a small series of in-depth semi-structured interviews undertaken individually with affected young drivers (n = 7) and each of their parent supervisors (n = 6). Young drivers were selected on the basis of their ADHD diagnosis, as a sample particularly affected by executive function impairments. Standardised rating scale measures confirmed the currency of the young drivers’ ADHD symptoms and executive function impairment. Results are discussed in terms of common experiences of the young affected drivers and those of their parents as supervising drivers of the learn-to-drive process and subsequent driving behaviour. Key themes included difficulties that were related to core executive function impairments symptomatic of ADHD. Themes also included common emotions that the young drivers associated with driving, with particular types of impact on their driving behaviour. Common strategies that were used by both the young driver and their parent during this learning process and their perceived effectiveness are also discussed. Those that were perceived to be most effective tended to focus on reducing the cognitive load for the young driver when introducing new information and skills.

The Kava Anxiety Depression Spectrum Study (KADSS): A randomized, placebo-controlled, cross-over trial using an aqueous extract of Piper methysticum.

Rationale: Piper methysticum (Kava) has been withdrawn in European, British, and Canadian markets due to concerns over hepatotoxic reactions. The WHO recently recommended research into “aqueous” extracts of Kava. Objective: The objective of this study was to conduct the first documented human clinical trial assessing the anxiolytic and antidepressant efficacy of an aqueous extract of Kava. Design and participants: The Kava Anxiety Depression Spectrum Study was a 3-week placebo-controlled, double-blind crossover trial that recruited 60 adult participants with 1 month or more of elevated generalized anxiety. Five Kava tablets per day were prescribed containing 250 mg of kavalactones/day. Results: The aqueous extract of Kava reduced participants' Hamilton Anxiety Scale score in the first controlled phase by −9.9 (CI = 7.1, 12.7) vs. −0.8 (CI = −2.7, 4.3) for placebo and in the second controlled phase by −10.3 (CI = 5.8, 14.7) vs. +3.3 (CI = −6.8, 0.2). The pooled effect of Kava vs. placebo across phases was highly significant (p < 0.0001), with a substantial effect size (d = 2.24, η² [sub]p[sub] = 0.428). Pooled analyses also revealed highly significant relative reductions in Beck Anxiety Inventory and Montgomery–Asberg Depression Rating Scale scores. The aqueous extract was found to be safe, with no serious adverse effects and no clinical hepatotoxicity. Conclusions: The aqueous Kava preparation produced significant anxiolytic and antidepressant activity and raised no safety concerns at the dose and duration studied. Kava appears equally effective in cases where anxiety is accompanied by depression. This should encourage further study and consideration of globally reintroducing aqueous rootstock extracts of Kava for the management of anxiety.

Validation of an Australian sign language instrument of outcome measurement for adults in mental health settings

Objective: There are currently no adult mental health outcome measures that have been translated into Australian sign language (Auslan). Without a valid and reliable Auslan outcome measure, empirical research into the efficacy of mental health interventions for sign language users is unattainable. To address this research problem the Outcome Rating Scale (ORS), a measure of general functioning, was translated into Auslan and recorded on to digital video disk for use in clinical settings. The purpose of the present study was therefore to examine the reliability, validity and acceptability of an Auslan version of the ORS (ORS-Auslan). Method: The ORS-Auslan was administered to 44 deaf people who use Auslan as their first language and who identify as members of a deaf community (termed ‘Deaf’ people) on their first presentation to a mental health or counselling facility and to 55 Deaf people in the general community. The community sample also completed an Auslan version of the Depression Anxiety Stress Scale-21 (DASS-21). Results: t-Tests indicated significant differences between the mean scores for the clinical and community sample. Internal consistency was acceptable given the low number of items in the ORS-Auslan. Construct validity was established by significant correlations between total scores on the DASS-21-Auslan and ORS-Auslan. Acceptability of ORS-Auslan was evident in the completion rate of 93% compared with 63% for DASS-21-Auslan. Conclusions: This is the only Auslan outcome measure available that can be used across a wide variety of mental health and clinical settings. The ORS-Auslan provides mental health clinicians with a reliable and valid, brief measure of general functioning that can significantly distinguish between clinical and non-clinical presentations for members of the Deaf community.

Predictors of future falls in Parkinson disease

Background: Falls are a major health and injury problem for people with Parkinson disease (PD). Despite the severe consequences of falls, a major unresolved issue is the identification of factors that predict the risk of falls in individual patients with PD. The primary aim of this study was to prospectively determine an optimal combination of functional and disease-specific tests to predict falls in individuals with PD. ----- ----- Methods: A total of 101 people with early-stage PD undertook a battery of neurologic and functional tests in their optimally medicated state. The tests included Tinetti, Berg, Timed Up and Go, Functional Reach, and the Physiological Profile Assessment of Falls Risk; the latter assessment includes physiologic tests of visual function, proprioception, strength, cutaneous sensitivity, reaction time, and postural sway. Falls were recorded prospectively over 6 months. ----- ----- Results: Forty-eight percent of participants reported a fall and 24% more than 1 fall. In the multivariate model, a combination of the Unified Parkinson's Disease Rating Scale (UPDRS) total score, total freezing of gait score, occurrence of symptomatic postural orthostasis, Tinetti total score, and extent of postural sway in the anterior-posterior direction produced the best sensitivity (78%) and specificity (84%) for predicting falls. From the UPDRS items, only the rapid alternating task category was an independent predictor of falls. Reduced peripheral sensation and knee extension strength in fallers contributed to increased postural instability. ----- ----- Conclusions: Falls are a significant problem in optimally medicated early-stage PD. A combination of both disease-specific and balance- and mobility-related measures can accurately predict falls in individuals with PD.

Reference bias : presentation of extreme health states prior to EQ-VAS improves health-related quality of life scores. A randomised crossover trial.

Background: Clinical practice and clinical research has made a concerted effort to move beyond the use of clinical indicators alone and embrace patient focused care through the use of patient reported outcomes such as healthrelated quality of life. However, unless patients give consistent consideration to the health states that give meaning to measurement scales used to evaluate these constructs, longitudinal comparison of these measures may be invalid. This study aimed to investigate whether patients give consideration to a standard health state rating scale (EQ-VAS) and whether consideration of good and poor health state descriptors immediately changes their selfreport. Methods: A randomised crossover trial was implemented amongst hospitalised older adults (n = 151). Patients were asked to consider descriptions of extremely good (Description-A) and poor (Description-B) health states. The EQ-VAS was administered as a self-report at baseline, after the first descriptors (A or B), then again after the remaining descriptors (B or A respectively). At baseline patients were also asked if they had considered either EQVAS anchors. Results: Overall 106/151 (70%) participants changed their self-evaluation by ≥5 points on the 100 point VAS, with a mean (SD) change of +4.5 (12) points (p < 0.001). A total of 74/151 (49%) participants did not consider the best health VAS anchor, of the 77 who did 59 (77%) thought the good health descriptors were more extreme (better) then they had previously considered. Similarly 85/151 (66%) participants did not consider the worst health anchor of the 66 who did 63 (95%) thought the poor health descriptors were more extreme (worse) then they had previously considered. Conclusions: Health state self-reports may not be well considered. An immediate significant shift in response can be elicited by exposure to a mere description of an extreme health state despite no actual change in underlying health state occurring. Caution should be exercised in research and clinical settings when interpreting subjective patient reported outcomes that are dependent on brief anchors for meaning. Trial Registration: Australian and New Zealand Clinical Trials Registry (#ACTRN12607000606482) http://www.anzctr. org.au

Probabilistic subgroup identification using Bayesian finite mixture modelling : a case study in Parkinson’s disease phenotype identification

This article explores the use of probabilistic classification, namely finite mixture modelling, for identification of complex disease phenotypes, given cross-sectional data. In particular, if focuses on posterior probabilities of subgroup membership, a standard output of finite mixture modelling, and how the quantification of uncertainty in these probabilities can lead to more detailed analyses. Using a Bayesian approach, we describe two practical uses of this uncertainty: (i) as a means of describing a person’s membership to a single or multiple latent subgroups and (ii) as a means of describing identified subgroups by patient-centred covariates not included in model estimation. These proposed uses are demonstrated on a case study in Parkinson’s disease (PD), where latent subgroups are identified using multiple symptoms from the Unified Parkinson’s Disease Rating Scale (UPDRS).

How design of the physical environment impacts early learning : educators and parents perspectives

This research explores the quality and importance of the physical environment of two early learning centres on the Sunshine Coast in Queensland, utilising qualitative interviews with parents (n=4) and educators (n=4) to understand how design might impact on children’s development and a quantitative rating (the Early Childhood Physical Environment Rating Scale; ECPERS) to assess the quality of the physical built environment and infrastructure. With an average ECPERS quality rating, thematic analysis of the interviews revealed that educators and parents viewed the physical environment as important to a child’s development, although the quality of staff was predominant. Early learning centres should be ‘homely’, inviting, bright and linked to the outdoors, with participants describing how space “welcomes the child, makes them feel safe and encourages learning”. Four key themes characterised views: Emotional Connection (quality of staff and physical environment), Experiencing Design (impact of design on child development), Hub for Community Integration (relationships and resources) and Future Vision (ideal physical environment, technology and ratings). With participants often struggling to clearly articulate their thoughts on design issues, a collaborative and jargon-free approach to designing space is required. These findings will help facilitate discussion about the role and design of the physical environment in early childhood centres, with the tangible examples of ‘ideal space’ enhancing communication between architects and educators about how best to design and reconfigure space to enhance learning outcomes.

The development and evaluation of a culturally affirmative counselling model for deaf clients in Australia

In Australia, there is only one, newly established, dedicated mental health service catering specifically for the signing *Deaf community. It is staffed by four part-time hearing professionals and based in Brisbane. There are currently no Deaf psychologists or psychiatrists and there is no valid or reliable empirical evidence on outcomes for Deaf people accessing specialised or mainstream mental health services. Further compounding these issues, is the fact that there are no sign language versions of the most common standardised mental health or psychological instruments available to clinicians in Australia. Contemporary counselling literature is acknowledging the role of the therapeutic alliance and the impact of 'common factors' on therapeutic outcomes. However, these issues are complicated by the relationship between the Deaf client and the hearing therapist being a cross-cultural exchange. The disability model of deafness is contentious and few professionals in Australia have the requisite knowledge and understanding of deafness from a cultural perspective to attend to the therapeutic relationship with this in mind. Consequently, Deaf people are severely disadvantaged by the current lack of services, resources and skilled professionals in the field of deafness and psychology in this country. The primary aim of the following program of research has been to propose a model for culturally affirmative service delivery and to provide clinicians with tools to evaluate the effect of their therapeutic work with Deaf people seeking mental health treatment. The research document is presented as a thesis by publication and comprises four specific objectives formulated in response to the lack of existing services and resources. The first objective was to explore the use of social constructionist counselling techniques and a reflecting team with Deaf clients, hearing therapists and an interpreter. Following the establishment of a pilot counselling clinic, indepth semi-structured interviews were conducted with two long-term clients following the one year pilot of this service. These interviews generated recommendations for the development of a new 'enriched' model of counselling to be implemented and evaluated in later stages of the research program. The second objective was to identify appropriate psychometric measures that could be translated into Australian Sign Language (Auslan) for research into efficacy, effectiveness and counselling outcomes. Two instruments were identified as potentially suitable; the Outcome Rating Scale (ORS), a measure of global functioning, and the Session Rating Scale (SRS), a measure of therapeutic alliance. A specialised team of bi-lingual and bi-cultural interpreters, native signers and the primary researcher for this thesis, produced the ORS-Auslan and the SRS-Auslan in DVD format, using the translation and back-translation process. The third objective was to establish the validity and reliability of these new Auslan measures based on normative data from the Deaf community. Data from the ORS-Auslan was collected from one clinical and one non-clinical sample of Deaf people. Statistical analyses revealed that the ORS-Auslan is reliable, valid and adequately distinguishes between clinical and non-clinical presentations. Furthermore, construct validity has been established using a yet to be validated sign language version of the Depression, Anxiety and Stress Scale-21 items (DASS-21), providing a platform for further research using the DASS-21 with Deaf people. The fourth objective was to evaluate counselling outcomes following the implementation of an enriched counselling service, based on the findings generated by the first objective, and using the newly translated Auslan measures. A second university counselling clinic was established and implemented over the course of one year. Practice-based evidence guided the research and the ORS-Auslan and the SRS-Auslan were administered at every session and provided outcome data on Deaf clients' global functioning. Data from six clients over the course of ten months indicated that this culturally affirmative model was an effective approach for these six clients. This is the first time that outcome data have been collected in Australia using valid and reliable Auslan measures to establish preliminary evidence for the effectiveness of any therapeutic intervention for clinical work with adult, signing Deaf clients. The research generated by this thesis contributes theoretical knowledge, professional development and practical resources that can be used by a variety of mental health clinicians in the context of mental health service delivery to Deaf clients in Australia.

Bayesian mixtures for modelling complex medical data : a case study in Parkinson’s disease

Mixture models are a flexible tool for unsupervised clustering that have found popularity in a vast array of research areas. In studies of medicine, the use of mixtures holds the potential to greatly enhance our understanding of patient responses through the identification of clinically meaningful clusters that, given the complexity of many data sources, may otherwise by intangible. Furthermore, when developed in the Bayesian framework, mixture models provide a natural means for capturing and propagating uncertainty in different aspects of a clustering solution, arguably resulting in richer analyses of the population under study. This thesis aims to investigate the use of Bayesian mixture models in analysing varied and detailed sources of patient information collected in the study of complex disease. The first aim of this thesis is to showcase the flexibility of mixture models in modelling markedly different types of data. In particular, we examine three common variants on the mixture model, namely, finite mixtures, Dirichlet Process mixtures and hidden Markov models. Beyond the development and application of these models to different sources of data, this thesis also focuses on modelling different aspects relating to uncertainty in clustering. Examples of clustering uncertainty considered are uncertainty in a patient’s true cluster membership and accounting for uncertainty in the true number of clusters present. Finally, this thesis aims to address and propose solutions to the task of comparing clustering solutions, whether this be comparing patients or observations assigned to different subgroups or comparing clustering solutions over multiple datasets. To address these aims, we consider a case study in Parkinson’s disease (PD), a complex and commonly diagnosed neurodegenerative disorder. In particular, two commonly collected sources of patient information are considered. The first source of data are on symptoms associated with PD, recorded using the Unified Parkinson’s Disease Rating Scale (UPDRS) and constitutes the first half of this thesis. The second half of this thesis is dedicated to the analysis of microelectrode recordings collected during Deep Brain Stimulation (DBS), a popular palliative treatment for advanced PD. Analysis of this second source of data centers on the problems of unsupervised detection and sorting of action potentials or "spikes" in recordings of multiple cell activity, providing valuable information on real time neural activity in the brain.

Dirichlet process mixture models for unsupervised clustering of symptoms in Parkinson's disease

In this paper, the goal of identifying disease subgroups based on differences in observed symptom profile is considered. Commonly referred to as phenotype identification, solutions to this task often involve the application of unsupervised clustering techniques. In this paper, we investigate the application of a Dirichlet Process mixture (DPM) model for this task. This model is defined by the placement of the Dirichlet Process (DP) on the unknown components of a mixture model, allowing for the expression of uncertainty about the partitioning of observed data into homogeneous subgroups. To exemplify this approach, an application to phenotype identification in Parkinson’s disease (PD) is considered, with symptom profiles collected using the Unified Parkinson’s Disease Rating Scale (UPDRS). Clustering, Dirichlet Process mixture, Parkinson’s disease, UPDRS.