Evaluation of outcome in child and adolescent mental health services: children with persistent conduct problems

This article focuses on the evaluation of outcome in child and adolescent mental health services. We examined the outcomes of 46 children with persistent conduct problems by gathering at baseline and six months information from multiple informants on multiple domains including the functioning of the child, risk factors, and parents’ and children’s perceptions of the treatment process. A statistically significant reduction in oppositional/conduct symptoms was reported six months after the initial clinical contact. However, the majority of the group still scored within the clinical range. The various outcome measures are correlated to only a mild to moderate degree. Teachers did not notice the same degree of change at school, despite the changes noticed by parents. Symptom improvement and satisfaction with a service are two separate issues. Parents’ satisfaction was related to their perception of the therapist and the therapy offered. Their satisfaction was high if they perceived that the therapist was able to communicate well, show care and concern, and if the therapy was perceived as organized. Much can be learned from a comprehensive outcome measurement system within a mental health service.

Robustness in health research : do differences in health measures, techniques, and time frame matter?

Survey-based health research is in a boom phase following an increased amount of health spending in OECD countries and the interest in ageing. A general characteristic of survey-based health research is its diversity. Different studies are based on different health questions in different datasets; they use different statistical techniques; they differ in whether they approach health from an ordinal or cardinal perspective; and they differ in whether they measure short-term or long-term effects. The question in this paper is simple: do these differences matter for the findings? We investigate the effects of life-style choices (drinking, smoking, exercise) and income on six measures of health in the US Health and Retirement Study (HRS) between 1992 and 2002: (1) self-assessed general health status, (2) problems with undertaking daily tasks and chores, (3) mental health indicators, (4) BMI, (5) the presence of serious long-term health conditions, and (6) mortality. We compare ordinal models with cardinal models; we compare models with fixed effects to models without fixed-effects; and we compare short-term effects to long-term effects. We find considerable variation in the impact of different determinants on our chosen health outcome measures; we find that it matters whether ordinality or cardinality is assumed; we find substantial differences between estimates that account for fixed effects versus those that do not; and we find that short-run and long-run effects differ greatly. All this implies that health is an even more complicated notion than hitherto thought, defying generalizations from one measure to the others or one methodology to another.

The validation of an Australian sign language instrument for outcome measurement of adults in mental health settings

Objective: There are currently no adult mental health outcome measures that have been translated into Australian sign language (Auslan). Without a valid and reliable Auslan outcome measure, empirical research into the efficacy of mental health interventions for sign language users is unattainable. To address this research problem the Outcome Rating Scale (ORS), a measure of general functioning, was translated into Auslan and recorded on to digital video disk for use in clinical settings. The purpose of the present study was therefore to examine the reliability, validity and acceptability of an Auslan version of the ORS (ORS-Auslan).
Method: The ORS-Auslan was administered to 44 deaf people who use Auslan as their first language and who identify as members of a deaf community (termed ‘Deaf’ people) on their first presentation to a mental health or counselling facility and to 55 Deaf people in the general community. The community sample also completed an Auslan version of the Depression Anxiety Stress Scale-21 (DASS-21).
Results: t-Tests indicated significant differences between the mean scores for the clinical and community sample. Internal consistency was acceptable given the low number of items in the ORS-Auslan. Construct validity was established by significant correlations between total scores on the DASS-21-Auslan and ORS-Auslan. Acceptability of ORS-Auslan was evident in the completion rate of 93% compared with 63% for DASS-21-Auslan.
Conclusions: This is the only Auslan outcome measure available that can be used across a wide variety of mental health and clinical settings. The ORS-Auslan provides mental health clinicians with a reliable and valid, brief measure of general functioning that can significantly distinguish between clinical and non-clinical presentations for members of the Deaf community.

Attitudes of mental health staff to routine outcome measurement

Background: Routine outcome measurement is mandated in public mental health services in Australia, but uptake and compliance are variable. This may be because of uncertainties and resistances among clinicians.
Aims: To survey attitudes and practices to routine outcome measurement among staff in adult area mental health services and to elucidate their correlates.
Method: As part of a larger study, a specifically designed questionnaire was distributed to all staff.
Results: A high return rate was achieved. A wide range of opinion was found. Staff who had attended training reported the measures as easier to use than those who had not. Staff who had recently seen feedback rated outcome measures as more valuable but less easy to use than those who had not seen feedback. Compared to other disciplines, medical staff and psychologists tended to rate outcome measures as less useful.
Conclusions: The results have implications for the implementation and sustainability of routine outcome measurement. They highlight the need for staff to receive targeted training and usable reports, and to have access to resources to extract meaning and value from outcome measures.

The validity of the CGI severity and improvement scales as measures of clinical effectiveness suitable for routine clinical use

Objective The Clinical Global Impression Scale (CGI) is established as a core metric in psychiatric research. This study aims to test the validity of CGI as a clinical outcome measure suitable for routine use in a private inpatient setting.

Methods The CGI was added to a standard battery of routine outcome measures in a private psychiatric hospital. Data were collected on consecutive admissions over a period of 24 months, which included clinical diagnosis, demographics, service utilization and four routine measures (CGI, HoNOS, MHQ-14 and DASS-21) at both admission and discharge. Descriptive and comparative data analyses were performed.

Results Of 786 admissions in total, there were 624 and 614 CGI-S ratings completed at the point of admission and discharge, respectively, and 610 completed CGI-I ratings. The admission and discharge CGI-S scores were correlated (r = 0.40), and the indirect improvement measures obtained from their differences were highly correlated with the direct CGI-I scores (r = 0.71). The CGI results reflected similar trends seen in the other three outcome measures.

Conclusions The CGI is a valid clinical outcome measure suitable for routine use in an inpatient setting. It offers a number of advantages, including its established utility in psychiatric research, sensitivity to change, quick and simple administration, utility across diagnostic groupings, and reliability in the hands of skilled clinicians.

Consistency of the Health of the Nation Outcome Scales (HoNOS) at inpatient-to-community transition

OBJECTIVES: The Health of the Nation Outcome Scales (HoNOS) are mandated outcome-measures in many mental-health jurisdictions. When HoNOS are used in different care settings, it is important to assess if setting specific bias exists. This article examines the consistency of HoNOS in a sample of psychiatric patients transitioned from acute inpatient care and community centres.

SETTING: A regional mental health service with both acute and community facilities.

PARTICIPANTS: 111 psychiatric patients were transferred from inpatient care to community care from 2012 to 2014. Their HoNOS scores were extracted from a clinical database; Each inpatient-discharge assessment was followed by a community-intake assessment, with the median period between assessments being 4 days (range 0-14). Assessor experience and professional background were recorded.

PRIMARY AND SECONDARY OUTCOME MEASURES: The difference of HoNOS at inpatient-discharge and community-intake were assessed with Pearson correlation, Cohen's κ and effect size.

RESULTS: Inpatient-discharge HoNOS was on average lower than community-intake HoNOS. The average HoNOS was 8.05 at discharge (median 7, range 1-22), and 12.16 at intake (median 12, range 1-25), an average increase of 4.11 (SD 6.97). Pearson correlation between two total scores was 0.073 (95% CI -0.095 to 0.238) and Cohen's κ was 0.02 (95% CI -0.02 to 0.06). Differences did not appear to depend on assessor experience or professional background.

CONCLUSIONS: Systematic change in the HoNOS occurs at inpatient-to-community transition. Some caution should be exercised in making direct comparisons between inpatient HoNOS and community HoNOS scores.

The "Orthoplastic" combined surgical approach to open tibia fractures: a multicentric prospective outcome study

Introduction: Open fractures of the leg represent a severe trauma. The combined approach, shared between plastic and orthopaedic surgeons, is considered to be important, although this multidisciplinary treatment is not routinely performed. Aim of this study was to verify whether the orthoplastic treatment is of any advantage over the traditional simply orthopedic treatment, through a multicentric inclusion of these unfrequent injuries into a prospective study. Material and methods: The following trauma centres were involved: Rizzoli Orthopaedic Institute/University of Bologna (leading centre) and Maggiore Hospital (Bologna, Italy), Frenchay Hospital (Bristol, United Kingdom), Jinnah Hospital (Lahore, Pakistan). All patients consecutively hospitalized in the mentioned centres between January 2012 and December 2013 due to tibial open fractures were included in the study and prospectively followed up to December 2014. Demographics and other clinical features were recorded, including the type of treatment (orthopaedic or orthoplastic). The considered outcome measures included duration of hospitalization, time for bone union and soft tissue closure, Enneking score at 3, 6 and 12 months, the incidence of osteomyelitis and other complications. Results: A total of 164 patients were included in the study. Out of them 68% were treated with an orthoplastic approach, whereas 32% received a purely orthopedic treatment. All considered outcome measures showed to be improved by the orthoplastic approach, compared to the orthopaedic one: time for soft tissue closure (2 versus 25 weeks), duration of hospital stay (22 versus 55 days), time for bone union (6 versus 8.5 months) , number of additional operations (0.6 versus 1.2) and functional recovery of the limb at 12 months (27 versus 19, Enneking’s score). All results were statistically significant. Conclusion: The combined orthoplastic approach to the treatment of open tibia fractures, in particular for high grade injuries (Gustilo 3B), is proven to improve the outcome of these severe injuries.

Complete continence after botulinum neurotoxin type A injections for refractory idiopathic detrusor overactivity incontinence: patient-reported outcome at 4 weeks

Objective improvement following intradetrusor injections of botulinum neurotoxin type A (BoNTA) is well documented. Although patient-related outcome measures are highly recommended for monitoring overactive bladder symptoms, no study before has dealt with the question of patient-reported complete continence after BoNTA treatment using validated questionnaires.

Comparison of two scoring systems for evaluation of treatment outcome in patients with complete bilateral cleft lip and palate

Objective : To compare two scoring systems: the Huddart/Bodenham system (HB system) and the Bauru-BCLP yardstick (BCLP yardstick), which classify treatment outcome in terms of dental arch relationships in patients with complete bilateral cleft lip and palate (CBCLP). The predictive value of these scoring systems for treatment outcome was also evaluated. Design : Retrospective longitudinal study. Patients : Dental arch relationships of 43 CBCLP patients were evaluated at 6, 9, and 12 years. Setting : Treatment outcome in BCLP patients using two scoring systems. Main Outcome Measures : For each age group, the HB scores were correlated with the BCLP yardstick scores using Spearman's correlation coefficient. The predictive value of the two scoring systems was evaluated by backward regression analysis. Results : Intraobserver Kappa values for the BCLP yardstick scoring for the two observers were .506 and .627, respectively, and the interobserver reliability ranged from .427 and .581. The intraobserver reliability for the HB system ranged from .92 to .97 and the interobserver reliability from .88 to .96. The BCLP yardstick scores of 6 and 9 years together were predictors for the outcome at 12 years (explained variance 41.3%). Adding the incisor and lateral HB scores in the regression model increased the explained variance to 67%. Conclusions : The BCLP yardstick and the HB system are reliable scoring systems for evaluation of dental arch relationships of CBCLP patients. The HB system categorizes treatment outcome into similar categories as the BCLP yardstick. In case a more sensitive measure of treatment outcome is needed, selectively both scoring systems should be used.

Bone-anchored Hearing Aids: correlation between pure-tone thresholds and outcome in three user groups

OBJECTIVE: To investigate correlations between preoperative hearing thresholds and postoperative aided thresholds and speech understanding of users of Bone-anchored Hearing Aids (BAHA). Such correlations may be useful to estimate the postoperative outcome with BAHA from preoperative data. STUDY DESIGN: Retrospective case review. SETTING: Tertiary referral center. PATIENTS:: Ninety-two adult unilaterally implanted BAHA users in 3 groups: (A) 24 subjects with a unilateral conductive hearing loss, (B) 38 subjects with a bilateral conductive hearing loss, and (C) 30 subjects with single-sided deafness. INTERVENTIONS: Preoperative air-conduction and bone-conduction thresholds and 3-month postoperative aided and unaided sound-field thresholds as well as speech understanding using German 2-digit numbers and monosyllabic words were measured and analyzed. MAIN OUTCOME MEASURES: Correlation between preoperative air-conduction and bone-conduction thresholds of the better and of the poorer ear and postoperative aided thresholds as well as correlations between gain in sound-field threshold and gain in speech understanding. RESULTS: Aided postoperative sound-field thresholds correlate best with BC threshold of the better ear (correlation coefficients, r2 = 0.237 to 0.419, p = 0.0006 to 0.0064, depending on the group of subjects). Improvements in sound-field threshold correspond to improvements in speech understanding. CONCLUSION: When estimating expected postoperative aided sound-field thresholds of BAHA users from preoperative hearing thresholds, the BC threshold of the better ear should be used. For the patient groups considered, speech understanding in quiet can be estimated from the improvement in sound-field thresholds.

Microvascular decompression for trigeminal neuralgia in the elderly: long-term treatment outcome and comparison with younger patients

OBJECTIVE: Multiple studies have proved that microvascular decompression (MVD) is the treatment of choice in cases of medically refractory trigeminal neuralgia (TN). In the elderly, however, the surgical risks related to MVD are assumed to be unacceptably high and various alternative therapies have been proposed. We evaluated the outcomes of MVD in patients aged older than 65 years of age and compared them with the outcomes in a matched group of younger patients. The focus was on procedure-related morbidity rate and long-term outcome. METHODS: This was a retrospective study of 112 patients with TN operated on consecutively over 22 years. The main outcome measures were immediate and long-term postoperative pain relief and neurological status, especially function of trigeminal, facial, and cochlear nerves, as well as surgical complications. A questionnaire was used to assess long-term outcome: pain relief, duration of a pain-free period, need for pain medications, time to recurrence, pain severity, and need for additional treatment. RESULTS: The mean age was 70.35 years. The second and third branches of the trigeminal nerve were most frequently affected (37.3%). The mean follow-up period was 90 months (range, 48-295 months). Seventy-five percent of the patients were completely pain free, 11% were never pain free, and 14% experienced recurrences. No statistically significant differences existed in the outcome between the younger and older patient groups. Postoperative morbidity included trigeminal hypesthesia in 6.25%, hypacusis in 5.4%, and complete hearing loss, vertigo, and partial facial nerve palsy in 0.89% each. Cerebrospinal fluid leak and meningitis occurred in 1 patient each. There were no mortalities in both groups. CONCLUSION: MVD for TN is a safe procedure even in the elderly. The risk of serious morbidity or mortality is similar to that in younger patients. Furthermore, no significant differences in short- and long-term outcome were found. Thus, MVD is the treatment of choice in patients with medically refractory TN, unless their general condition prohibits it.

Current practice in neuropsychological outcome reporting after aneurysmal subarachnoid haemorrhage

BACKGROUND Neuropsychological deficits (NPD) are common in patients with aneurysmal subarachnoid haemorrhage (aSAH). NPD are one of the major limiting factors for patients with an otherwise acceptable prognosis for sustained quality of life. There are only a few studies reporting outcome after aSAH, which used a standardized neuropsychological test battery as a primary or secondary outcome measure. Aim of this study was to determine the current practice of reporting NPD following aSAH in clinical studies. METHODS A MEDLINE analysis was performed using the search term "subarachnoid haemorrhage outcome". The latest 1,000 articles were screened. We recorded study design, number of patients, and the presence of neuropsychological outcome report. Additionally, the time of testing after aSAH, the neuropsychological tests administered, as well as the percentage of patients with NPD were analyzed. RESULTS A total of 324 publications between 2009 and 2012 were selected for further review. Of those, 21 studies (6.5%) reported neuropsychological outcome, in 2,001 of 346,666 patients (0.6%). The assessment of NPD differed broadly using both subjective and objective cognitive evaluation, and a large variety of tests were used. CONCLUSION Neuropsychological outcome is underreported, and there is great variety in assessment in currently published clinical articles on aSAH. Prospective randomized trials treating aSAH may benefit from implementing more comprehensive and standardized neuropsychological outcome measures. This approach might identify otherwise unnoticed treatment effects in future interventional studies of aSAH patients.

Does a Pre-Treatment Diagnostic Interview Affect the Outcome of Internet-Based Self-Help for Social Anxiety Disorder? A Randomized Controlled Trial

BACKGROUND: Numerous studies suggest that Internet-based self-help treatments are effective in treating anxiety disorders. Trials evaluating such interventions differ in their screening procedures and in the amount of clinician contact in the diagnostic assessment phase. The present study evaluates the impact of a pre-treatment diagnostic interview on the outcome of an Internet-based treatment for Social Anxiety Disorder (SAD). METHOD: One hundred and nine participants seeking treatment for SAD were randomized to either an interview-group (IG, N = 53) or to a non-interview group (NIG, N = 56). All participants took part in the same 10-week cognitive-behavioural unguided self-help programme. Before receiving access to the programme, participants of the IG underwent a structured diagnostic interview. Participants of the NIG started directly with the programme. RESULTS: Participants in both groups showed significant and substantial improvement on social anxiety measures from pre- to post-assessment (d IG = 1.30-1.63; d NIG = 1.00-1.28) and from pre- to 4-month follow-up assessment (d IG = 1.38-1.87; d NIG = 1.10-1.21). Significant between-groups effects in favour of the IG were found on secondary outcome measures of depression and general distress (d = 0.18-0.42). CONCLUSIONS: These findings suggest that Internet-based self-help is effective in treating SAD, whether or not a diagnostic interview is involved. However, the pre-treatment interview seems to facilitate change on secondary outcomes such as depression and general distress.

Idiographic Self-Monitoring Instruments to Empower Client Participation and Evaluate Outcome in Intensive Family Preservation Services

Intensive Family Preservation Services seek to reflect the values of focusing on client strengths and viewing clients as colleagues. To promote those values, Intensive Family Preservation Programs should include a systematic form of client self monitoring in their packages of outcome measures. This paper presents a model of idiographic self-monitoring used in time series, single system research design developed for Family Partners, a family preservation program of the School for Contemporary Education in Annandale, Virginia. The evaluation model provides a means of empowering client families to utilize their strengths and promote their status as colleague in determining their own goals, participating in the change process, and measuring their own progress.

Structural equation modeling of the medical outcome study: Short Form 36 (SF-36)

The factorial validity of the SF-36 was evaluated using confirmatory factor analysis (CFA) methods, structural equation modeling (SEM), and multigroup structural equation modeling (MSEM). First, the measurement and structural model of the hypothesized SF-36 was explicated. Second, the model was tested for the validity of a second-order factorial structure, upon evidence of model misfit, determined the best-fitting model, and tested the validity of the best-fitting model on a second random sample from the same population. Third, the best-fitting model was tested for invariance of the factorial structure across race, age, and educational subgroups using MSEM.^ The findings support the second-order factorial structure of the SF-36 as proposed by Ware and Sherbourne (1992). However, the results suggest that: (a) Mental Health and Physical Health covary; (b) general mental health cross-loads onto Physical Health; (c) general health perception loads onto Mental Health instead of Physical Health; (d) many of the error terms are correlated; and (e) the physical function scale is not reliable across these two samples. This hierarchical factor pattern was replicated across both samples of health care workers, suggesting that the post hoc model fitting was not data specific. Subgroup analysis suggests that the physical function scale is not reliable across the "age" or "education" subgroups and that the general mental health scale path from Mental Health is not reliable across the "white/nonwhite" or "education" subgroups.^ The importance of this study is in the use of SEM and MSEM in evaluating sample data from the use of the SF-36. These methods are uniquely suited to the analysis of latent variable structures and are widely used in other fields. The use of latent variable models for self reported outcome measures has become widespread, and should now be applied to medical outcomes research. Invariance testing is superior to mean scores or summary scores when evaluating differences between groups. From a practical, as well as, psychometric perspective, it seems imperative that construct validity research related to the SF-36 establish whether this same hierarchical structure and invariance holds for other populations.^ This project is presented as three articles to be submitted for publication. ^