Validity and reliability of a health care service evaluation instrument for tuberculosis

OBJECTIVE To evaluate the validity and reliability of an instrument that evaluates the structure of primary health care units for the treatment of tuberculosis.METHODS This cross-sectional study used simple random sampling and evaluated 1,037 health care professionals from five Brazilian municipalities (Natal, state of Rio Grande do Norte; Cabedelo, state of Paraíba; Foz do Iguaçu, state of Parana; Sao José do Rio Preto, state of Sao Paulo, and Uberaba, state of Minas Gerais) in 2011. Structural indicators were identified and validated, considering different methods of organization of the health care system in the municipalities of different population sizes. Each structure represented the organization of health care services and contained the resources available for the execution of health care services: physical resources (equipment, consumables, and facilities); human resources (number and qualification); and resources for maintenance of the existing infrastructure and technology (deemed as the organization of health care services). The statistical analyses used in the validation process included reliability analysis, exploratory factor analysis, and confirmatory factor analysis.RESULTS The validation process indicated the retention of five factors, with 85.9% of the total variance explained, internal consistency between 0.6460 and 0.7802, and quality of fit of the confirmatory factor analysis of 0.995 using the goodness-of-fit index. The retained factors comprised five structural indicators: professionals involved in the care of tuberculosis patients, training, access to recording instruments, availability of supplies, and coordination of health care services with other levels of care. Availability of supplies had the best performance and the lowest coefficient of variation among the services evaluated. The indicators of assessment of human resources and coordination with other levels of care had satisfactory performance, but the latter showed the highest coefficient of variation. The performance of the indicators “training” and “access to recording instruments” was inferior to that of other indicators.CONCLUSIONS The instrument showed feasibility of application and potential to assess the structure of primary health care units for the treatment of tuberculosis.

Evaluation of efficacy, reliability, and tolerability of sibutramine in obese patients, with an echocardiographic study

This is a double-blind, placebo-controlled study of the efficacy, safety, and tolerability of sibutramine in the management of obese patients for a 6-month period. METHOD: Sixty-one obese patients (BMI >30, <40 kg/m2), aged 18-65 years were evaluated. In the first phase of the study (30 days), the patients were given a placebo. We monitored compliance with a low-calorie diet (1200 kcal/day) and to the placebo. In the next stage, the double-blind phase (6 months), we compared placebo and sibutramine (10 mg/day). The criteria for evaluating efficacy were weight loss, reduction in body mass index (BMI), and abdominal and hip circumferences. Tolerability was assessed based on reported side effects, variation in arterial blood pressure and heart rate, metabolic profile (fasting glucose, total cholesterol and its fractions, and triglycerides), laboratory tests (renal and hepatic functions), and flow Doppler echocardiogram. RESULTS: We observed a greater weight loss (7.3 kg, 8% vs 2.6 kg, 2.8%) and a reduction in body mass index (7.4% vs 2.1%) in the sibutramine group than in the placebo group. Classifying the patients into 4 subgroups according to weight loss (weight gain, loss <5%, loss of 5% to 9.9%, and loss >10%), we observed a weight loss of >5% in 40% of the patients on sibutramine compared with 12.9% in the placebo group. We also detected weight gain in 45.2% of the placebo group compared to 20% in the sibutramine group. The sibutramine group showed improvement in HDL- cholesterol values (increased by 17%) and triglyceride values (decreased by 12.8%). This group also showed an increase in systolic blood pressure (6.7%, 5 mmHg). There were no changes in echocardiograms comparing the beginning and end of follow-up, and side effects did not lead to discontinuation of treatment. DISCUSSION: Sibutramine proved to be effective for weight loss providing an 8% loss of the initial weight. Compliance to prolonged treatment was good, and side effects did not result in discontinuation of treatment. These data confirmed the good efficacy, tolerability, and safety profiles of sibutramine for treatment of obesity.

Translation into Brazilian Portuguese, cultural adaptation and evaluation of the reliability of the Disabilities of the Arm, Shoulder and Hand Questionnaire

The objective of the present study was to translate, adapt and validate a Brazilian Portuguese version of the Disabilities of the Arm, Shoulder and Hand (DASH) Questionnaire. The study was carried out in two steps. The first was to translate the DASH into Portuguese and to perform cultural adaptation and the second involved the determination of the reliability and validity of the DASH for the Brazilian population. For this purpose, 65 rheumatoid arthritis patients of either sex (according to the classification criteria of the American College of Rheumatology), ranging in age from 18 to 60 years and presenting no other diseases involving the upper limbs, were interviewed. The patients were selected consecutively at the rheumatology outpatient clinic of UNIFESP. The following results were obtained: in the first step (translation and cultural adaptation), all patients answered the questions. In the second step, Spearman's correlation coefficients for interobserver evaluation ranged from 0.762 to 0.995, values considered to be highly reliable. In addition, intraclass correlation coefficients ranged from 0.97 to 0.99, also highly reliable values. Spearman's correlation coefficients and the intraclass correlation coefficients obtained during intra-observer evaluation ranged from 0.731 to 0.937 and from 0.90 to 0.96, respectively, being highly reliable values. The Ritchie Index showed a weak correlation with Brazilian DASH scores, while the visual analog scale of pain showed a good correlation with DASH score. We conclude that the Portuguese version of the DASH is a reliable instrument.

Evaluation of maternal and neonatal hospital care: quality index of completeness

OBJECTIVE Develop an index to evaluate the maternal and neonatal hospital care of the Brazilian Unified Health System.METHODS This descriptive cross-sectional study of national scope was based on the structure-process-outcome framework proposed by Donabedian and on comprehensive health care. Data from the Hospital Information System and the National Registry of Health Establishments were used. The maternal and neonatal network of Brazilian Unified Health System consisted of 3,400 hospitals that performed at least 12 deliveries in 2009 or whose number of deliveries represented 10.0% or more of the total admissions in 2009. Relevance and reliability were defined as criteria for the selection of variables. Simple and composite indicators and the index of completeness were constructed and evaluated, and the distribution of maternal and neonatal hospital care was assessed in different regions of the country.RESULTS A total of 40 variables were selected, from which 27 single indicators, five composite indicators, and the index of completeness of care were built. Composite indicators were constructed by grouping simple indicators and included the following variables: hospital size, level of complexity, delivery care practice, recommended hospital practice, and epidemiological practice. The index of completeness of care grouped the five variables and classified them in ascending order, thereby yielding five levels of completeness of maternal and neonatal hospital care: very low, low, intermediate, high, and very high. The hospital network was predominantly of small size and low complexity, with inadequate child delivery care and poor development of recommended and epidemiological practices. The index showed that more than 80.0% hospitals had a low index of completeness of care and that most qualified heath care services were concentrated in the more developed regions of the country.CONCLUSIONS The index of completeness proved to be of great value for monitoring the maternal and neonatal hospital care of Brazilian Unified Health System and indicated that the quality of health care was unsatisfactory. However, its application does not replace specific evaluations.

A questionnaire to evaluate the impact of chronic diseases: validated translation and Illness Effects Questionnaire (IEQ) reliability study

INTRODUCTION: Patients' perception about their health condition, mainly involving chronic diseases, has been investigated in many studies and it has been associated to depression, compliance with the treatment, quality of life and prognosis. The Illness Effects Questionnaire (IEQ) is a tool which makes the standardized evaluation of patients' perception about their illness possible, so that it is brief and accessible to the different clinical settings. This work aims to begin the transcultural adaptation of the IEQ to Brazil through the validated translation and the reliability study. METHODS: The back-translation method and the test-retest reliability study were used in a sample of 30 adult patients under chronic hemodialysis. The reliability indexes were estimated using the Pearson, Spearman, Weighted Kappa and Cronbach's alpha coefficients. RESULTS: The semantic equivalence was reached through the validated translation. In this study, the reliability indexes obtained were respectively: 0.85 and 0.75 (p < 0.001); 0.68 and 0.92 (p < 0.0001). DISCUSSION: The reliability indexes obtained attest to the stability of responses in both evaluations. Additional procedures are necessary for the transcultural adaptation of the IEQ to be complete. CONCLUSION: The results indicate the translation validity and the reliability of the Brazilian version of the IEQ for the sample studied.

Reliability of the interpretation of coronary angiography by the simple visual method

OBJECTIVE: Evaluation of inter and intraobserver reproducibility of by the visual method interpretation of cineangiogram in a clinically based context. METHODS: Five interventional cardiologists analyzed 11 segments of 8 coronary cineangiograms at a two month apart sessions. The percent luminal reduction by the lesions were analyzed by two different classifications: in one (A) the lesions were graded in 0% = absent, 1-50% = mild, 51 - 69 = moderate, and > or = 70% = severe; the other classification (B) was a dichotomic one : <70% = nonsignificant and > or = 70%=significant lesions. The agreement were measured by the kappa (k) index. RESULTS: Interobserver agreement was moderate for classification A (1st measurement, k = 0.36 -- 0.63, k m = 0.49; 2nd measurement, k = 0.39-0.68, k m = 0.52) and good for classification B (1st measurement, k = 0.55-0.73, k m = 0.63; 2nd measurement, k = 0.37-0.82, k m = 0.61). Intraobserver levels of agreement were k = 0.57-0.95 for classification A and 0.62-1.0 for classification B. CONCLUSION: The higher level of reproducibility obtained by adopting the dichotomous criteria usually considered for ischemic limits demonstrates that in the present clinical context, the reliability of the simple visual method is adequate for the identification of patients with clinically significant lesions and candidates for myocardial revascularization procedures.

Reproducibility and Reliability of the Quality of Life Questionnaire in Patients With Atrial Fibrillation

Abstract Background: Studies have shown the impact of atrial fibrillation (AF) on the patients' quality of life. Specific questionnaires enable the evaluation of relevant events. We previously developed a questionnaire to assess the quality of life of patients with AF (AFQLQ version 1), which was reviewed in this study, and new domains were added. Objective: To demonstrate the reproducibility of the AFQLQ version 2 (AFQLQ v.2), which included the domains of fatigue, illness perception and well-being. Methods: We applied 160 questionnaires (AFQLQ v.2 and SF-36) to 40 patients, at baseline and 15 days after, to measure inter- and intraobserver reproducibility. The analysis of quality of life stability was determined by test-retest, applying the Bartko intraclass correlation coefficient (ICC). Internal consistency was assessed by Cronbach's alpha test. Results: The total score of the test-retest (n = 40) had an ICC of 0.98 in the AFQLQ v.2, and of 0.94 in the SF36. In assessing the intra- and interobserver reproducibility of the AFQLQ v.2, the ICC reliability was 0.98 and 0.97, respectively. The internal consistency had a Cronbach's alpha coefficient of 0.82, compatible with good agreement of the AFQLQ v.2. Conclusion: The AFQLQ v.2 performed better than its previous version. Similarly, the domains added contributed to make it more comprehensive and robust to assess the quality of life of patients with AF.

Evaluation of the histopathological classifications of American cutaneous and mucocutaneous leishmaniasis

In order to evaluate the reliability of histopathological classifications of cutaneous and mucocutaneous leishmaniasis the authors compared the histopathological patterns of two biopsies taken simultaneously from the same patient, and classified the material according to Ridley et al. (1980), to Magalhães et al. (1986a), and to a more simplified classification with only three patterns. District histopathological aspects, were observed in different lesions or even in the same lesion. The authors concluded that histopathological patterns do not represent a stage of tegumentary leishmaniasis, thus they can not be correlated with prognosis and therapeutical response as suggested in the literature.

Evaluation of the Primary Care in leprosy control: proposal of an instrument for users

Objective Developing an instrument to evaluate the performance of primary health care in the leprosy control actions, from the perspective of users and do the face and content validation. Method This is a methodological study carried out in four stages: development of instrument, face and content validation, pre-test, and analysis of test-retest reliability. Results The initial instrument submitted to the judgment of 15 experts was composed of 157 items. The face and content validation and pre-test of instrument were essential for the exclusion of items and adjustment of instrument to evaluate the object under study. In the analysis of test-retest reliability, the instrument proved to be reliable. Conclusion The instrument is considered adequate, but further studies are needed to test the psychometric properties.


Scientific Literature on the Reliability and Validity of the Manchester Triage System (MTS) Protocol: A Integrative Literature Review

OBJECTIVE To analyze the scientific production about the validity and reliability of the Manchester Triage System (MTS) protocol. METHOD A descriptive study of an integrative literature review. Articles on the validity and reliability of the MTS developed with children and adults published between 1999 and 2013 were included. RESULTS 14 articles were selected from a total of 8438, nine of validity and five of reliability. The reliability of the MTS ranged from moderate to almost perfect, with higher intra-evaluation. Regarding validity, the results seem to point to equivalent and satisfactory sensibility and specificity levels of the MTS. The instrument proved to be a good predictor of the need for hospitalization and of hospital mortality. CONCLUSION The reliability and validity of the MTS obtained in the studies is varied. It is recommended that new studies indicate necessary modifications to the MTS so that it is more safely used by nurses.

Anatomical and functional characteristics of the pelvic floor in nulliparous women submitted to three-dimensional endovaginal ultrasonography: case control study and evaluation of interobserver agreement

PURPOSE: To determine anatomical and functional pelvic floor measurements performed with three-dimensional (3-D) endovaginal ultrasonography in asymptomatic nulliparous women without dysfunctions detected in previous dynamic 3-D anorectal ultrasonography (echo defecography) and to demonstrate the interobserver reliability of these measurements. METHODS: Asymptomatic nulliparous volunteers were submitted to echo defecography to identify dynamic dysfunctions, including anatomical (rectocele, intussusceptions, entero/sigmoidocele and perineal descent) and functional changes (non-relaxation or paradoxical contraction of the puborectalis muscle) in the posterior compartment and assessed with regard to the biometric index of levator hiatus, pubovisceral muscle thickness, urethral length, anorectal angle, anorectal junction position and bladder neck position with the 3-D endovaginal ultrasonography. All measurements were compared at rest and during the Valsalva maneuver, and perineal and bladder neck descent was determined. The level of interobserver agreement was evaluated for all measurements. RESULTS: A total of 34 volunteers were assessed by echo defecography and by 3-D endovaginal ultrasonography. Out of these, 20 subjects met the inclusion criteria. The 14 excluded subjects were found to have posterior dynamic dysfunctions. During the Valsalva maneuver, the hiatal area was significantly larger, the urethra was significantly shorter and the anorectal angle was greater. Measurements at rest and during the Valsalva maneuver differed significantly with regard to anorectal junction and bladder neck position. The mean values for normal perineal descent and bladder neck descent were 0.6 cm and 0.5 cm above the symphysis pubis, respectively. The intraclass correlation coefficient ranged from 0.62-0.93. CONCLUSIONS: Functional biometric indexes, normal perineal descent and bladder neck descent values were determined for young asymptomatic nulliparous women with the 3-D endovaginal ultrasonography. The method was found to be reliable to measure pelvic floor structures at rest and during Valsalva, and might therefore be suitable for identifying dysfunctions in symptomatic patients.

A Portuguese version of the Christo Inventory for Substance-Misuse Services: a simple outcome evaluation tool

The Christo Inventory for Substance-Misuse Services (CISS) is a single page outcome evaluation tool completed by drug alcohol service workers either on the basis of direct client interviews or of personal experience of their client supplemented by existing assessment notes. It was developed to assist substance misuse services to empirically demonstrate the effectiveness of their treatments to their respective funding bodies. Its 0 to 20 unidimensional scale consists of 10 items reflecting clients' problems with social functioning, general health, sexual/injecting risk behavior, psychological functioning, occupation, criminal involvement, drug/alcohol use, ongoing support, compliance, and working relationships. Good reliability and validity has already been demonstrated for the CISS [Christo et al., Drug and Alcohol Dependence 2000; 59: 189-197] but the original was written in English and a Portuguese version is presented here. The present review explores its applicability to a Brazilian setting, summarizes its characteristics and uses, and describes the process of translation to Portuguese. A pilot study conducted in a substance misuse service for adolescents indicated it is likely to be suitable for use among a Brazilian population. The simplicity, flexibility and brevity of the CISS make it a useful tool allowing comparison of clients within and between many different service settings.

Reliability and clinical utility of a Portuguese version of the Abnormal Involuntary Movements Scale (AIMS) for tardive dyskinesia in Brazilian patients

The objective of the present study was to evaluate the reliability and clinical utility of a Portuguese version of the Abnormal Involuntary Movements Scale (AIMS). Videotaped interviews with 16 psychiatric inpatients treated with antipsychotic drugs for at least 5 years were evaluated. Reliability was assessed by the intraclass correlation coefficient (ICC) between three raters, two with and one without clinical training in psychopathology. Clinical utility was assessed by the difference between the scores of patients with (N = 11) and without (N = 5) tardive dyskinesia (TD). Patients with TD exhibited a higher severity of global evaluation by the AIMS (sum of scores: 4.2 ± 0.9 vs 0.4 ± 0.2; score on item 8: 2.3 ± 0.3 vs 0.4 ± 0.2, TD vs controls). The ICC for the global evaluation was fair between the two skilled raters (0.58-0.62) and poor between these raters and the rater without clinical experience (0.05-0.29). Thus, we concluded that the Portuguese version of the AIMS shows an acceptable inter-rater reliability, but only between clinically skilled raters, and that it is clinically useful.