The development and field test of a Mealtime Interaction Clinical Observation Tool: a pilot study and clinical research portfolio

Objective: The purpose of this study was to develop and test psychometric properties of a Mealtime Interaction Clinical Observation Tool (MICOT) that could be used to facilitate assessment and behavioural intervention in childhood feeding difficulties. Methods: Thematic analysis of four focus groups with feeding and behaviour experts identified the content and structure of the MICOT. Following refinement, inter-rater reliability was tested between three healthcare professionals. Results: Six themes were identified for the MICOT, which utilises a traffic-light system to identify areas of strength and areas for intervention. Despite poor inter-rater reliability, for which a number of reasons are postulated, some correlation between psychologists’ ratings was evident. Healthcare professionals liked the tool and reported that it could have good clinical utility. Conclusion: The study provides a promising first version of a clinical observation tool that facilitates assessment and behavioural intervention in childhood feeding difficulties.

Assessment of interstitial lung disease in Sjögren’s syndrome by lung ultrasound: a pilot study of correlation with high-resolution chest tomography

The background of this study is to assess the accuracy of lung ultrasound (LUS) to diagnose interstitial lung disease (ILD) in Sjögren’s syndrome (Sjs), in patients who have any alterations in pulmonary function tests (PFT) or respiratory symptoms. LUS was correlated with chest tomography (hrCT), considering it as the imaging gold standard technique to diagnose ILD. This is a pilot, multicenter, cross-sectional, and consecutive-case study. The inclusion criteria are ≥18 years old, Signs and symptoms: according to ACEG 2002 criteria, respiratory symptoms (dyspnea, cough), or any alterations in PFR. LUS was done following the International Consensus Conference on Lung Ultrasound protocol for interstitial syndrome (B pattern). Of the 50 patients in follow-up, 13 (26%) met the inclusion criteria. All were women with age 63.62 years (range 39–88). 78.6% of the cases had primary Sjs (SLE, RA, n = 2). The intra-rater reliability k is 1, according to Gwet’s Ac1 and GI index (probability to concordance—e(K)—, by Cohen, of 0.52). LUS has a sensitivity of 1 (95% CI 0.398–1.0), specificity of 0.89 (95% CI 0.518–0.997), and a positive probability reason of 9.00 (95% CI 7.1–11.3) to detect ILD. The correlation of Pearson is r = 0.84 (p < 0.001). To check the accuracy of LUS to diagnose ILD, a completely bilateral criterion of yes/no for interstitial pattern was chosen, AUC reaches significance, 0.94 (0.07) (95% CI 0.81–1.0, p = 0.014). LUS reaches an excellent correlation to hrCT in Sjs affected with ILD, and might be a useful technique in daily clinical practice for the assessment of pulmonary disease in the sicca syndrome. © 2016 SIMI

Evaluación de las propiedades clinimétricas de la escala POSAS y su aplicación en pacientes de la Fundación del Quemado en Bogotá

•Objetivos: Se tradujo, adaptó y evaluaron las propiedades clinimétricas de la escala POSAS en pacientes con cicatrices hipertróficas (CHT) y queloides (CQ) cómo secuelas de quemadura, que fueron manejados con Z plastias en la Fundación del Quemado en Bogotá (Colombia), entre Junio de 2015 a Abril de 2016. •Métodos: Estudio de evaluación de las propiedades clinimétricas de una escala. Se hizo una traducción y adaptación transcultural siguiendo el método de traducción-retrotraducción. Se aplicó el instrumento adaptado a cincuenta y dos pacientes (n=52) antes y después de la intervención quirúrgica. Se evaluó la validez, confiabilidad, sensibilidad al cambio y la utilidad de la escala. •Resultados: Se hallaron diferencias significativas en los puntajes obtenidos del Observador y del Paciente, antes y después de la intervención quirúrgica (p<0.000); a excepción de prurito. La escala POSAS demostró ser altamente confiable para la Escala del Observador y del Paciente (α = 0.912 y 0.765). Hubo alta correlación en las evaluaciones de dos observadores para las variables ordinales de la Escala del Observador (r>0.6). La concordancia entre las evaluaciones de dos observadores para las variables categóricas de la Escala del Paciente fue buena para la evaluación antes de la intervención para pigmentación y relieve (κ>0.61). Se demostró que el instrumento es capaz de detectar cambios clínicos en el tiempo (p<0.0000), a excepción de prurito (p= 0.271). •Conclusiones: La escala POSAS demostró ser un instrumento válido, confiable y útil para evaluar la calidad de la cicatriz en pacientes con CHT y CQ cómo secuelas de quemadura.

Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist

Background: The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114). Methods: 75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item. Results: 88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.Conclusions: Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved.

The family attitude scale: Reliability and validity of a new scale for measuring the emotional climate of families

Research on outcomes from psychiatric disorders has highlighted the importance of expressed emotion (EE), but its cost-effective measurement remains a challenge. This article describes development of the Family Attitude Scale (FAS), a 30-item instrument that can be completed by any informant. Its psychometric characteristics are reported in parents of undergraduate students and in 70 families with a schizophrenic member. The total FAS had high internal consistency in all samples, and reports of angry behaviour in FAS items showed acceptable inter-rater agreement. The FAS was associated with the reported anger, anger expression and anxiety of respondents. Substantial associations between the parents' FAS and the anger and anger expression of students was also observed. Parents of schizophrenic patients had higher FAS scores than parents of students, and the FAS was higher if disorder duration was longer or patient functioning was poorer. Hostility, high criticism and low warmth on the Camberwell Family Interview (CFI) were associated with a more negative FAS. The highest FAS in the family was a good predictor of a highly critical environment on the CFI. The FAS is a reliable and valid indicator of relationship stress and expressed anger that has wide applicability. (C) 1997 Elsevier Science Ireland Ltd.

Content validity and reliability of test of gross motor development in Chilean children

ABSTRACT OBJECTIVE To validate a Spanish version of the Test of Gross Motor Development (TGMD-2) for the Chilean population. METHODS Descriptive, transversal, non-experimental validity and reliability study. Four translators, three experts and 92 Chilean children, from five to 10 years, students from a primary school in Santiago, Chile, have participated. The Committee of Experts has carried out translation, back-translation and revision processes to determine the translinguistic equivalence and content validity of the test, using the content validity index in 2013. In addition, a pilot implementation was achieved to determine test reliability in Spanish, by using the intraclass correlation coefficient and Bland-Altman method. We evaluated whether the results presented significant differences by replacing the bat with a racket, using T-test. RESULTS We obtained a content validity index higher than 0.80 for language clarity and relevance of the TGMD-2 for children. There were significant differences in the object control subtest when comparing the results with bat and racket. The intraclass correlation coefficient for reliability inter-rater, intra-rater and test-retest reliability was greater than 0.80 in all cases. CONCLUSIONS The TGMD-2 has appropriate content validity to be applied in the Chilean population. The reliability of this test is within the appropriate parameters and its use could be recommended in this population after the establishment of normative data, setting a further precedent for the validation in other Latin American countries.

Validity and reliability of the Spanish version of the DN4 (Douleur Neuropathique 4 questions) questionnaire for differential diagnosis of pain syndromes associated to a neuropathic or somatic component

BACKGROUND This study assesses the validity and reliability of the Spanish version of DN4 questionnaire as a tool for differential diagnosis of pain syndromes associated to a neuropathic (NP) or somatic component (non-neuropathic pain, NNP). METHODS A study was conducted consisting of two phases: cultural adaptation into the Spanish language by means of conceptual equivalence, including forward and backward translations in duplicate and cognitive debriefing, and testing of psychometric properties in patients with NP (peripheral, central and mixed) and NNP. The analysis of psychometric properties included reliability (internal consistency, inter-rater agreement and test-retest reliability) and validity (ROC curve analysis, agreement with the reference diagnosis and determination of sensitivity, specificity, and positive and negative predictive values in different subsamples according to type of NP). RESULTS A sample of 164 subjects (99 women, 60.4%; age: 60.4 +/- 16.0 years), 94 (57.3%) with NP (36 with peripheral, 32 with central, and 26 with mixed pain) and 70 with NNP was enrolled. The questionnaire was reliable [Cronbach's alpha coefficient: 0.71, inter-rater agreement coefficient: 0.80 (0.71-0.89), and test-retest intra-class correlation coefficient: 0.95 (0.92-0.97)] and valid for a cut-off value > or = 4 points, which was the best value to discriminate between NP and NNP subjects. DISCUSSION This study, representing the first validation of the DN4 questionnaire into another language different than the original, not only supported its high discriminatory value for identification of neuropathic pain, but also provided supplemental psychometric validation (i.e. test-retest reliability, influence of educational level and pain intensity) and showed its validity in mixed pain syndromes.

Reliability of the Brief Neuropsychological Assessment Instrument Neupsilin

This study aimed to evaluate the reliability of Neupsilin Brief Neuropsychological Assessment Instrument, a brief battery developed in Brazil. Hundred two Brazilian man and women participated, from 18 to 40 years of age. It was evaluated the test-retest reliability of the Neupsilin tasks and the reliability of the correction of the constructional praxis task by different evaluators. The data were analyzed by Spearman’s correlation, intraclass correlation and Cronbach’s alpha. Language, memory, praxis and executive functions presented the highest correlations in the test-retest analyses. The agreement in the correction of the constructional praxis task was moderate to high. The results indicate temporal reliability of Neupsilin tasks and inter-rater agreement in the correction of the constructional praxis task. Suggestions to improve the tasks, the validity and reliability of Neupsilin were presented.

The reliability of counting actinic keratosis

Many epidemiological studies and clinical trials have been performed concerning actinic keratoses. The most eligible endpoint in the majority of articles is counting of actinic keratoses before and after treatments, nevertheless some authors support that this is not a reliable form of evaluation. The aim of this study was to evaluate the actinic keratoses counting by various raters and suggest approaches to increase the reliability. Cross-sectional study: forty-three patients were evaluated by four raters (inter- and intra-rater assessment) on the face and forearms. The mean actinic keratoses counts on the face and forearms were 7.7 and 9.1. The overall agreement among the raters for the facial and forearm actinic keratoses was 0.74 and 0.77. The intra-rater assessment showed high rates of agreement for the face (ICC = 0.93) and forearms (ICC = 0.83). Higher agreement occurred when counting up to five lesions. Four raters led to increased measurement variability and loss of reliability. Higher rates of agreement may be achieved with small number of lesions, limitation and/or segmentation of body areas to reduce their number, in AK prevention designs, are strategies that may lead to a greater reliability of these measurements. © 2013 Springer-Verlag Berlin Heidelberg.

Towards criterion validity in classroom language analysis: methodological constraints of metadiscourse and inter-rater agreement

This paper reports on a process to validate a revised version of a system for coding classroom discourse in foreign language lessons, a context in which the dual role of language (as content and means of communication) and the speakers' specific pedagogical aims lead to a certain degree of ambiguity in language analysis. The language used by teachers and students has been extensively studied, and a framework of concepts concerning classroom discourse well-established. Models for coding classroom language need, however, to be revised when they are applied to specific research contexts. The application and revision of an initial framework can lead to the development of earlier models, and to the re-definition of previously established categories of analysis that have to be validated. The procedures followed to validate a coding system are related here as guidelines for conducting research under similar circumstances. The advantages of using instruments that incorporate two types of data, that is, quantitative measures and qualitative information from raters' metadiscourse, are discussed, and it is suggested that such procedure can contribute to the process of validation itself, towards attaining reliability of research results, as well as indicate some constraints of the adopted research methodology.

The Development, Reliability, and Validity of the OARS Multidimensional Functional Assessment Questionnaire

This report outlines the development, validity, and reliability of Part A of the OARS Multidimensional Functional Assessment Questionnaire. Part A permits assessment of individuals' functioning on each of five dimensions (social, economic, mental health, physical health and self-care capacity), the detailed information in each area being summarized on a 6-point rating scale by a rater. Content and consensual validity were ensured by the manner of construction. Information on criterion validity was obtained for all dimensions except social. The criterion used and their associated Kendall's Tau values were: an objective economic scale (.62); ratings based on personal interviews by geropsychiatrists (.60); physician's associates (.82); and physical therapists (.89). For 11 geographically dispersed raters from research and clinic settings, intraclass correlational coefficients, based on 30 subjects, ranged from .66 on physical health to .87 in self-care capacity; 74% of the ratings were in complete agreement, 24% differed by one point.

Validity And Reliability Of A Pediatric Patient Classification Instrument.

to assess the construct validity and reliability of the Pediatric Patient Classification Instrument. correlation study developed at a teaching hospital. The classification involved 227 patients, using the pediatric patient classification instrument. The construct validity was assessed through the factor analysis approach and reliability through internal consistency. the Exploratory Factor Analysis identified three constructs with 67.5% of variance explanation and, in the reliability assessment, the following Cronbach's alpha coefficients were found: 0.92 for the instrument as a whole; 0.88 for the Patient domain; 0.81 for the Family domain; 0.44 for the Therapeutic procedures domain. the instrument evidenced its construct validity and reliability, and these analyses indicate the feasibility of the instrument. The validation of the Pediatric Patient Classification Instrument still represents a challenge, due to its relevance for a closer look at pediatric nursing care and management. Further research should be considered to explore its dimensionality and content validity.

Validity and reliability of methods for the detection of secondary caries around amalgam restorations in primary teeth

Secondary caries has been reported as the main reason for restoration replacement. The aim of this in vitro study was to evaluate the performance of different methods - visual inspection, laser fluorescence (DIAGNOdent), radiography and tactile examination - for secondary caries detection in primary molars restored with amalgam. Fifty-four primary molars were photographed and 73 suspect sites adjacent to amalgam restorations were selected. Two examiners evaluated independently these sites using all methods. Agreement between examiners was assessed by the Kappa test. To validate the methods, a caries-detector dye was used after restoration removal. The best cut-off points for the sample were found by a Receiver Operator Characteristic (ROC) analysis, and the area under the ROC curve (Az), and the sensitivity, specificity and accuracy of the methods were calculated for enamel (D2) and dentine (D3) thresholds. These parameters were found for each method and then compared by the McNemar test. The tactile examination and visual inspection presented the highest inter-examiner agreement for the D2 and D3 thresholds, respectively. The visual inspection also showed better performance than the other methods for both thresholds (Az = 0.861 and Az = 0.841, respectively). In conclusion, the visual inspection presented the best performance for detecting enamel and dentin secondary caries in primary teeth restored with amalgam.