Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist

Background: The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114). Methods: 75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item. Results: 88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.Conclusions: Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved.

Population PK-guided simulation study in children exposed to drugs in human milk

Risks of significant infant drug exposure through human milk arepoorly defined due to lack of large-scale PK data. We propose to useBayesian approach based on population PK (popPK)-guided modelingand simulation for risk prediction. As a proof-of-principle study, weexploited fluoxetine milk concentration data from 25 women. popPKparameters including milk-to-plasma ratio (MP ratio) were estimatedfrom the best model. The dose of fluoxetine the breastfed infant wouldreceive through mother's milk, and infant plasma concentrations wereestimated from 1000 simulated mother-infant pairs, using randomassignment of feeding times and milk volume. A conservative estimateof CYP2D6 activity of 20% of the allometrically-adjusted adult valuewas assumed. Derived model parameters, including MP ratio were consistentwith those reported in the literature. Visual predictive check andother model diagnostics showed no signs of model misspecifications.The model simulation predicted that infant exposure levels to fluoxetinevia mother's milk were below 10% of weight-adjusted maternal therapeuticdoses in >99% of simulated infants. Predicted median ratio ofinfant-mother serum levels at steady state was 0.093 (range 0.033-0.31),consistent with literature reported values (mean=0.07; range 0-0.59).Predicted incidence of relatively high infant-mother ratio (>0.2) ofsteady-state serum fluoxetine concentrations was <1.3%. Overall, ourpredictions are consistent with clinical observations. Our approach maybe valid for other drugs, allowing in silico prediction of infant drugexposure risks through human milk. We will discuss application of thisapproach to another drug used in lactating women.

Prediction of infant drug exposure through breastfeeding: population PK modeling and simulation of fluoxetine

BACKGROUND: Risks of significant infant drug exposurethrough breastmilk are poorly defined for many drugs, and largescalepopulation data are lacking. We used population pharmacokinetics(PK) modeling to predict fluoxetine exposure levels ofinfants via mother's milk in a simulated population of 1000 motherinfantpairs.METHODS: Using our original data on fluoxetine PK of 25breastfeeding women, a population PK model was developed withNONMEM and parameters, including milk concentrations, wereestimated. An exponential distribution model was used to account forindividual variation. Simulation random and distribution-constrainedassignment of doses, dosing time, feeding intervals and milk volumewas conducted to generate 1000 mother-infant pairs with characteristicssuch as the steady-state serum concentrations (Css) and infantdose relative to the maternal weight-adjusted dose (relative infantdose: RID). Full bioavailability and a conservative point estimate of1-month-old infant CYP2D6 activity to be 20% of the adult value(adjusted by weigth) according to a recent study, were assumed forinfant Css calculations.RESULTS: A linear 2-compartment model was selected as thebest model. Derived parameters, including milk-to-plasma ratios(mean: 0.66; SD: 0.34; range, 0 - 1.1) were consistent with the valuesreported in the literature. The estimated RID was below 10% in >95%of infants. The model predicted median infant-mother Css ratio was0.096 (range 0.035 - 0.25); literature reported mean was 0.07 (range0-0.59). Moreover, the predicted incidence of infant-mother Css ratioof >0.2 was less than 1%.CONCLUSION: Our in silico model prediction is consistent withclinical observations, suggesting that substantial systemic fluoxetineexposure in infants through human milk is rare, but further analysisshould include active metabolites. Our approach may be valid forother drugs. [supported by CIHR and Swiss National Science Foundation(SNSF)]

Relatedness influences signal reliability in evolving robots.

Communication is an indispensable component of animal societies, yet many open questions remain regarding the factors affecting the evolution and reliability of signalling systems. A potentially important factor is the level of genetic relatedness between signallers and receivers. To quantitatively explore the role of relatedness in the evolution of reliable signals, we conducted artificial evolution over 500 generations in a system of foraging robots that can emit and perceive light signals. By devising a quantitative measure of signal reliability, and comparing independently evolving populations differing in within-group relatedness, we show a strong positive correlation between relatedness and reliability. Unrelated robots produced unreliable signals, whereas highly related robots produced signals that reliably indicated the location of the food source and thereby increased performance. Comparisons across populations also revealed that the frequency for signal production-which is often used as a proxy of signal reliability in empirical studies on animal communication-is a poor predictor of signal reliability and, accordingly, is not consistently correlated with group performance. This has important implications for our understanding of signal evolution and the empirical tools that are used to investigate communication.

Reliability and validity of the cross-culturally adapted French version of the Core Outcome Measures Index (COMI) in patients with low back pain.

PURPOSE: To conduct a cross-cultural adaptation of the Core Outcome Measures Index (COMI) into French according to established guidelines. METHODS: Seventy outpatients with chronic low back pain were recruited from six spine centres in Switzerland and France. They completed the newly translated COMI, and the Roland Morris disability (RMQ), Dallas Pain (DPQ), adjectival pain rating scale, WHO Quality of Life, and EuroQoL-5D questionnaires. After ~14 days RMQ and COMI were completed again to assess reproducibility; a transition question (7-point Likert scale; "very much worse" through "no change" to "very much better") indicated any change in status since the first questionnaire. RESULTS: COMI whole scores displayed no floor effects and just 1.5% ceiling effects. The scores for the individual COMI items correlated with their corresponding full-length reference questionnaire with varying strengths of correlation (0.33-0.84, P < 0.05). COMI whole scores showed a very good correlation with the "multidimensional" DPQ global score (Rho = 0.71). 55 patients (79%) returned a second questionnaire with no/minimal change in their back status. The reproducibility of individual COMI 5-point items was good, with test-retest differences within one grade ranging from 89% for 'social/work disability' to 98% for 'symptom-specific well-being'. The intraclass correlation coefficient for the COMI whole score was 0.85 (95% CI 0.76-0.91). CONCLUSIONS: In conclusion, the French version of this short, multidimensional questionnaire showed good psychometric properties, comparable to those reported for German and Spanish versions. The French COMI represents a valuable tool for future multicentre clinical studies and surgical registries (e.g. SSE Spine Tango) in French-speaking countries.

A neuropsychological instrument measuring age-related cerebral decline in older drivers: development, reliability, and validity of MedDrive.

When facing age-related cerebral decline, older adults are unequally affected by cognitive impairment without us knowing why. To explore underlying mechanisms and find possible solutions to maintain life-space mobility, there is a need for a standardized behavioral test that relates to behaviors in natural environments. The aim of the project described in this paper was therefore to provide a free, reliable, transparent, computer-based instrument capable of detecting age-related changes on visual processing and cortical functions for the purposes of research into human behavior in computational transportation science. After obtaining content validity, exploring psychometric properties of the developed tasks, we derived (Study 1) the scoring method for measuring cerebral decline on 106 older drivers aged ≥70 years attending a driving refresher course organized by the Swiss Automobile Association to test the instrument's validity against on-road driving performance (106 older drivers). We then validated the derived method on a new sample of 182 drivers (Study 2). We then measured the instrument's reliability having 17 healthy, young volunteers repeat all tests included in the instrument five times (Study 3) and explored the instrument's psychophysical underlying functions on 47 older drivers (Study 4). Finally, we tested the instrument's responsiveness to alcohol and effects on performance on a driving simulator in a randomized, double-blinded, placebo, crossover, dose-response, validation trial including 20 healthy, young volunteers (Study 5). The developed instrument revealed good psychometric properties related to processing speed. It was reliable (ICC = 0.853) and showed reasonable association to driving performance (R (2) = 0.053), and responded to blood alcohol concentrations of 0.5 g/L (p = 0.008). Our results suggest that MedDrive is capable of detecting age-related changes that affect processing speed. These changes nevertheless do not necessarily affect driving behavior.

Intraclass reliability of the Alberta Infant Motor Scale in the Brazilian version

This study had as its objective to analyze the intraclass reliability of the Alberta Infant Motor Scale (AIMS), in the Brazilian version, in preterm and term infants. It was a methodological study, conducted from November 2009 to April 2010, with 50 children receiving care in two public institutions in Fortaleza, Ceará, Brazil. Children were grouped according to gestational age as preterm and term, and evaluated by three evaluators in the communication laboratory of a public institution or at home. The intraclass correlation indices for the categories prone, supine, sitting and standing ranged from 0.553 to 0.952; most remained above 0.800, except for the standing category of the third evaluator, in which the index was 0.553. As for the total score and percentile, rates ranged from 0.843 to 0.954. The scale proved to be a reliable instrument for assessing gross motor performance of Brazilian children, particularly in Ceará, regardless of gestational age at birth.

Validity and reliability of a questionnaire on knowledge of cardiovascular risk factors for use in Brazil

Using a sample of patients with coronary artery disease, this methodological study aimed to conduct a cross-cultural adaptation and validation of a questionnaire on knowledge of cardiovascular risk factors (Q-FARCS), lifestyle changes, and treatment adherence for use in Brazil. The questionnaire has three scales: general knowledge of risk factors (RFs); specific knowledge of these RFs; and lifestyle changes achieved. Cross-cultural adaptation included translation, synthesis, back-translation, expert committee review, and pretesting. Face and content validity, reliability, and construct validity were measured. Cronbach’s alpha for the total sample (n = 240) was 0.75. Assessment of psychometric properties revealed adequate face and content validity, and the construct revealed seven components. It was concluded that the Brazilian version of Q-FARCS had adequate reliability and validity for the assessment of knowledge of cardiovascular RFs.

Impact of the disease: acceptability, ceiling and floor effects and reliability of an instrument on heart failure

This study evaluated the acceptability, ceiling/floor effects, and the reliability of the instrument for measuring the Impact of the Disease on the Daily Life of Patients with Valvular Disease (IDCV) when applied to 135 patients with heart failure (HF). Acceptability was evaluated by the percentage of unanswered items and by the proportion of patients who responded to all items; the ceiling/floor effects by the percentage of patients who scored in the top of 10% best and worst results of the scale, respectively. Reliability was estimated by internal consistency (Cronbach's alpha coefficient) and stability of the measure (intraclass correlation coefficient - ICC). All patients responded to all items. Ceiling/floor effects evidenced were of moderate magnitude. The Cronbach's alpha was satisfactory for the majority of the domains and ICC> 0.90 in all the domains. The IDCV proved to be an easy to understand questionnaire, with evidence of reliability in patients with HF.

Scientific Literature on the Reliability and Validity of the Manchester Triage System (MTS) Protocol: A Integrative Literature Review

OBJECTIVE To analyze the scientific production about the validity and reliability of the Manchester Triage System (MTS) protocol. METHOD A descriptive study of an integrative literature review. Articles on the validity and reliability of the MTS developed with children and adults published between 1999 and 2013 were included. RESULTS 14 articles were selected from a total of 8438, nine of validity and five of reliability. The reliability of the MTS ranged from moderate to almost perfect, with higher intra-evaluation. Regarding validity, the results seem to point to equivalent and satisfactory sensibility and specificity levels of the MTS. The instrument proved to be a good predictor of the need for hospitalization and of hospital mortality. CONCLUSION The reliability and validity of the MTS obtained in the studies is varied. It is recommended that new studies indicate necessary modifications to the MTS so that it is more safely used by nurses.

Effects of web-based electrocardiography simulation on strategies and learning styles

OBJECTIVETo identify the association between the use of web simulation electrocardiography and the learning approaches, strategies and styles of nursing degree students.METHODA descriptive and correlational design with a one-group pretest-posttest measurement was used. The study sample included 246 students in a Basic and Advanced Cardiac Life Support nursing class of nursing degree.RESULTSNo significant differences between genders were found in any dimension of learning styles and approaches to learning. After the introduction of web simulation electrocardiography, significant differences were found in some item scores of learning styles: theorist (p < 0.040), pragmatic (p < 0.010) and approaches to learning.CONCLUSIONThe use of a web electrocardiogram (ECG) simulation is associated with the development of active and reflexive learning styles, improving motivation and a deep approach in nursing students.

Spacial debris collector robot : An AI-based and autonomous debris collector satellite simulation

In this paper the core functions of an artificial intelligence (AI) for controlling a debris collector robot are designed and implemented. Using the robot operating system (ROS) as the base of this work a multi-agent system is built with abilities for task planning.