Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

Background: The design of Virtual Patients (VPs) is essential. So far there are no validated evaluation instruments for VP design published. Summary of work: We examined three sources of validity evidence of an instrument to be filled out by students aimed at measuring the quality of VPs with a special emphasis on fostering clinical reasoning: (1) Content was examined based on theory of clinical reasoning and an international VP expert team. (2) Response process was explored in think aloud pilot studies with students and content analysis of free text questions accompanying each item of the instrument. (3) Internal structure was assessed by confirmatory factor analysis (CFA) using 2547 student evaluations and reliability was examined utilizing generalizability analysis. Summary of results: Content analysis was supported by theory underlying Gruppen and Frohna’s clinical reasoning model on which the instrument is based and an international VP expert team. The pilot study and analysis of free text comments supported the validity of the instrument. The CFA indicated that a three factor model comprising 6 items showed a good fit with the data. Alpha coefficients per factor were 0,74 - 0,82. The findings of the generalizability studies indicated that 40-200 student responses are needed in order to obtain reliable data on one VP. Conclusions: The described instrument has the potential to provide faculty with reliable and valid information about VP design. Take-home messages: We present a short instrument which can be of help in evaluating the design of VPs.

Reliability of lithium dilution cardiac output in anaesthetized sheep

BACKGROUND: Cardiac output (CO) measurement with lithium dilution (COLD) has not been fully validated in sheep using precise ultrasonic flow probe technology (COUFP). Sheep generate important cardiovascular research models and the use of COLD has become more popular in experimental settings. METHODS: Ultrasonic transit-time perivascular flow probes were surgically implanted on the pulmonary artery of 13 sheep. Paired COLD readings were taken at six time points, before and after implantation of a left ventricular assist device (LVAD) and compared with COUFP recorded just after lithium injection. RESULTS: The mean COLD was 5.7 litre min(-1) (range 3.8-9.6 litre min(-1)) and mean COUFP 5.9 litre min(-1) (range 4.0-9.2 litre min(-1)). The bias (standard deviation) was 0.3 (1.0) litre min(-1) [5.1 (16.9)%] and limits of agreement (LOA) were -1.7 to 2.3 litre min(-1) (-28.8 to 39.0%) with a percentage error (PE) of 34.4%. Data to assess trending [rate (95% confidence intervals)] included a 78 (62-93)% concordance rate in the four-quadrant plot (n=27). In the half moon polar plot (n=19), the mean polar angle was +5°, the radial LOA were -49 to +35° and 68 (47-89)% of data points fell within 22.5° of the mean polar angle. Both tests indicated moderate to poor trending ability. CONCLUSION: COLD is not precise when evaluated against COUFP in sheep based on the statistical criteria set, but the results are comparable with previously published animal studies. KEYWORDS:

Resource allocation with project management software

We present results of a benchmark test evaluating the resource allocation capabilities of the project management software packages Acos Plus.1 8.2, CA SuperProject 5.0a, CS Project Professional 3.0, MS Project 2000, and Scitor Project Scheduler 8.0.1. The tests are based on 1560 instances of precedence– and resource–constrained project scheduling problems. For different complexity scenarios, we analyze the deviation of the makespan obtained by the software packages from the best feasible makespan known. Among the tested software packages, Acos Plus.1 and Project Scheduler show the best resource allocation performance. Moreover, our numerical analysis reveals a considerable performance gap between the implemented methods and state–of–the–art project scheduling algorithms, especially for large–sized problems. Thus, there is still a significant potential for improving solutions to resource allocation problems in practice.

Reliability and quality of water isotope data collected with a lowbudget rain collector

RATIONALELow-budget rain collectors for water isotope analysis, such as the `ball-in-funnel type collector' (BiFC), are widely used in studies on stable water isotopes of rain. To date, however, an experimental quality assessment of such devices in relation to climatic factors does not exist. METHODSWe used Cavity Ring-Down Spectrometry (CRDS) to quantify the effects of evaporation on the O-18 values of reference water under controlled conditions as a function of the elapsed time between rainfall and collection for isotope analysis, the sample volume and the relative humidity (RH: 31% and 67%; 25 degrees C). The climate chamber conditions were chosen to reflect the warm and dry end of field conditions that favor evaporative enrichment (EE). We also tested the performance of the BiFC in the field, and compared our H-2/O-18 data obtained by isotope ratio mass spectrometry (IRMS) with those from the Swiss National Network for the Observation of Isotopes in the Water Cycle (ISOT). RESULTSThe EE increased with time, with a 1 increase in the O-18 values after 10days (RH: 25%; 25 degrees C; 35mL (corresponding to a 5mm rain event); p <0.001). The sample volume strongly affected the EE (max. value +1.5 parts per thousand for 7mL samples (i.e., 1mm rain events) after 72h at 31% and 67% RH; p <0.001), whereas the relative humidity had no significant effect. Using the BiFC in the field, we obtained very tight relationships of the H-2/O-18 values (r(2) 0.95) for three sites along an elevational gradient, not significantly different from that of the next ISOT station. CONCLUSIONSSince the chosen experimental conditions were extreme compared with the field conditions, it was concluded that the BiFC is a highly reliable and inexpensive collector of rainwater for isotope analysis. Copyright (c) 2014 John Wiley & Sons, Ltd.

Cardiothoracic ratio in postmortem computed tomography: reliability and threshold for the diagnosis of cardiomegaly

The aim of this study was to evaluate the reliability of the cardiothoracic ratio (CTR) in postmortem computed tomography (PMCT) and to assess a CTR threshold for the diagnosis of cardiomegaly based on the weight of the heart at autopsy. PMCT data of 170 deceased human adults were retrospectively evaluated by two blinded radiologists. The CTR was measured on axial computed tomography images and the actual cardiac weight was weighed at autopsy. Inter-rater reliability, sensitivity, and specificity were calculated. Receiver operating characteristic curves were calculated to assess enlarged heart weight by CTR. The autopsy definition of cardiomegaly was based on normal values of the Zeek method (within a range of both, one or two SD) and the Smith method (within the given range). Intra-class correlation coefficients demonstrated excellent agreements (0.983) regarding CTR measurements. In 105/170 (62 %) cases the CTR in PMCT was >0.5, indicating enlarged heart weight, according to clinical references. The mean heart weight measured in autopsy was 405 ± 105 g. As a result, 114/170 (67 %) cases were interpreted as having enlarged heart weights according to the normal values of Zeek within one SD, while 97/170 (57 %) were within two SD. 100/170 (59 %) were assessed as enlarged according to Smith's normal values. The sensitivity/specificity of the 0.5 cut-off of the CTR for the diagnosis of enlarged heart weight was 78/71 % (Zeek one SD), 74/55 % (Zeek two SD), and 76/59 % (Smith), respectively. The discriminative power between normal heart weight and cardiomegaly was 79, 73, and 74 % for the Zeek (1SD/2SD) and Smith methods respectively. Changing the CTR threshold to 0.57 resulted in a minimum specificity of 95 % for all three definitions of cardiomegaly. With a CTR threshold of 0.57, cardiomegaly can be identified with a very high specificity. This may be useful if PMCT is used by forensic pathologists as a screening tool for medico-legal autopsies.

Low reliability of perceptual priming: consequences for the interpretation of functional dissociations between explicit and implicit memory

In this study, three experiments are presented that investigate the reliability of memory measures. In Experiment 1, the well-known dissociation between explicit (recall, recognition) and implicit memory (picture clarification) as a function of age in a sample of 335 persons aged between 65 and 95 was replicated. Test-retest reliability was significantly lower in implicit than in explicit measures. In Experiment 2, parallel-test reliabilities in a student sample confirmed the finding of Experiment 1. In Experiment 3, the reliability of cued recall and word stem completion was investigated. There were significant priming effects and a dissociation between explicit and implicit memory as a function of levels of processing. However, the reliability of implicit memory measures was again substantially lower than in explicit tests in all test conditions. As a consequence, differential reliabilities of direct and indirect memory tests should be considered as a possible determinant of dissociations between explicit and implicit memory as a function of experimental or quasi-experimental manipulations.

Poor Reliability between Cochrane Reviewers and Blinded External Reviewers When Applying the Cochrane Risk of Bias Tool in Physical Therapy Trials

OBJECTIVES To test the inter-rater reliability of the RoB tool applied to Physical Therapy (PT) trials by comparing ratings from Cochrane review authors with those of blinded external reviewers. METHODS Randomized controlled trials (RCTs) in PT were identified by searching the Cochrane Database of Systematic Reviews for meta-analysis of PT interventions. RoB assessments were conducted independently by 2 reviewers blinded to the RoB ratings reported in the Cochrane reviews. Data on RoB assessments from Cochrane reviews and other characteristics of reviews and trials were extracted. Consensus assessments between the two reviewers were then compared with the RoB ratings from the Cochrane reviews. Agreement between Cochrane and blinded external reviewers was assessed using weighted kappa (κ). RESULTS In total, 109 trials included in 17 Cochrane reviews were assessed. Inter-rater reliability on the overall RoB assessment between Cochrane review authors and blinded external reviewers was poor (κ  =  0.02, 95%CI: -0.06, 0.06]). Inter-rater reliability on individual domains of the RoB tool was poor (median κ  = 0.19), ranging from κ  =  -0.04 ("Other bias") to κ  =  0.62 ("Sequence generation"). There was also no agreement (κ  =  -0.29, 95%CI: -0.81, 0.35]) in the overall RoB assessment at the meta-analysis level. CONCLUSIONS Risk of bias assessments of RCTs using the RoB tool are not consistent across different research groups. Poor agreement was not only demonstrated at the trial level but also at the meta-analysis level. Results have implications for decision making since different recommendations can be reached depending on the group analyzing the evidence. Improved guidelines to consistently apply the RoB tool and revisions to the tool for different health areas are needed.

Effects of externally supplied protein on root morphology and biomass allocation in Arabidopsis

Growth, morphogenesis and function of roots are influenced by the concentration and form of nutrients present in soils, including low molecular mass inorganicN(IN, ammonium, nitrate) and organicN(ON, e. g. amino acids). Proteins, ON of high molecular mass, are prevalent in soils but their possible effects on roots have received little attention. Here, we investigated how externally supplied protein of a size typical of soluble soil proteins influences root development of axenically grown Arabidopsis. Addition of low to intermediate concentrations of protein (bovine serum albumen, BSA) to IN-replete growth medium increased root dry weight, root length and thickness, and root hair length. Supply of higher BSA concentrations inhibited root development. These effects were independent of total N concentrations in the growth medium. The possible involvement of phytohormones was investigated using Arabidopsis with defective auxin (tir1-1 and axr2-1) and ethylene (ein2-1) responses. That no phenotype was observed suggests a signalling pathway is operating independent of auxin and ethylene responses. This study expands the knowledge on N form-explicit responses to demonstrate that ON of high molecular mass elicits specific responses.

Intra-session test-retest reliability of pelvic floor muscle electromyography during running

INTRODUCTION AND HYPOTHESIS The prevalence of female stress urinary incontinence is high, and young adults are also affected, including athletes, especially those involved in "high-impact" sports. To date there have been almost no studies testing pelvic floor muscle (PFM) activity during dynamic functional whole body movements. The aim of this study was the description and reliability test of PFM activity and time variables during running. METHODS A prospective cross-sectional study including ten healthy female subjects was designed with the focus on the intra-session test-retest reliability of PFM activity and time variables during running derived from electromyography (EMG) and accelerometry. RESULTS Thirteen variables were identified based on ten steps of each subject: Six EMG variables showed good reliability (ICC 0.906-0.942) and seven time variables did not show good reliability (ICC 0.113-0.731). Time variables (e.g. time difference between heel strike and maximal acceleration of vaginal accelerator) showed low reliability. However, relevant PFM EMG variables during running (e.g., pre-activation, minimal and maximal activity) could be identified and showed good reliability. CONCLUSION Further adaptations regarding measurement methods should be tested to gain better control of the kinetics and kinematics of the EMG probe and accelerometers. To our knowledge this is the first study to test the reliability of PFM activity and time variables during dynamic functional whole body movements. More knowledge of PFM activity and time variables may help to provide a deeper insight into physical strain with high force impacts and important functional reflexive contraction patterns of PFM to maintain or to restore continence.

Distributed Rate Allocation in Inter-Session Network Coding

In this work, we propose a distributed rate allocation algorithm that minimizes the average decoding delay for multimedia clients in inter-session network coding systems. We consider a scenario where the users are organized in a mesh network and each user requests the content of one of the available sources. We propose a novel distributed algorithm where network users determine the coding operations and the packet rates to be requested from the parent nodes, such that the decoding delay is minimized for all clients. A rate allocation problem is solved by every user, which seeks the rates that minimize the average decoding delay for its children and for itself. Since this optimization problem is a priori non-convex, we introduce the concept of equivalent packet flows, which permits to estimate the expected number of packets that every user needs to collect for decoding. We then decompose our original rate allocation problem into a set of convex subproblems, which are eventually combined to obtain an effective approximate solution to the delay minimization problem. The results demonstrate that the proposed scheme eliminates the bottlenecks and reduces the decoding delay experienced by users with limited bandwidth resources. We validate the performance of our distributed rate allocation algorithm in different video streaming scenarios using the NS-3 network simulator. We show that our system is able to take benefit of inter-session network coding for simultaneous delivery of video sessions in networks with path diversity.

Functional connectivity in BOLD and CBF data: similarity and reliability of resting brain networks

Resting-state functional connectivity (FC) fMRI (rs-fcMRI) offers an appealing approach to mapping the brain's intrinsic functional organization. Blood oxygen level dependent (BOLD) and arterial spin labeling (ASL) are the two main rs-fcMRI approaches to assess alterations in brain networks associated with individual differences, behavior and psychopathology. While the BOLD signal is stronger with a higher temporal resolution, ASL provides quantitative, direct measures of the physiology and metabolism of specific networks. This study systematically investigated the similarity and reliability of resting brain networks (RBNs) in BOLD and ASL. A 2×2×2 factorial design was employed where each subject underwent repeated BOLD and ASL rs-fcMRI scans on two occasions on two MRI scanners respectively. Both independent and joint FC analyses revealed common RBNs in ASL and BOLD rs-fcMRI with a moderate to high level of spatial overlap, verified by Dice Similarity Coefficients. Test-retest analyses indicated more reliable spatial network patterns in BOLD (average modal Intraclass Correlation Coefficients: 0.905±0.033 between-sessions; 0.885±0.052 between-scanners) than ASL (0.545±0.048; 0.575±0.059). Nevertheless, ASL provided highly reproducible (0.955±0.021; 0.970±0.011) network-specific CBF measurements. Moreover, we observed positive correlations between regional CBF and FC in core areas of all RBNs indicating a relationship between network connectivity and its baseline metabolism. Taken together, the combination of ASL and BOLD rs-fcMRI provides a powerful tool for characterizing the spatiotemporal and quantitative properties of RBNs. These findings pave the way for future BOLD and ASL rs-fcMRI studies in clinical populations that are carried out across time and scanners.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. METHODS/DESIGN This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. DISCUSSION This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.