Inter-observer agreement for spectral- and time-domain optical coherence tomography image grading: a prospective study

The purpose of this study was to compare inter-observer agreement of Stratus™ OCT versus Spectralis™ OCT image grading in patients with neovascular age-related macular degeneration (AMD). Thirty eyes with neovascular AMD were examined with Stratus™ OCT and Spectralis™ OCT. Four different scan protocols were used for imaging. Three observers graded the images for the presence of various pathologies. Inter-observer agreement between OCT models was assessed by calculating intra-class correlation coefficients (ICC). In Stratus™ OCT highest interobserver agreement was found for subretinal fluid (ICC: 0.79), and in Spectralis™ OCT for intraretinal cysts (IRC) (ICC: 0.93). Spectralis™ OCT showed superior interobserver agreement for IRC and epiretinal membranes (ERM) (ICC(Stratus™): for IRC 0.61; for ERM 0.56; ICC(Spectralis™): for IRC 0.93; for ERM 0.84). Increased image resolution of Spectralis™ OCT did improve the inter-observer agreement for grading intraretinal cysts and epiretinal membranes but not for other retinal changes.

Io volcano observer (IVO) integrated approach to optimizing system design for radiation challenges

One of the major challenges for a mission to the Jovian system is the radiation tolerance of the spacecraft (S/C) and the payload. Moreover, being able to achieve science observations with high signal to noise ratios (SNR), while passing through the high flux radiation zones, requires additional ingenuity on the part of the instrument provider. Consequently, the radiation mitigation is closely intertwined with the payload, spacecraft and trajectory design, and requires a systems-level approach. This paper presents a design for the Io Volcano Observer (IVO), a Discovery mission concept that makes multiple close encounters with Io while orbiting Jupiter. The mission aims to answer key outstanding questions about Io, especially the nature of its intense active volcanism and the internal processes that drive it. The payload includes narrow-angle and wide-angle cameras (NAC and WAC), dual fluxgate magnetometers (FGM), a thermal mapper (ThM), dual ion and neutral mass spectrometers (INMS), and dual plasma ion analyzers (PIA). The radiation mitigation is implemented by drawing upon experiences from designs and studies for missions such as the Radiation Belt Storm Probes (RBSP) and Jupiter Europa Orbiter (JEO). At the core of the radiation mitigation is IVO's inclined and highly elliptical orbit, which leads to rapid passes through the most intense radiation near Io, minimizing the total ionizing dose (177 krads behind 100 mils of Aluminum with radiation design margin (RDM) of 2 after 7 encounters). The payload and the spacecraft are designed specifically to accommodate the fast flyby velocities (e.g. the spacecraft is radioisotope powered, remaining small and agile without any flexible appendages). The science instruments, which collect the majority of the high-priority data when close to Io and thus near the peak flux, also have to mitigate transient noise in their detectors. The cameras use a combination of shielding and CMOS detectors with extremely fast readout to mi- imize noise. INMS microchannel plate detectors and PIA channel electron multipliers require additional shielding. The FGM is not sensitive to noise induced by energetic particles and the ThM microbolometer detector is nearly insensitive. Detailed SNR calculations are presented. To facilitate targeting agility, all of the spacecraft components are shielded separately since this approach is more mass efficient than using a radiation vault. IVO uses proven radiation-hardened parts (rated at 100 krad behind equivalent shielding of 280 mils of Aluminum with RDM of 2) and is expected to have ample mass margin to increase shielding if needed.

Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

OBJECTIVE: To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. DESIGN: Observer agreement study. DATA SOURCES: A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. DATA EXTRACTION: Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. RESULTS: The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. CONCLUSIONS: Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.

Shying away: An observer-based microprocess analysis of experiential avoidance and resource activation in outpatients with generalized anxiety spectrum disorder

Despite long-standing calls for patient-focused research on individuals with generalized anxiety spectrum disorder there is little systematized knowledge about the in-session behaviors of these patients. The primary objective of this study was to describe of in-session trajectories of the patients' level of explication (as an indicator of an elaborated exposure of negative emotionality) and the patients' focus on their own resources and how these trajectories are associated with post-treatment outcome. In respect to GAD patients, a high level of explication might be seen as an indicator of successful exposure of avoided negative emotionality during therapy sessions. Observers made minute-by-minute ratings of 1100 minutes of video of 20 patients-therapists dyads. The results indicated that a higher level of explication generally observed at a later stage during the therapy sessions and the patients' focus on competencies at an early stage was highly associated with positive therapy outcome at assessment at post treatment, independent of pretreatment distress, rapid response of well-being and symptom reduction, as well as the therapists' professional experience and therapy lengths. These results will be discussed under the perspective of emotion regulation of patients and therapist's counterregulation. It is assumed that GAD-Patients are especially skilled in masking difficult emotions. Explication level and emotion regulation are important variables for this patient group but there's relation to outcome is different.

Inter-observer agreement for diagnostic classification of esophageal motility disorders defined in high-resolution manometry

High-resolution esophageal manometry (HRM) is a recent development used in the evaluation of esophageal function. Our aim was to assess the inter-observer agreement for diagnosis of esophageal motility disorders using this technology. Practitioners registered on the HRM Working Group website were invited to review and classify (i) 147 individual water swallows and (ii) 40 diagnostic studies comprising 10 swallows using a drop-down menu that followed the Chicago Classification system. Data were presented using a standardized format with pressure contours without a summary of HRM metrics. The sequence of swallows was fixed for each user but randomized between users to avoid sequence bias. Participants were blinded to other entries. (i) Individual swallows were assessed by 18 practitioners (13 institutions). Consensus agreement (≤2/18 dissenters) was present for most cases of normal peristalsis and achalasia but not for cases of peristaltic dysmotility. (ii) Diagnostic studies were assessed by 36 practitioners (28 institutions). Overall inter-observer agreement was 'moderate' (kappa 0.51) being 'substantial' (kappa > 0.7) for achalasia type I/II and no lower than 'fair-moderate' (kappa >0.34) for any diagnosis. Overall agreement was somewhat higher among those that had performed >400 studies (n = 9; kappa 0.55) and 'substantial' among experts involved in development of the Chicago Classification system (n = 4; kappa 0.66). This prospective, randomized, and blinded study reports an acceptable level of inter-observer agreement for HRM diagnoses across the full spectrum of esophageal motility disorders for a large group of clinicians working in a range of medical institutions. Suboptimal agreement for diagnosis of peristaltic motility disorders highlights contribution of objective HRM metrics.

Quantitative multi-slice computed tomography assessment of the mitral valvular complex for transcatheter mitral valve interventions part 1: systematic measurement methodology and inter-observer variability

AIMS Transcatheter mitral valve replacement (TMVR) is an emerging technology with the potential to treat patients with severe mitral regurgitation at excessive risk for surgical mitral valve surgery. Multimodal imaging of the mitral valvular complex and surrounding structures will be an important component for patient selection for TMVR. Our aim was to describe and evaluate a systematic multi-slice computed tomography (MSCT) image analysis methodology that provides measurements relevant for transcatheter mitral valve replacement. METHODS AND RESULTS A systematic step-by-step measurement methodology is described for structures of the mitral valvular complex including: the mitral valve annulus, left ventricle, left atrium, papillary muscles and left ventricular outflow tract. To evaluate reproducibility, two observers applied this methodology to a retrospective series of 49 cardiac MSCT scans in patients with heart failure and significant mitral regurgitation. For each of 25 geometrical metrics, we evaluated inter-observer difference and intra-class correlation. The inter-observer difference was below 10% and the intra-class correlation was above 0.81 for measurements of critical importance in the sizing of TMVR devices: the mitral valve annulus diameters, area, perimeter, the inter-trigone distance, and the aorto-mitral angle. CONCLUSIONS MSCT can provide measurements that are important for patient selection and sizing of TMVR devices. These measurements have excellent inter-observer reproducibility in patients with functional mitral regurgitation.

An examination chair to measure internal rotation of the hip in routine settings: a validation study

OBJECTIVE: To determine the performance of a newly developed examination chair as compared with the clinical standard of assessing internal rotation (IR) of the flexed hip with a goniometer. METHODS: The examination chair allowed measurement of IR in a sitting position simultaneously in both hips, with hips and knees flexed 90 degrees, lower legs hanging unsupported and a standardized load of 5 kg applied to both ankles using a bilateral pulley system. Clinical assessment of IR was performed in supine position with hips and knees flexed 90 degrees using a goniometer. Within the framework of a population-based inception cohort study, we calculated inter-observer agreement in two samples of 84 and 64 consecutive, unselected young asymptomatic males using intra-class correlation coefficients (ICC) and determined the correlation between IR assessed with examination chair and clinical assessment. RESULTS: Inter-observer agreement was excellent for the examination chair (ICC right hip, 0.92, 95% confidence interval [CI] 0.89-0.95; ICC left hip, 0.90, 95% CI 0.86-0.94), and considerably higher than that seen with clinical assessment (ICC right hip, 0.65, 95% CI 0.49-0.77; ICC left hip, 0.69, 95% CI 0.54-0.80, P for difference in ICC between examination chair and clinical assessment