Student's academic efficacy or Inefficacy? an example on how to evaluate the psychometric properties of a measuring instrument and evaluate the effects of item wording

This study evaluated the effect of item inversion on the construct validity and reliability of psychometric scales and proposed a theoretical framework for the evaluation of the psychometric properties of data gathered with psychometric instruments. To this propose, we used the Maslach Burnout Inventory, which is the most used psychometric inventory to measure burnout in different professional context (Students, Teachers, Police, Doctors, Nurses, etc…). The version of the MBI used was the MBI-Student Survey (MBI-SS). This inventory is composed of three key dimensions: Exhaustion, Cynicism and Professional Efficacy. The two first dimensions—which have positive formulated items—are moderate to strong positive correlated, and show moderate to strong negative correlations with the 3rd dimension—which has negative formulated items. We tested the hypothesis that, in college students, formulating the 3rd dimension of burnout as Inefficacy (reverting the negatively worded items in the Efficacy dimension) improves the correlation of the 3rd dimension with the other two dimensions, improves its internal consistency, and the overall MBI-SS’ construct validity and reliability. Confirmatory factor analysis results, estimated by Maximum Likelihood, revealed adequate factorial fit for both forms of the MBI-SS (with Efficacy) vs. the MBI-SSi (with Inefficacy). Also both forms showed adequate convergent and discriminant related validity. However, reliability and convergent validity were higher for the MBI-SSi. There were also stronger (positive) correlations between the 3 factors in MBI-SSi than the ones observed in MBI-SS. Results show that positively rewording of the 3rd dimension of the MBI-SS improves its validity and reliability. We therefore propose that the 3rd dimension of the MBI-SS should be named Professional Inefficacy and its items should be positively worded.

Analyzing the Complexity and Reliability of Switch-Frequency-Reconfigurable Antennas Using Graph Models

This paper addresses the functional reliability and the complexity of reconfigurable antennas using graph models. The correlation between complexity and reliability for any given reconfigurable antenna is defined. Two methods are proposed to reduce failures and improve the reliability of reconfigurable antennas. The failures are caused by the reconfiguration technique or by the surrounding environment. These failure reduction methods proposed are tested and examples are given which verify these methods.

Reliability in kimono grip strength tests and comparison between elite and non-elite Brazilian jiu-jitsu players

Background and Study Aim: The grip strength endurance is important for Brazilian Jiu-Jitsu (BJJ). Thus, the aims of this study were: a) to test the reliability of two kimono grip strength tests named maximum static lift (MSL) and maximum number of repetitions (MNR) and b) to examine differences between elite and non-elite BJJ players in these tests. Material/Methods: Thirty BJJ players participated into two phases: "A" to test reliability and "B" to compare elite and non-elite. In phase A, twenty participants performed the MSL and, 15 min later, the MNR in two occasions with 24-h interval. In phase B, ten other BJJ practitioners (non-elite) and ten athletes (elite) performed the same tests. The intraclass correlation coefficient (ICC) two way fixed model (3,1), Bland-Altman plot and the limits of agreement were used to test reliability, correlation between the tests were evaluated by Pearson correlations and independent T test (P<0.05) was utilized to compare elite vs. non-elite. Results: The ICC was high for repeated measurements on different days of phase A (MSL: r=0.99 and MNR: r=0.97). Limits of agreement for time of suspension were -6.9 to 2.4-s, with a mean difference of -2.3 s (CI: -3.3 to -1.2-s), while for number of repetitions the limits of agreement were -2.9 to 2.3-rep, with a mean difference of -0.3-rep (CI: -0.9 to 0.3-rep). In phase B, elite presented better performance for both tests (P<0.05) compared to non-elite (56 +/- 10-s vs. 37 +/- 11-s in MSL and 15 +/- 4-rep vs. 8 +/- 3-rep in MNR). Moderate correlation were found between MSL and MNR for absolute values during test (r=0.475; p=0.034), and retest phases (r=0.489; p=0.029), while moderate and high correlations in the test (r=0.615; p=0.004) and retest phases (r=0.716; p=0.001) were found for relative values, respectively. Conclusions: These proposed tests are reliable and both static and dynamic grip strength endurance tests seem to differentiate BJJ athletes from different levels.

Translation and Cross-cultural Adaptation of the Lower Extremity Functional Scale Into a Brazilian Portuguese Version and Validation on Patients With Knee Injuries

STUDY DESIGN: Clinical measurement. OBJECTIVE: To translate and culturally adapt the Lower Extremity Functional Scale (LEFS) into a Brazilian Portuguese version, and to test the construct and content validity and reliability of this version in patients with knee injuries. BACKGROUND: There is no Brazilian Portuguese version of an instrument to assess the function of the lower extremity after orthopaedic injury. METHODS: The translation of the original English version of the LEFS into a Brazilian Portuguese version was accomplished using standard guidelines and tested in 31 patients with knee injuries. Subsequently, 87 patients with a variety of knee disorders completed the Brazilian Portuguese LEES, the Medical Outcomes Study 36-Item Short-Form Health Survey, the Western Ontario and McMaster Universities Osteoarthritis Index, and the International Knee Documentation Committee Subjective Knee Evaluation Form and a visual analog scale for pain. All patients were retested within 2 days to determine reliability of these measures. Validation was assessed by determining the level of association between the Brazilian Portuguese LEFS and the other outcome measures. Reliability was documented by calculating internal consistency, test-retest reliability, and standard error of measurement. RESULTS: The Brazilian Portuguese LEES had a high level of association with the physical component of the Medical Outcomes Study 36-Item Short-Form Health Survey (r = 0.82), the Western Ontario and McMaster Universities Osteoarthritis Index (r = 0.87), the International Knee Documentation Committee Subjective Knee Evaluation Form (r = 0.82), and the pain visual analog scale (r = -0.60) (all, P<.05). The Brazilian Portuguese LEES had a low level of association with the mental component of the Medical Outcomes Study 36-Item Short-Form Health Survey (r = 0.38, P<.05). The internal consistency (Cronbach alpha = .952) and test-retest reliability (intraclass correlation coefficient = 0.957) of the Brazilian Portuguese version of the LEES were high. The standard error of measurement was low (3.6) and the agreement was considered high, demonstrated by the small differences between test and retest and the narrow limit of agreement, as observed in Bland-Altman and survival-agreement plots. CONCLUSION: The translation of the LEFS into a Brazilian Portuguese version was successful in preserving the semantic and measurement properties of the original version and was shown to be valid and reliable in a Brazilian population with knee injuries. J Ort hop Sports Phys Ther 2012;42(11):932-939, Epub 9 October 2012. doi:10.2519/jospt.2012.4101

Comparison of the reliability of multifocal visual evoked cortical potentials generated by pattern reversal and pattern pulse stimulation

This study compared the effectiveness of the multifocal visual evoked cortical potentials (mfVEP) elicited by pattern pulse stimulation with that of pattern reversal in producing reliable responses (signal-to-noise ratio >1.359). Participants were 14 healthy subjects. Visual stimulation was obtained using a 60-sector dartboard display consisting of 6 concentric rings presented in either pulse or reversal mode. Each sector, consisting of 16 checks at 99% Michelson contrast and 80 cd/m² mean luminance, was controlled by a binary m-sequence in the time domain. The signal-to-noise ratio was generally larger in the pattern reversal than in the pattern pulse mode. The number of reliable responses was similar in the central sectors for the two stimulation modes. At the periphery, pattern reversal showed a larger number of reliable responses. Pattern pulse stimuli performed similarly to pattern reversal stimuli to generate reliable waveforms in R1 and R2. The advantage of using both protocols to study mfVEP responses is their complementarity: in some patients, reliable waveforms in specific sectors may be obtained with only one of the two methods. The joint analysis of pattern reversal and pattern pulse stimuli increased the rate of reliability for central sectors by 7.14% in R1, 5.35% in R2, 4.76% in R3, 3.57% in R4, 2.97% in R5, and 1.78% in R6. From R1 to R4 the reliability to generate mfVEPs was above 70% when using both protocols. Thus, for a very high reliability and thorough examination of visual performance, it is recommended to use both stimulation protocols.

Validity of an automatic measure protocol in distal femur for allograft selection from a three-dimensional virtual bone bank system

Osteoarticular allograft is one possible treatment in wide surgical resections with large defects. Performing best osteoarticular allograft selection is of great relevance for optimal exploitation of the bone databank, good surgery outcome and patient’s recovery. Current approaches are, however, very time consuming hindering these points in practice. We present a validation study of a software able to perform automatic bone measurements used to automatically assess the distal femur sizes across a databank. 170 distal femur surfaces were reconstructed from CT data and measured manually using a size measure protocol taking into account the transepicondyler distance (A), anterior-posterior distance in medial condyle (B) and anterior-posterior distance in lateral condyle (C). Intra- and inter-observer studies were conducted and regarded as ground truth measurements. Manual and automatic measures were compared. For the automatic measurements, the correlation coefficients between observer one and automatic method, were of 0.99 for A measure and 0.96 for B and C measures. The average time needed to perform the measurements was of 16 h for both manual measurements, and of 3 min for the automatic method. Results demonstrate the high reliability and, most importantly, high repeatability of the proposed approach, and considerable speed-up on the planning.

A pain assessment scale for population-based studies: Development and validation of the Pain Module of the Standard Evaluation Questionnaire

The objectives of this study were to develop and validate a tool for assessing pain in population-based observational studies and to develop three subscales for back/neck, upper extremity and lower extremity pain. Based on a literature review, items were extracted from validated questionnaires and reviewed by an expert panel. The initial questionnaire consisted of a pain manikin and 34 items relating to (i) intensity of pain in different body regions (7 items), (ii) pain during activities of daily living (18 items) and (iii) various pain modalities (9 items). Psychometric validation of the initial questionnaire was performed in a random sample of the German-speaking Swiss population. Analyses included tests for reliability, correlation analysis, principal components factor analysis, tests for internal consistency and validity. Overall, 16,634 of 23,763 eligible individuals participated (70%). Test-retest reliability coefficients ranged from 0.32 to 0.97, but only three coefficients were below 0.60. Subscales were constructed combining four items for each of the subscales. Item-total coefficients ranged from 0.76 to 0.86 and Cronbach's alpha were 0.75 or higher for all subscales. Correlation coefficients between subscales and three validated instruments (WOMAC, SPADI and Oswestry) ranged from 0.62 to 0.79. The final Pain Standard Evaluation Questionnaire (SEQ Pain) included 28 items and the pain manikin and accounted for the multidimensionality of pain by assessing pain location and intensity, pain during activity, triggers and time of onset of pain and frequency of pain medication. It was found to be reliable and valid for the assessment of pain in population-based observational studies.

Cultural beliefs and hypertension in rural Philippines

Hypertension (HTN), the major risk factor for cardiovascular disease (CVD), is emerging as a major public health problem in the Philippines. CVD has been the leading cause of mortality in the Philippines since 1990. ^ Although research has shown that certain populations have a greater propensity for HTN, and that culture may be a factor, empirical investigations of the influence of cultural beliefs on HTN are lacking. ^ The operational aims of this study were to: (a) develop and examine the reliability (test-retest, internal consistency) and validity (content) of a questionnaire which measures factors related to HTN; (b) administer the questionnaire; and (c) measure blood pressure, height, and weight of the ≥ 30 year old residents of San Antonio, Nueva Ecija, Philippines. ^ The analytic aims were to determine the: (a) cultural beliefs relating to HTN; (b) associations between cultural beliefs and HTN; and (c) extent to which cultural beliefs versus biological, behavioral, socioeconomic, and access factors are associated with HTN. ^ A cluster survey was conducted among 336 residents ≥ 30 years old in May, 1998. Sixty clusters of households were derived using probability proportionate to size sampling technique. Seven households per cluster were visited and one respondent per household was randomly chosen for interview and measurement of blood pressure, height and weight. A response rate of 84% (336/400) was achieved. ^ Results showed that the test-retest reliability of cultural belief items was 0.69–0.96. Internal consistency reliability was 0.74. ^ HTN (SBP ≥ 140; or DBP ≥ 90 mmHg; or currently taking anti-hypertensive medication) prevalence was 23/100. Univariate logistic regression showed cultural beliefs to be significantly associated (p < 0.037) with HTN. However, multivariate analysis showed that only age ≥ 50 (p = 0.000), family history of HTN (p = 0.004) and body mass index ≥ 25 (p = 0.003) were significant predictors. ^ In the absence of fully implemented programs to prevent and control HTN, the current prevalence is only expected to increase, leading to substantial increases in morbidity and mortality and health care cost. It is recommended that research which focuses on designing, implementing, and evaluating culturally appropriate community-wide programs on HTN prevention and control be undertaken in this community. ^

Coloured Letters and Numbers (CLaN): A reliable factor-analysis based synaesthesia questionnaire

Synaesthesia is a heterogeneous phenomenon, even when considering one particular sub-type. The purpose of this study was to design a reliable and valid questionnaire for grapheme-colour synaesthesia that captures this heterogeneity. By the means of a large sample of 628 synaesthetes and a factor analysis, we created the Coloured Letters and Numbers (CLaN) questionnaire with 16 items loading on 4 different factors (i.e., localisation, automaticity/attention, deliberate use, and longitudinal changes). These factors were externally validated with tests which are widely used in the field of synaesthesia research. The questionnaire showed good test–retest reliability and construct validity (i.e., internally and externally). Our findings are discussed in the light of current theories and new ideas in synaesthesia research. More generally, the questionnaire is a useful tool which can be widely used in synaesthesia research to reveal the influence of individual differences on various performance measures and will be useful in generating new hypotheses.

Patient and Informant Views on Visual Hallucinations in Parkinson Disease.

OBJECTIVE Visual hallucinations (VHs) are a very personal experience, and it is not clear whether information about them is best provided by informants or patients. Some patients may not share their hallucinatory experiences with caregivers to avoid distress or for fear of being labeled insane, and others do not have informants at all, which limits the use of informant-based questionnaires. The aim of this study was to compare patient and caregiver views about VHs in Parkinson disease (PD), using the North-East Visual Hallucinations Interview (NEVHI). METHODS Fifty-nine PD patient-informant pairs were included. PD patients and informants were interviewed separately about VHs using the NEVHI. Informants were additionally interviewed using the four-item version of the Neuropsychiatric Inventory. Inter-reliability and concurrent validity of the different measures were compared. RESULTS VHs were more commonly reported by patients than informants. The inter-rater agreement between NEVHI-patient and NEVHI-informant was moderate for complex VHs (Cohen's kappa = 0.44; 95% confidence interval [CI]: 0.13-0.75; t = 3.43, df = 58, p = 0.001) and feeling of presence (Cohen's kappa = 0.35; 95% CI: 0.00-0.70; t = 2.75, df = 58, p = 0.006), but agreement was poor for illusions (Cohen's kappa = 0.25; 95% CI: -0.07-0.57; t = 2.36, df = 58, p = 0.018) and passage hallucinations (Cohen's kappa = 0.16; 95% CI: -0.04-0.36; t = 2.26, df = 58, p = 0.024). CONCLUSION When assessing VHs in PD patients, it is best to rely on patient information, because not all patients share the details of their hallucinations with their caregivers.

The new final Clinical Skills examination in human medicine in Switzerland: Essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group

Objective: Since 2011, the new national final examination in human medicine has been implemented in Switzerland, with a structured clinical-practical part in the OSCE format. From the perspective of the national Working Group, the current article describes the essential steps in the development, implementation and evaluation of the Federal Licensing Examination Clinical Skills (FLE CS) as well as the applied quality assurance measures. Finally, central insights gained from the last years are presented. Methods: Based on the principles of action research, the FLE CS is in a constant state of further development. On the foundation of systematically documented experiences from previous years, in the Working Group, unresolved questions are discussed and resulting solution approaches are substantiated (planning), implemented in the examination (implementation) and subsequently evaluated (reflection). The presented results are the product of this iterative procedure. Results: The FLE CS is created by experts from all faculties and subject areas in a multistage process. The examination is administered in German and French on a decentralised basis and consists of twelve interdisciplinary stations per candidate. As important quality assurance measures, the national Review Board (content validation) and the meetings of the standardised patient trainers (standardisation) have proven worthwhile. The statistical analyses show good measurement reliability and support the construct validity of the examination. Among the central insights of the past years, it has been established that the consistent implementation of the principles of action research contributes to the successful further development of the examination. Conclusion: The centrally coordinated, collaborative-iterative process, incorporating experts from all faculties, makes a fundamental contribution to the quality of the FLE CS. The processes and insights presented here can be useful for others planning a similar undertaking. Keywords: national final examination, licensing examination, summative assessment, OSCE, action research

Encouragement to Increase the Use of Psychosocial Skills in the Diagnosis and Therapy of Patients With Functional Dysphonia.

Clinicians believe that psychosocial factors play a causal role in the etiology of many forms of functional dysphonia (FD). But for decades, all attempts to confirm such causation have failed. This paper aims to show the logic of this failure, to discuss the possibilities of employing psychology in therapy nonetheless, and to encourage clinicians to use their psychosocial knowledge and skills. The failure to confirm psychic and social factors as causal in the etiology of FD is basically a consequence of a principal shortcoming of evidence-based medicine (EBM). As the gold standard for validity, reliability, and objectivity in medical research, EBM is based on calculability and hence the processing of quantitative data. But life paths and life situations are best or sometimes only expressible in qualitative, experiential, and idiographic terms. Thus EBM-guided evaluation undervalues most psychosocial studies. This report of an experienced multidisciplinary voice team proposes alternative pathways for integrating psychosocial knowledge into the diagnosis and the treatment of FD. The difference between the fields of activity of psychotherapists and speech-language pathologists is discussed, and the latter group is shown the potential benefits of using more of their psychosocial knowledge and skills.

Comparison of 2.5D and 3D Quantification of Femoral Head Coverage in Normal Control Subjects and Patients with Hip Dysplasia.

Hip dysplasia is characterized by insufficient femoral head coverage (FHC). Quantification of FHC is of importance as the underlying goal of the surgery to treat hip dysplasia is to restore a normal acetabular morphology and thereby to improve FHC. Unlike a pure 2D X-ray radiograph-based measurement method or a pure 3D CT-based measurement method, previously we presented a 2.5D method to quantify FHC from a single anteriorposterior (AP) pelvic radiograph. In this study, we first quantified and compared 3D FHC between a normal control group and a patient group using a CT-based measurement method. Taking the CT-based 3D measurements of FHC as the gold standard, we further quantified the bias, precision and correlation between the 2.5D measurements and the 3D measurements on both the control group and the patient group. Based on digitally reconstructed radiographs (DRRs), we investigated the influence of the pelvic tilt on the 2.5D measurements of FHC. The intraclass correlation coefficients (ICCs) for absolute agreement was used to quantify interobserver reliability and intraobserver reproducibility of the 2.5D measurement technique. The Pearson correlation coefficient, r, was used to determine the strength of the linear association between the 2.5D and the 3D measurements. Student's t-test was used to determine whether the differences between different measurements were statistically significant. Our experimental results demonstrated that both the interobserver reliability and the intraobserver reproducibility of the 2.5D measurement technique were very good (ICCs > 0.8). Regression analysis indicated that the correlation was very strong between the 2.5D and the 3D measurements (r = 0.89, p < 0.001). Student's t-test showed that there were no statistically significant differences between the 2.5D and the 3D measurements of FHC on the patient group (p > 0.05). The results of this study provided convincing evidence demonstrating the validity of the 2.5D measurements of FHC from a single AP pelvic radiograph and proved that it could serve as a surrogate for 3D CT-based measurements. Thus it may be possible to use this method to avoid a CT scan for the purpose of estimating 3D FHC in diagnosis and post-operative treatment evaluation of patients with hip dysplasia.

Pelvic floor muscle electromyography during different running speeds: an exploratory and reliability study.

PURPOSE Stress urinary incontinence (SUI) affects women of all ages including young athletes, especially those involved in high-impact sports. To date, hardly any studies are available testing pelvic floor muscles (PFM) during sports activities. The aim of this study was the description and reliability test of six PFM electromyography (EMG) variables during three different running speeds. The secondary objective was to evaluate whether there was a speed-dependent difference between the PFM activity variables. METHODS This trial was designed as an exploratory and reliability study including ten young healthy female subjects to characterize PFM pre-activity and reflex activity during running at 7, 9 and 11 km/h. Six variables for each running speed, averaged over ten steps per subject, were presented descriptively, tested regarding their reliability (Friedman, ICC, SEM, MD) and speed difference (Friedman). RESULTS PFM EMG variables varied between 67.6 and 106.1 %EMG, showed no systematic error and were low for SEM and MD using the single value model. Applying the average model over ten steps, ICC (3,k) were >0.75 and SEM and MD about 50 % lower than for the single value model. Activity was found to be highest in 11 km/h. CONCLUSION EMG variables showed excellent ICC and very low SEM and MD. Further studies should investigate inter-session reliability and PFM reactivity patterns of SUI patients using the average over ten steps for each variable as it showed very high ICC and very low SEM and MD. Subsequently, longer running distances and other high-impact sports disciplines could be studied.

THE APPLICATION OF CIRRHOSIS MORTALITY AND SUICIDE MORTALITY AS INDICATORS FOR ALCOHOLISM PLANNING

This study provides a review of the current alcoholism planning process of the Houston-Galveston planning process of the Houston-Galveston Area Council, an agency carrying out planning for a thirteen county region in surrounding Houston, Texas. The four central groups involved in this planning are identified, and the role that each plays and how it effects the planning outcomes is discussed.^ The most substantive outcome of the Houston-Galveston Area Council's alcoholism planning, the Regional Alcoholism/Alcohol Abuse Plan is examined. Many of the shortcomings in the data provided, and the lack of other data necessary for planning are offered.^ A problem oriented planning model is presented as an alternative to the Houston-Galveston Area Council's current service oriented approach to alcoholism planning. Five primary phases of the model, identification of the problem, statement of objectives, selection of alternative programs, implementation, and evaluation, are presented, and an overview of the tasks involved in the application of this model to alcoholism planning is offered.^ A specific aspect of the model, the use of problem status indicators is explored using cirrhosis and suicide mortality data. A review of the literature suggests that based on five criteria, availability, subgroup identification, validity, reliability, and sensitivity, both suicide and cirrhosis are suitable as indicators of the alcohol problem when combined with other indicators.^ Cirrhosis and suicide mortality data are examined for the thirteen county Houston-Galveston Region for the years 1969 through 1976. Data limitations preclude definite conclusions concerning the alcohol problem in the region. Three hypotheses about the nature of the regional alcohol problem are presented. First, there appears to be no linear trend in the number of alcoholics that are at risk of suicide and cirrhosis mortality. Second, the number of alcoholics in the metropolitan areas seems to be greater than the number of rural areas. Third, the number of male alcoholics at risk of cirrhosis and suicide mortality is greater than the number of female alcoholics.^