Sample size requirements for interval estimation of the strengthof association effect sizes in multiple regression analysis

Resumen tomado de la publicación

Meta-analysis to integrate effect sizes within a paper: Possible misuse and Type-1 error inflation

In recent years an increasing number of papers have employed meta-analysis to integrate effect sizes of researchers’ own series of studies within a single paper (“internal meta-analysis”). Although this approach has the obvious advantage of obtaining narrower confidence intervals, we show that it could inadvertently inflate false-positive rates if researchers are motivated to use internal meta-analysis in order to obtain a significant overall effect. Specifically, if one decides whether to stop or continue a further replication experiment depending on the significance of the results in an internal meta-analysis, false-positive rates would increase beyond the nominal level. We conducted a set of Monte-Carlo simulations to demonstrate our argument, and provided a literature review to gauge awareness and prevalence of this issue. Furthermore, we made several recommendations when using internal meta-analysis to make a judgment on statistical significance.

Effect sizes and standardization in neighbourhood models of forest stands: potential biases and misinterpretations

Effects of conspecific neighbours on survival and growth of trees have been found to be related to species abundance. Both positive and negative relationships may explain observed abundance patterns. Surprisingly, it is rarely tested whether such relationships could be biased or even spurious due to transforming neighbourhood variables or influences of spatial aggregation, distance decay of neighbour effects and standardization of effect sizes. To investigate potential biases, communities of 20 identical species were simulated with log-series abundances but without species-specific interactions. No relationship of conspecific neighbour effects on survival or growth with species abundance was expected. Survival and growth of individuals was simulated in random and aggregated spatial patterns using no, linear, or squared distance decay of neighbour effects. Regression coefficients of statistical neighbourhood models were unbiased and unrelated to species abundance. However, variation in the number of conspecific neighbours was positively or negatively related to species abundance depending on transformations of neighbourhood variables, spatial pattern and distance decay. Consequently, effect sizes and standardized regression coefficients, often used in model fitting across large numbers of species, were also positively or negatively related to species abundance depending on transformation of neighbourhood variables, spatial pattern and distance decay. Tests using randomized tree positions and identities provide the best benchmarks by which to critically evaluate relationships of effect sizes or standardized regression coefficients with tree species abundance. This will better guard against potential misinterpretations.

Robust effect sizes and their confidence intervals for group difference between trajectories in hierarchical linear growth model

Hierarchical linear growth model (HLGM), as a flexible and powerful analytic method, has played an increased important role in psychology, public health and medical sciences in recent decades. Mostly, researchers who conduct HLGM are interested in the treatment effect on individual trajectories, which can be indicated by the cross-level interaction effects. However, the statistical hypothesis test for the effect of cross-level interaction in HLGM only show us whether there is a significant group difference in the average rate of change, rate of acceleration or higher polynomial effect; it fails to convey information about the magnitude of the difference between the group trajectories at specific time point. Thus, reporting and interpreting effect sizes have been increased emphases in HLGM in recent years, due to the limitations and increased criticisms for statistical hypothesis testing. However, most researchers fail to report these model-implied effect sizes for group trajectories comparison and their corresponding confidence intervals in HLGM analysis, since lack of appropriate and standard functions to estimate effect sizes associated with the model-implied difference between grouping trajectories in HLGM, and also lack of computing packages in the popular statistical software to automatically calculate them. ^ The present project is the first to establish the appropriate computing functions to assess the standard difference between grouping trajectories in HLGM. We proposed the two functions to estimate effect sizes on model-based grouping trajectories difference at specific time, we also suggested the robust effect sizes to reduce the bias of estimated effect sizes. Then, we applied the proposed functions to estimate the population effect sizes (d ) and robust effect sizes (du) on the cross-level interaction in HLGM by using the three simulated datasets, and also we compared the three methods of constructing confidence intervals around d and du recommended the best one for application. At the end, we constructed 95% confidence intervals with the suitable method for the effect sizes what we obtained with the three simulated datasets. ^ The effect sizes between grouping trajectories for the three simulated longitudinal datasets indicated that even though the statistical hypothesis test shows no significant difference between grouping trajectories, effect sizes between these grouping trajectories can still be large at some time points. Therefore, effect sizes between grouping trajectories in HLGM analysis provide us additional and meaningful information to assess group effect on individual trajectories. In addition, we also compared the three methods to construct 95% confident intervals around corresponding effect sizes in this project, which handled with the uncertainty of effect sizes to population parameter. We suggested the noncentral t-distribution based method when the assumptions held, and the bootstrap bias-corrected and accelerated method when the assumptions are not met.^

Comprehensive treatments for social cognitive deficits in schizophrenia: A critical review and effect-size analysis of controlled studies

Recent advances in psychosocial treatments for schizophrenia have targeted social cognitive deficits. A critical literature review and effect-size (ES) analysis was conducted to investigate the efficacy of comprehensive programs of social cognitive training in schizophrenia. Results revealed 16 controlled studies consisting of seven models of comprehensive treatment with only three of these treatment models investigated in more than one study. The effects of social cognitive training were reported in 11/15 studies that included facial affect recognition skills (ES=.84) and 10/13 studies that included theory-of-mind (ES=.70) as outcomes. Less than half (4/9) of studies that measured attributional style as an outcome reported effects of treatment, but effect sizes across studies were significant (ESs=.30-.52). The effect sizes for symptoms were modest, but, with the exception of positive symptoms, significant (ESs=.32-.40). The majority of trials were randomized (13/16), selected active control conditions (11/16) and included at least 30 participants (12/16). Concerns for this area of research include the absence of blinded outcome raters in more than 50% of trials and low rates of utilization of procedures for maintaining treatment fidelity. These findings provide preliminary support for the broader use of comprehensive social cognitive training procedures as a psychosocial intervention for schizophrenia.

Experimental manipulation of colony genetic diversity had no effect on short-term task efficiency in the Argentine ant Linepithema humile

Genetic diversity might increase the performance of social groups by improving task efficiency or disease resistance, but direct experimental tests of these hypotheses are rare. We manipulated the level of genetic diversity in colonies of the Argentine ant Linepithema humile, and then recorded the short-term task efficiency of these experimental colonies. The efficiency of low and high genetic diversity colonies did not differ significantly for any of the following tasks: exploring a new territory, foraging, moving to a new nest site, or removing corpses. The tests were powerful enough to detect large effects, but may have failed to detect small differences. Indeed, observed effect sizes were generally small, except for the time to create a trail during nest emigration. In addition, genetic diversity had no statistically significant impact on the number of workers, males and females produced by the colony, but these tests had low power. Higher genetic diversity also did not result in lower variance in task efficiency and productivity. In contrast to genetic diversity, colony size was positively correlated with the efficiency at performing most tasks and with colony productivity. Altogether, these results suggest that genetic diversity does not strongly improve short-term task efficiency in L. humile, but that worker number is a key factor determining the success of this invasive species.

Effect of a lifestyle intervention on adiposity and fitness in socially disadvantaged subgroups of preschoolers: a cluster-randomized trial (Ballabeina).

OBJECTIVE: A multidimensional lifestyle intervention performed in 652 preschoolers (72% of migrant, 38% of low educational level (EL) parents) reduced body fat, but not BMI and improved fitness. The objective of this study is to examine whether the intervention was equally effective in children of migrant and/or low EL parents.¦METHODS: Cluster-randomized controlled single blinded trial, conducted in 2008/09 in 40 randomly selected preschools in Switzerland. The culturally tailored intervention consisted of a physical activity program and lessons on nutrition, media use and sleep. Primary outcomes included BMI and aerobic fitness. Secondary outcomes included %body fat, waist circumference and motor agility.¦RESULTS: Children of migrant parents benefitted similarly from the intervention compared to their counterparts (p for interaction≥ 0.09). However, children of low EL parents benefitted less, although these differences did not reach statistical significance (p for interaction≥ 0.06). Average intervention effect sizes for BMI were -0.10, -0.05, -0.11 and 0.04 kg/m(2) and for aerobic fitness were 0.55, 0.20, 0.37 and -0.05 stages for children of non-migrant, migrant, middle/high EL and low EL parents, respectively.¦CONCLUSIONS: This intervention was similarly effective among preschoolers of migrant parents compared to their counterparts, while children of low EL parents benefitted less.

The Effect of a Stickhandling and Puck Control (SPC) Training Intervention on SPC Skills and Wrist Shot Performance Variables in Female Collegiate Ice Hockey Players

The purpose of the study was to investigate the effect of a 16 session stickhandling and puck control (SPC) off-ice training intervention on SPC skills and wrist shot performance variables. Eighteen female collegiate ice hockey players participated in a crossover design training intervention, whereby players were randomly assigned to two groups. Each group completed 16 SPC training sessions in two conditions [normal vision (NV) and restricted vision (RV)]. Measures obtained after the training intervention revealed significant improvements in SPC skills and wrist shot accuracy. Order of training condition did not reach significance, meaning that SPC improvement occurred as a result of total training volume as opposed to order of training condition. However, overall changes in the RV-NV condition revealed consistently higher effect sizes, meaning a greater improvement in performance. Therefore, support can be provided for this technical approach to SPC training and an alternative method of challenging SPC skills.

Is the allegiance effect an epiphenomenon of true efficacy differences between treatments? a meta-analysis

Many meta-analyses of comparative outcome studies found a substantial association of researcher allegiance (RA) and relative treatment effects. Therefore, RA is regarded as a biasing factor in comparative outcome research (RA bias hypothesis). However, the RA bias hypothesis has been criticized as causality might be reversed. That is, RA might be a reflection of true efficacy differences between treatments (true efficacy hypothesis). Consequently, the RA-outcome association would not be indicative of bias but an epiphenomenon of true efficacy differences. This meta-analysis tested the validity of the true efficacy hypothesis. This was done by controlling the RA-outcome association for true efficacy differences by restricting analysis to direct comparisons of treatments with equivalent efficacy. We included direct comparisons of different versions of trauma-focused therapy (TFT) in the treatment of posttraumatic stress disorder (PTSD). RA was measured from the research reports. Relative effect sizes for symptoms of PTSD were calculated. Random effects meta-regression was conducted. Twenty-nine comparisons of TFTs from 20 studies were identified. Initial heterogeneity among relative effect sizes was low. RA was a significant predictor of outcome and explained 12% of the variance in outcomes. The true efficacy hypothesis predicted the RA-outcome association to be zero; however, a substantial association was found. Thus, this study does not support the true efficacy hypothesis. Given findings from psychotherapy research and other fields that support a biasing influence of researcher preferences, RA should be regarded as a causal factor and conceptualized as a threat to the validity of conclusions from comparative outcome studies.

Effect of therapeutic exercise for hip osteoarthritis pain: results of a meta-analysis

OBJECTIVE: Recommendations for lower extremity osteoarthritis (OA) and exercise have been primarily based on knee studies. To provide more targeted recommendations for the hip, we gathered evidence for the efficacy of exercise for hip OA from randomized controlled trials. METHODS: A bibliographic search identified trials that were randomized, controlled, completed by >or=60% of subjects, and involved an exercise group (strengthening and/or aerobic) versus a non exercise control group for pain relief in hip OA. Two reviewers independently performed the data extraction and contacted the authors when necessary. Effect sizes (ES) of treatment versus control and the I(2) statistic to assess heterogeneity across trials were calculated. Trial data were combined using a random-effects meta-analysis. RESULTS: Nine trials met the inclusion criteria (1,234 subjects), 7 of which combined hip and knee OA; therefore, we contacted the authors who provided the data on hip OA patients. In comparing exercise treatment versus control, we found a beneficial effect of exercise with an ES of -0.38 (95% confidence interval [95% CI] -0.68, -0.08; P = 0.01), but with high heterogeneity (I(2) = 75%) among trials. Heterogeneity was caused by 1 trial consisting of an exercise intervention that was not administered in person. Removing this study left 8 trials (n = 493) with similar exercise strategy (specialized hands-on exercise training, all of which included at least some element of muscle strengthening), and demonstrated exercise benefit with an ES of -0.46 (95% CI -0.64, -0.28; P < 0.0001). CONCLUSION: Therapeutic exercise, especially with an element of strengthening, is an efficacious treatment for hip OA.

Effect of psychological interventions on depressive symptoms in long-term rehabilitation after an acquired brain injury: a systematic review and meta-analysis

OBJECTIVE To summarize empirical studies on the effectiveness of psychological interventions in long-term rehabilitation after an acquired brain injury (ABI) in reducing depressive symptoms. DATA SOURCES A systematic literature search was conducted on MEDLINE, PsycINFO, Embase, and CINAHL to identify articles published between January 1990 and October 2011. Search terms included the 3 concepts (1) "brain injur*" or "stroke," (2) "psychotherap*" or "therapy" or "intervention" or "rehabilitation," and (3) "depress*." STUDY SELECTION Studies evaluating psychological interventions in patients after ABI were included. Time since injury was on average more than 1 year. Trials reported data on validated depression questionnaires before and after the psychological intervention. DATA EXTRACTION Two independent reviewers extracted information from the sample, the intervention, and the outcome of the included studies and calculated effect sizes (ESs) from depression questionnaires. Thirteen studies were included in a pre-post analysis. Seven studies were eligible for a meta-analysis of ESs in active interventions and control conditions. DATA SYNTHESIS Pre-post ESs were significant in 4 of 13 studies. The overall ES of .69 (95% confidence interval [CI], .29-1.09) suggests a medium effectiveness of psychological interventions on depressive symptoms compared with control conditions. Moderator analysis of the number of sessions and adequate randomization procedure did not show significant ES differences between strata. Studies with adequate randomization did not, however, suggest the effectiveness of psychological interventions on depressive symptoms after ABI. CONCLUSIONS Psychological interventions are a promising treatment option for depressive symptoms in long-term rehabilitation after ABI. Since only a few adequately randomized controlled trials (RCTs) exist, more RCTs are required to confirm this initial finding.

Effect of Cognitive Impairment on Driving-Relevant Cognition in Older Persons

Intact cognitive abilities are fundamental for driving. Driving-relevant cognition may be affected in older drivers due to aging or cognitive impairment. The aim of this study was to investigate the effects of cognitive impairment on driving-relevant cognition in older persons. Performance in selective and divided attention, eye-hand-coordination, executive functions and the ability to regulate distance and speed of 18 older persons with CI-Group (cognitive impairment group) was compared to performance of older control group (18 age and gender-matched cognitively normal subjects) and young control group (18 gender-matched young subjects). The CI-Group showed poorer performance than the other two control groups in all cognitive tasks (significance level (p) < 0.001, effect size (partial η2) = 0.63). Differences between cognitively impaired and cognitively normal subjects were still significant after controlling for age (effect sizes from 0.14 to 0.28). Dual tasking affected performance of cognitively impaired subjects more than performance of the other two groups (p = 0.016, partial η2 = 0.14). Results show that cognitive impairment has age-independent detrimental effects on selective and divided attention, eye-hand-coordination, executive functions and the ability to regulate distance and speed. Largest effect sizes are found for reaction times in attention tasks.