Use of quality assessment tools in systematic reviews was varied and inconsistent

OBJECTIVES To assess the use of quality assessment tools among a cross-section of systematic reviews (SRs) and to further evaluate whether quality was used as a parameter in the decision to include primary studies within subsequent meta-analysis. STUDY DESIGN AND SETTING We searched PubMed for SRs (interventional, observational, and diagnostic) published in Core Clinical Journals between January 1 and March 31, 2014. RESULTS Three hundred nine SRs were identified. Quality assessment was undertaken in 222 (71.8%) with isolated use of the Cochrane risk of bias tool (26.1%, n = 58) and the Newcastle-Ottawa Scale (15.3%, n = 34) most common. A threshold level of primary study quality for subsequent meta-analysis was used in 12.9% (40 of 309) of reviews. Overall, fifty-four combinations of quality assessment tools were identified with a similar preponderance of tools used among observational and interventional reviews. Multiple tools were used in 11.7% (n = 36) of SRs overall. CONCLUSION We found that quality assessment tools were used in a majority of SRs; however, a threshold level of quality for meta-analysis was stipulated in just 12.9% (n = 40). This cross-sectional analysis provides further evidence of the need for more active or intuitive editorial processes to enhance the reporting of SRs.

Reporting and handling missing outcome data in mental health: a systematic review of Cochrane systematic reviews and meta-analyses

OBJECTIVES The purpose of the study was to provide empirical evidence about the reporting of methodology to address missing outcome data and the acknowledgement of their impact in Cochrane systematic reviews in the mental health field. METHODS Systematic reviews published in the Cochrane Database of Systematic Reviews after January 1, 2009 by three Cochrane Review Groups relating to mental health were included. RESULTS One hundred ninety systematic reviews were considered. Missing outcome data were present in at least one included study in 175 systematic reviews. Of these 175 systematic reviews, 147 (84%) accounted for missing outcome data by considering a relevant primary or secondary outcome (e.g., dropout). Missing outcome data implications were reported only in 61 (35%) systematic reviews and primarily in the discussion section by commenting on the amount of the missing outcome data. One hundred forty eligible meta-analyses with missing data were scrutinized. Seventy-nine (56%) of them had studies with total dropout rate between 10 and 30%. One hundred nine (78%) meta-analyses reported to have performed intention-to-treat analysis by including trials with imputed outcome data. Sensitivity analysis for incomplete outcome data was implemented in less than 20% of the meta-analyses. CONCLUSIONS Reporting of the techniques for handling missing outcome data and their implications in the findings of the systematic reviews are suboptimal.

The evidence from systematic reviews and meta-analyses published in orthodontic literature. Where do we stand?

AIM To analyse meta-analyses included in systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) focusing on orthodontic literature and to assess the quality of the existing evidence. MATERIALS AND METHODS Electronic searching was undertaken to identify SRs published in five major orthodontic journals and the CDSR between January 2000 and June 2014. Quality assessment of the overall body of evidence from meta-analyses was conducted using the Grading of Recommendations Assessment, Development and Evaluation working group (GRADE) tool. RESULTS One hundred and fifty-seven SRs were identified; meta-analysis was present in 43 of these (27.4 per cent). The highest proportion of SRs that included a meta-analysis was found in Orthodontics and Craniofacial Research (6/13; 46.1 per cent), followed by the CDSR (12/33; 36.4 per cent) and the American Journal of Orthodontics and Dentofacial Orthopaedics (15/44; 34.1 per cent). Class II treatment was the most commonly addressed topic within SRs in orthodontics (n = 18/157; 11.5 per cent). The number of trials combined to produce a summary estimate was small for most meta-analyses with a median of 4 (range: 2-52). Only 21 per cent (n = 9) of included meta-analyses were considered to have a high/moderate quality of evidence according to GRADE, while the majority were of low or very low quality (n = 34; 79.0 per cent). CONCLUSIONS Overall, approximately one quarter of orthodontic SRs included quantitative synthesis, with a median of four trials per meta-analysis. The overall quality of evidence from the selected orthodontic SRs was predominantly low to very low indicating the relative lack of high quality of evidence from SRs to inform clinical practice guidelines.

Quantitative Determination of the Relationship between Internal Validity and Bias in Software Engineering Experiments: Consequences for Systematic Literature Reviews

Quality assessment is one of the activities performed as part of systematic literature reviews. It is commonly accepted that a good quality experiment is bias free. Bias is considered to be related to internal validity (e.g., how adequately the experiment is planned, executed and analysed). Quality assessment is usually conducted using checklists and quality scales. It has not yet been proven;however, that quality is related to experimental bias. Aim: Identify whether there is a relationship between internal validity and bias in software engineering experiments. Method: We built a quality scale to determine the quality of the studies, which we applied to 28 experiments included in two systematic literature reviews. We proposed an objective indicator of experimental bias, which we applied to the same 28 experiments. Finally, we analysed the correlations between the quality scores and the proposed measure of bias. Results: We failed to find a relationship between the global quality score (resulting from the quality scale) and bias; however, we did identify interesting correlations between bias and some particular aspects of internal validity measured by the instrument. Conclusions: There is an empirically provable relationship between internal validity and bias. It is feasible to apply quality assessment in systematic literature reviews, subject to limits on the internal validity aspects for consideration.

Identification of Degraded Land in the Canary Islands; Tests and Reviews

Degraded Land is an area that either by natural causes (fires, floods, storms or volcanic eruptions) or more by direct or indirect causes of human action, has been altered or modified from its natural state. Restoration is an activity that initiates or accelerates the recovery of an ecosystem. It can be defined as the set of actions taken in order to reverse or reduce the damage caused in the territory. In the case of the Canary Islands there is a high possibility for the territory to suffer processes that degrade the environment, given that the islands are very fragile ecosystems. Added to this they are territories isolated from the continent, which complicates the process of restoring them. In this paper, the different types of common degraded areas in the Canary Islands are identified, as well as the proposed solutions for remediation, such as afforestation of agricultural land or landfill closure and restoration.

Mining mobile apps reviews to support release planning

The mobile apps market is a tremendous success, with millions of apps downloaded and used every day by users spread all around the world. For apps’ developers, having their apps published on one of the major app stores (e.g. Google Play market) is just the beginning of the apps lifecycle. Indeed, in order to successfully compete with the other apps in the market, an app has to be updated frequently by adding new attractive features and by fixing existing bugs. Clearly, any developer interested in increasing the success of her app should try to implement features desired by the app’s users and to fix bugs affecting the user experience of many of them. A precious source of information to decide how to collect users’ opinions and wishes is represented by the reviews left by users on the store from which they downloaded the app. However, to exploit such information the app’s developer should manually read each user review and verify if it contains useful information (e.g. suggestions for new features). This is something not doable if the app receives hundreds of reviews per day, as happens for the very popular apps on the market. In this work, our aim is to provide support to mobile apps developers by proposing a novel approach exploiting data mining, natural language processing, machine learning, and clustering techniques in order to classify the user reviews on the basis of the information they contain (e.g. useless, suggestion for new features, bugs reporting). Such an approach has been empirically evaluated and made available in a web-­‐based tool publicly available to all apps’ developers. The achieved results showed that the developed tool: (i) is able to correctly categorise user reviews on the basis of their content (e.g. isolating those reporting bugs) with 78% of accuracy, (ii) produces clusters of reviews (e.g. groups together reviews indicating exactly the same bug to be fixed) that are meaningful from a developer’s point-­‐of-­‐view, and (iii) is considered useful by a software company working in the mobile apps’ development market.

Randomised trial of educational visits to enhance use of systematic reviews in 25 obstetric units

Objective To evaluate the effectiveness of an educational visit to help obstetricians and midwives select and use evidence from a Cochrane database containing 600 systematic reviews.

Clarifying the abstracts of systematic literature reviews*

Background: There is a small body of research on improving the clarity of abstracts in general that is relevant to improving the clarity of abstracts of systematic reviews.

Asymmetric effects of online consumer reviews

Consumers tend to seek heuristic information cues to simplify the amount of information involved in tourist decisions. Accordingly, star ratings in online reviews are a critical heuristic element of the perceived evaluation of online consumer information. The objective of this article is to assess the effect of review ratings on usefulness and enjoyment. The empirical application is carried out on a sample of 5,090 reviews of 45 restaurants in London and New York. The results show that people perceive extreme ratings (positive or negative) as more useful and enjoyable than moderate ratings, giving rise to a U-shaped line, with asymmetric effects: the size of the effect of online reviews depends on whether they are positive or negative.