Quantitative Determination of the Relationship between Internal Validity and Bias in Software Engineering Experiments: Consequences for Systematic Literature Reviews


Autoria(s): Dieste Tubio, Oscar; Juristo Juzgado, Natalia; Grimán, Anna; Saxena, Himanshu
Data(s)

2011

Resumo

Quality assessment is one of the activities performed as part of systematic literature reviews. It is commonly accepted that a good quality experiment is bias free. Bias is considered to be related to internal validity (e.g., how adequately the experiment is planned, executed and analysed). Quality assessment is usually conducted using checklists and quality scales. It has not yet been proven;however, that quality is related to experimental bias. Aim: Identify whether there is a relationship between internal validity and bias in software engineering experiments. Method: We built a quality scale to determine the quality of the studies, which we applied to 28 experiments included in two systematic literature reviews. We proposed an objective indicator of experimental bias, which we applied to the same 28 experiments. Finally, we analysed the correlations between the quality scores and the proposed measure of bias. Results: We failed to find a relationship between the global quality score (resulting from the quality scale) and bias; however, we did identify interesting correlations between bias and some particular aspects of internal validity measured by the instrument. Conclusions: There is an empirically provable relationship between internal validity and bias. It is feasible to apply quality assessment in systematic literature reviews, subject to limits on the internal validity aspects for consideration.

Formato

application/pdf

Identificador

http://oa.upm.es/11571/

Idioma(s)

eng

Publicador

Facultad de Informática (UPM)

Relação

http://oa.upm.es/11571/1/INVE_MEM_2011_102670.pdf

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6092577

Direitos

http://creativecommons.org/licenses/by-nc-nd/3.0/es/

info:eu-repo/semantics/openAccess

Fonte

Proceedings of the Fifth International Symposium on Empirical Software Engineering and Measurement, ESEM 2011 | Fifth International Symposium on Empirical Software Engineering and Measurement, ESEM 2011 | 19/09/2011 - 23/09/2011 | Banff, Albert, Canada

Palavras-Chave #Telecomunicaciones #Informática
Tipo

info:eu-repo/semantics/conferenceObject

Ponencia en Congreso o Jornada

PeerReviewed