Improving IRT parameter estimates with small sample sizes: Evaluating the efficacy of a new data augmentation technique

The 3PL model is a flexible and widely used tool in assessment. However, it suffers from limitations due to its need for large sample sizes. This study introduces and evaluates the efficacy of a new sample size augmentation technique called Duplicate, Erase, and Replace (DupER) Augmentation through a simulation study. Data are augmented using several variations of DupER Augmentation (based on different imputation methodologies, deletion rates, and duplication rates), analyzed in BILOG-MG 3, and results are compared to those obtained from analyzing the raw data. Additional manipulated variables include test length and sample size. Estimates are compared using seven different evaluative criteria. Results are mixed and inconclusive. DupER augmented data tend to result in larger root mean squared errors (RMSEs) and lower correlations between estimates and parameters for both item and ability parameters. However, some DupER variations produce estimates that are much less biased than those obtained from the raw data alone. For one DupER variation, it was found that DupER produced better results for low-ability simulees and worse results for those with high abilities. Findings, limitations, and recommendations for future studies are discussed. Specific recommendations for future studies include the application of Duper Augmentation (1) to empirical data, (2) with additional IRT models, and (3) the analysis of the efficacy of the procedure for different item and ability parameter distributions.

Literacy Research: Issues of Authority, Ownership, and Representation

With the “social turn” of language in the past decade within English studies, ethnographic and teacher research methods increasingly have acquired legitimacy as a means of studying student literacy. And with this legitimacy, graduate students specializing in literacy and composition studies increasingly are being encouraged to use ethnographic and teacher research methods to study student literacy within classrooms. Yet few of the narratives produced from these studies discuss the problems that frequently arise when participant observers enter the classroom. Recently, some researchers have begun to interrogate the extent to which ethnographic and teacher research methods are able to construct and disseminate knowledge in empowering ways (Anderson & Irvine, 1993; Bishop, 1993; Fine, 1994; Fleischer. 1994; McLaren, 1992). While ethnographic and teacher research methods have oftentimes been touted as being more democratic and nonhierarchical than quantitative methods—-which oftentimes erase individuals lived experiences with numbers and statistical formulas—-researchers are just beginning to probe the ways that ethnographic and teacher research models can also be silencing, unreflective, and oppressive. Those who have begun to question the ethics of conducting, writing about, and disseminating knowledge in education have coined the term “critical” research, a rather vague and loose term that proposes a position of reflexivity and self-critique for all research methods, not just ethnography or teacher research. Drawing upon theories of feminist consciousness-raising, liberatory praxis, and community-action research, theories of critical research aim to involve researchers and participants in a highly participatory framework for constructing knowledge, an inquiry that seeks to question, disrupt, or intervene in the conditions under study for some socially transformative end. While critical research methods are always contingent upon the context being studied, in general they are undergirded by principles of non-hierarchical relations, participatory collaboration, problem-posing, dialogic inquiry, and multiple and multi-voiced interpretations. In distinguishing between critical and traditional ethnographic processes, for instance, Peter McLaren says that critical ethnography asks questions such as “[u]nder what conditions and to what ends do we. as educational researchers, enter into relations of cooperation. mutuality, and reciprocity with those who we research?” (p. 78) and “what social effects do you want your evaluations and understandings to have?” (p. 83). In»the same vein, Michelle Fine suggests that critical researchers must move beyond notions of the etic/emic dichotomy of researcher positionality in order to “probe how we are in relation with the contexts we study and with our informants, understanding that we are all multiple in those relations” (p. 72). Researchers in composition and literacy stud¬ies who endorse critical research methods, then, aim to enact some sort of positive transformative change in keeping with the needs and interests of the participants with whom they work.