The Problem with Value-Added Measurement

Measuring student learning through standardized tests is a lot harder than modern education reformers would have you believe.

Commentary on "Alternative Strategies for Identifying High-Performing Charter Schools in Texas"

In the last few years policy makers and practitioners nationally have shown much interest in identifying, recognizing, and replicating successful charter schools, many of which are showing that they can educate low-income and otherwise at-risk students remarkably well. However past efforts to identify high performing schools have been problematic. Using these systematic, rigorous value-added methods, the authors identify 44 Open Enrollment charter schools that merit a “high-performer” rating. Nearly all of those campuses identified serve a disadvantaged student population. The article also finds that most of those high performers are highly cost-effective, earning high ratings on the cost-efficiency measures. The authors argue for more widespread use of value-added modeling in the state accountability system. The approach taken to identifying high-performers is sensible and fair, but any formulaic approach to school labels comes with some limitations.

Alternative Strategies for Identifying High-Performing Charter Schools in Texas

The Obama administration's recurring policy emphasis on high-performing charter schools begs the obvious question: how do you identify a high-performing charter school? That is a crucially important policy question because any evaluation strategy that incorrectly identifies charter school performance could have negative effects on the economically and/or academically disadvantaged students who frequently attend charter schools. If low-performing schools are mislabeled and allowed to persist or encouraged to expand, then students may be harmed directly. If high-performing schools are driven from the market by misinformation, then students will lose access to programs and services that can make a difference in their lives. Most of the scholarly analysis to date has focused on comparing the performance of students in charter schools to that of similar students in traditional public schools (TPS). By design, that research measures charter school performance only in relative terms. Charter schools that outperform similarly situated, but low performing, TPSs have positive effects, even if the charter schools are mediocre in an absolute sense. This analysis describes strategies for identifying high-performing charter schools by comparing charter schools with one another. We begin by describing salient characteristics of Texas charter schools. We follow that discussion with a look at how other researchers across the country have compared charter school effectiveness with TPS effectiveness. We then present several metrics that can be used to identify high-performing charter schools. Those metrics are not mutually exclusive—one could easily justify using multiple measures to evaluate school effectiveness—but they are also not equally informative. If the goal is to measure the contributions that schools are making to student knowledge and skills, then a value-added approach like the ones highlighted in this report is clearly superior to a levels-based approach like that taken under the current accountability system.