School choice in other places: Boston

There has been a lot of discussion about whether charter schools do, or do not, work better than public schools.

Part of the confusion has been a result of the difficulty in finding a fair comparison group of students in the traditional public school system to compare to students in the charter schools.

All sorts of methods have been concocted to generate suitable public school comparison groups to lay beside charter school data. So far, the best method we have seen takes advantage of the fact that in some areas charter schools are so desired that they have to conduct random lotteries to select students. Those students who don’t win the lottery wind up in the traditional system, and they are indeed a randomly selected sample that can be compared to the students who do get accepted to the charter school.

One of these ‘lottery studies’ was released by The Boston Foundation in 2009 and is titled: “Informing the Debate, Comparing Boston’s Charter, Pilot and Traditional Schools.”

The findings from The Boston Foundation’s team of Duke, Harvard and MIT researchers are unequivocal. Both charter middle school and high school students performed notably better than their traditional public school lottery losing counterparts performed. For middle school math, in particular (a weak area in Kentucky), the report says:

“The estimated impact on math achievement for Charter middle schools is extraordinarily large. Increasing performance by .5 standard deviations is the same as moving from the 50th to the 69th percentile in student performance. This is roughly half the size of the black-white achievement gap.”

This graph, taken from the report, shows the dramatic increased performance of charter middle school students in Boston over time. Note that initially, there isn’t much difference between charter school students and traditional public school students who lost the lottery. But, as charter school students spend more time in those charters, their relative performance in both math and English Language Arts (ELA) starts to really stand out, rising well above the performance of the lottery losers who were trapped in traditional public schools in Bean Town.


This brings up another important point about charter school research. If that research largely looks at students who have only spent limited time in a charter school, it isn’t going to very fairly show how charter schools really perform.

It simply takes time for charter schools to work. It is unfair to expect them to create miracles in only one or even two years.

This issue of time in charters helps explain why some reports don’t show much of a charter school impact. Such findings often are due to the report taking an inadequately long look at what is happening. More on that tomorrow.