All that educational progress: Part 2?

I’ve been assembling more comparison maps with the powerful NAEP Data Explorer tool from the National Assessment of Educational Progress.

Here is one I put together to show how Kentucky’s white students’ latest available NAEP writing performance compares to white students in other states that participated (five states shown in blue plus Washington, DC, didn’t participate in this voluntary assessment). I look at whites because they comprise 85 percent of the student population in Kentucky and because with demographics in other states now very different from Kentucky’s, simplistic comparisons of overall NAEP scores are misleading.


According to the NAEP Data Explorer, after all the time we spent on writing in Kentucky, as of 2007, white students in the 31 states shown in green, including a number in the South, were doing a better job than our white students. NAEP says with a 95 percent level of confidence that all states shown in green got writing scores that were statistically significantly higher than Kentucky’s on this sampled assessment.

Twelve states tied us, including Mississippi.

Only West Virginia did worse.

Now, how is that again about all the progress?

All that educational progress????

Once again, the Prichard Committee is talking about all the educational progress we have made in Kentucky at the same time they admit we have a long way to go.

I most definitely agree with the part about having a long way to go, but I just don’t see that what has happened so far is significant.

Take a look at this graph, which shows the latest available proficiency rate scores (percentage scoring at or above Proficient) from the National Assessment of Educational Progress (NAEP). The full bars show the most recently reported proficiency rates by subject and grade. Inside of each bar I show the earliest available proficiency rate data for that same subject and grade (click on the graph to enlarge it, if necessary).


Overall, notice that after 19 years of KERA (17 for writing, which was last given in 2007) we generally only find about one in three students are proficient in Kentucky in the listed subjects. For eighth grade math and writing, the proficiency rate is little more than just one in four.

Now, consider where we started on grade 4 reading and math and grade 8 math in the early days of KERA (other subjects didn’t start to test until 1996 or later).

In grade 4 reading, between 1992 and 2009, a 17-year period, our proficiency rate only increased by 13 percentage points. That works out to an average improvement rate of only 0.765 point per year. At that rate, it will take us more than 70 years to reach a proficiency rate of 90 percent! Clearly, we’ve barely started on the journey towards high academic performance, and we are a long way from attaining that goal in fourth grade reading.

I have run more estimates to reach 90 percent proficiency for other subjects in the graph, and you can click the “Read more” link to see that.

Bottom line, anyone who thinks the proficiency rates in the graph above signal significant progress doesn’t have very high standards. And, it is for certain that if we don’t do something dramatic (like establishing charter schools) to stimulate the sluggardly pace of improvement in our schools, the competition is going to bury our kids.

Read more »

Where’s All the Writing Improvement?

Kentucky Tonight just held a discussion of the pending CATS Task Force that will be looking at the state’s public school assessment program. It was three educators against our friend Martin Cothran from the Family Foundation, so the educators were outnumbered.

One item the educators brought up repeatedly was how much improvement they have seen in writing. That’s just more evidence our educators see too much in too little, so I called in to the show to share the latest results on eighth grade writing from the National Assessment of Educational Progress (NAEP). I thought you might like to know about that, too.

The NAEP 2007 Writing Report Card says Kentucky’s eighth graders made no statistically significant change in writing performance between 2002 and 2007. That’s half a decade of flat performance.

The 2007 NAEP proficiency rate for our eighth graders was only 26 percent – just one of four Kentucky kids scored proficient for writing. That was a statistically insignificant change from the 25 percent proficiency reported in 2002. In fact, since 1998, a decade ago, the NAEP eighth grade writing proficiency rate in Kentucky has only gone up 5 points – but there is a catch.

While posting its miniscule score improvement between 1998 and 2007, Kentucky’s exclusion rate for students with learning disabilities shot up faster than anywhere else in the country, rising from two to six percent of the entire NAEP raw sample of students. This big increase probably means most, if not all, of the tiny writing score rise since 1998 is an illusion. Scores look a bit better if you simply prevent a lot more of your weakest kids from taking the test.

By the way, over that same period, the CATS said our middle school kids “on-demand” writing sample, which was collected in a manner similar to the sort of testing conditions the NAEP uses, exploded from just 5.92 percent proficient to 42 percent proficient. So, CATS went from being graded far too hard to being graded much easier than the NAEP. That’s just perfect if you are an education person trying to make yourself look good.

It’s terrible if you are a member of the public searching for the truth about how schools are really doing.

And, all of this shows how CATS just confuses educators and the public alike.