NAEP prophesy proves true really fast

When I posted my first comments about the new math scores from the National Assessment of Educational Progress (NAEP) earlier today, I included the following caution: “So, watch out for those who try to make too much out of the fourth grade improvement.”

I knew that warning was needed, but I didn’t realize how quickly that would become true.

Let’s start with some “right stuff.” You cannot get a good idea about what is happening in education if you only look at the overall NAEP scores for all students. So, this first graph shows how our white kids compared to the rest of the nation since NAEP began fourth grade math testing back in 1992.


While our white kids have made progress, so have their peers in the rest of the nation. As a result, the Kentucky to national NAEP fourth grade math white score gap has hardly budged, moving from 10 points in 1992 to seven points today. With a rate of three points of closure in 17 years, we will need something like 40 more years to catch the rest of the country.

Somehow, considering that our NAEP proficiency rate is still very low for whites at only 39 percent, well behind a still-not-good-enough national math average of 50 percent, I don’t much feel ready to cheer.

Things look a lot worse when we consider our black student’s performance relative to blacks around the nation.



This second graph shows that story.


That’s right. Back in the early days of KERA our fourth grade blacks, while scoring very low on the NAEP math assessment, did do notably better than blacks in other states.

Well, that has changed. Our blacks lost their advantage and now score two points below the national black average.

So, who wants to cheer about this? Well, over at the Prichard Blog, they jumped right in, calling for “a mighty celebration.” Prichard supported their claim with a graph much like this one (except theirs left off the early years when test accommodations were not allowed on the NAEP).


This last graph conveys a very different picture, doesn’t it?

Of course, what is hidden in this all student data is the unfortunate fact that I talked about in my earlier post – Kentucky gets a huge advantage whenever overall NAEP scores for all students are compared because we simply don’t have many low scoring minority kids to drag our average down.

As I mentioned in the earlier post, the new NAEP report card shows that in the 1992 NAEP fourth grade math test 90 percent of the Kentucky students were white. In the new 2009 results that percentage has declined only seven points to 83 percent. However, across the nation the white percentages tested in those two years plummeted from 72 percent to only 54 percent.

By the way, Prichard left those early NAEP testing years off their graph. Here’s something you should know. Back in 1992 the NAEP didn’t allow students with learning disabilities to get any testing accommodations (which will raise their scores). If they took the test, they took it “cold,” just like all the other students. That changed in 2009 when NAEP did allow learning disabled students to take the NAEP with accommodations.

So, you might think that when accommodations were not allowed back in 1992 that a lot of our learning disabled kids got excluded from the NAEP.

Guess again!

Exclusion in 1992 equaled just three percent of the raw sample the NAEP wanted to test. In 2009, the exclusion rate was also three percent. However, in 2009 another 7 percent of the entire raw sample in Kentucky got testing accommodations that were not allowed at all back in 1992.

(Source: Tables A-4 and A-5 in the 2009 NAEP Report Card)

So, some of our grade 4 math score increase in 2009 is just due to kids getting score-inflating assistance that a notable portion of our kids didn’t get back in 1992.

One other point from Table A-5 in the new report. Back in 1992 the national average exclusion rate for learning disabled students was five percent, two points higher than Kentucky’s. In the new test, the national average exclusion rate dropped to only two percent, one point below us. So, over time, the national scores have been somewhat further depressed because more kids with disabilities are now taking the test across the nation. I guess Prichard didn’t want to sail into those muddy waters when they cut their graph off at the year 2000.

Anyway, there is plenty of evidence in the new NAEP math report that concerns in Kentucky about math – such as those I heard yesterday at the Education Assessment and Accountability Review Subcommittee – are well founded. I think when people carefully consider the new NAEP results, they aren’t going to do much cheering.

More on that later

(Note: data in graphs assembled with the NAEP Data Explorer Web tool.)