NAEP 2009 Reading: California Vs. Kentucky

It seems like a “no-brainer.” If you simplistically look at the overall fourth grade reading scores for all students from the newly released 2009 administration of the National Assessment for Educational Progress (NAEP), Kentucky ranks in a tie at 11th place from the top, while California ranks way down at 46th place.

So, Kentucky’s education system ‘whups’ California, right?

Well, maybe not. Let’s look a bit deeper.

First of all, let’s start with a clearly worded caution that appeared in older NAEP Report Cards such as on Page 13 in the 2005 NAEP Reading Report Card:

“In comparing states to one another, it is important to consider that overall averages do not take into account the different demographics of the states’ student populations.”

That caution isn’t so clearly stated in the 2009 NAEP Reading Report Card, but the intent and the problem still exist:

“Further, the many factors that may influence average student achievement scores also change over time. These include educational policies and practices, the quality of teachers, available resources, and the demographic characteristics of the student body.”

Keeping these cautions in mind, here is a breakout of the demographics for Kentucky and California on the 2009 NAEP. I put this graph together using data from the NAEP Data Explorer tool.


I don’t show two other categories, American Indian and Unclassified, as there are only trace numbers of students in those groups in both states.

When you look at the percentages of the different racial groups in the NAEP 2009 fourth grade reading samples, the disparity between California and Kentucky is striking. Kentucky is overwhelmingly populated by high scoring whites, while in California the majority of the students are lower scoring Hispanics.

Now here are the data on the reading proficiency rate (students scoring at or above NAEP “Proficient” (Again, assembled with the NAEP Data Explorer, though you can find this in the new Report Card, as well).


Right away, when we compare the only two racial groups that are consistently present in both states, we find that California’s whites tied Kentucky’s whites and California’s black students actually scored a bit higher than Kentucky’s blacks.

Suddenly, Kentucky’s really high overall rank and California’s really low rank don’t seem so revealing, do they? If we only look at the two racial groups where both states have notable populations, California actually would outscore Kentucky thanks to their slightly better black performance.



By the way, while the score graph above does include the proficiency rates for the other racial groups that are found in notable numbers in California, recheck the first graph – very few Hispanic and Asian/Pacific Islanders are found in Kentucky.

Clearly, the really big demographic difference between Kentucky and California is for Hispanics. So, let’s look at the Hispanic situation more closely.

Part of the reason California’s Hispanics score low is that many are also still learning English. Table A-4 in the Report Card shows the fully 30 percent of the entire raw sample of students NAEP wanted to test in California were English Language Learners. Most likely, the vast majority of them were Hispanics. Thus, since Hispanics make up about 51 percent of California’s NAEP tested students, I estimate that somewhere around 60 percent of them are still learning English.

The English learning problem is a challenge faced by only two percent of the students in Kentucky (Table A-4 in the Report Card). And, it is a problem because the NAEP is only given in English. So, California’s Hispanics who are still learning the language face a double challenge when they quest for high NAEP scores. For all practical purposes, the issue doesn’t even exist in the Bluegrass State.

Things look even worse for Kentucky in our California comparison when we examine Table A-7 in the new Reading Report Card. This table covers the percentages of English Language Learners and Students with Learning Disabilities that got excluded from NAEP testing.

Amazingly, despite their huge number of students learning English, California only excluded a total of 8 percent of its students classified as English Language Learners and Students with Learning Disabilities from taking the NAEP.

In Kentucky, we excluded a whopping 46 percent of these low-scoring students.

In fact, Table A-4 shows that out of the entire raw sample of students that the NAEP wanted to test, California only excluded 3 percent due to language or learning disability issues.

In Kentucky, we excluded a whopping eight percent of all the students in our raw sample that the NAEP wanted to test – that’s nearly one out of ten students! Most of those excluded were the very weakest from the very lowest scoring category – Students with Learning Disabilities. Kentucky played the NAEP exclusion game like it was the NCAA Basketball Tournament. Table A-7 shows only two other states in the entire country played the exclusion game harder.

If that large number of excluded Kentucky students had been tested, there is no doubt that they would have dragged Kentucky’s scores down – perhaps enough to wipe out our supposed gains, though no-one knows for sure.

Meanwhile, while Kentucky excludes rather than test, California educates almost all of its kids well enough that only a very few have to be excused from testing. And, that is true in most other states, as well.

Now, what do you think about that simplistic ranking of overall scores that makes Kentucky look so good? Does that simplistic ranking really tell you very much?

Does the claim that only Kentucky made progress in reading in both fourth and eighth grade look as credible? Maybe, we’re just best at gaming NAEP exclusion.