Getting a Better Picture from the NAEP

– Why you can’t ignore racial demographics

Over at the Prichard Committee’s Blog they have been busy posting all sorts of test results from the National Assessment of Educational Progress. Unfortunately, there is a consistent flaw in that effort – Prichard never tells its readers about the cautions in recent NAEP documents outlining how different exclusion rates, testing accommodation rates, and racial demographics can impact comparisons between Kentucky and the nation and other states.

You can find information about that in the “Pitfalls in Interpreting NAEP Scores” section of our freedomkentucky.org discussion of the NAEP, but here is a brief example of how the strong demographic advantages that Kentucky enjoys in NAEP can inflate impressions of how our public school system really performs. We use data from Kentucky’s top NAEP performance area – science.


The officially published NAEP Grade 8 Science average scores for Kentucky and the nation for all students in 2005, shown by the blue bars, were 153 and 147 respectively. That indicates Kentucky is doing a good job teaching science compared to the national norm.

However, once we start to consider the very strong differences in the demographic makeup of the student group in Kentucky and across the nation, that impression flip-flops in a hurry. After adjusting the national scores to the same racial demographics that we have in Kentucky, all of a sudden Kentucky winds up three points behind rather than six points ahead of the national average, as shown by the dark red bars.

So, is Kentucky’s science performance really all that great, or is this just a case of us getting an unfair advantage on NAEP because our state simply does not have the racial makeup that exists elsewhere?

And, is it fair to not even mention that possibility when even the latest NAEP reports say you have to consider demographic factors?

If you want to learn how I corrected the scores for racial differences, read on.