Prichard Committee Plays Word Games with Data

A new paper from the Prichard Committee for Academic Excellence continues that organization’s tradition of stretching data from the National Assessment of Educational Progress (NAEP) too far.

This time Prichard claims, “Kentucky students lead the nation in science test scores.”

Really?

Here are some facts. In the somewhat dated 2005 NAEP eighth grade science assessment that Prichard references, Kentucky’s performance was statistically lower than 11 states, in a statistical tie with another 12 states, and only statistically higher than 20 other states. A total of 44 states participated.

You can’t find this statistical significance information in the NAEP Report Card, unfortunately. You have to know how to use the on line NAEP Data Explorer, including knowing how to use the Data Explorer’s statistical significance test tool.

That level of research detail apparently escapes Prichard.

However, even if you simplistically rank states the way the Prichard Committee did and ignore any statistical significance tests (both activities, I want to stress, are not appropriate), then 20 states still outscore Kentucky.

Where did Prichard ever get the idea we “lead” the nation for eighth grade NAEP science? We outscored the national average slightly, but that isn’t leading – not by a long shot. That honor goes to North Dakota, which outscored us by more than 10 NAEP scale score points, a very statistically different score.

By the way, my “The Merriam-Webster Dictionary” says that the word “lead” means to “go at the head of” or “be first.” That agrees nicely with relevant definitions in the on line version of this dictionary.

Given the meaning of the word, do you think Prichard makes an unacceptable stretch by claiming Kentucky students “lead” the nation on eighth grade NAEP science when we clearly have plenty of states unquestionably ahead of us?