KDE stretching facts like a rubber band

Last week the Kentucky Department of Education (KDE) put out a news release about Advanced Placement test results that was obviously wrong. Based on the numbers in the original release, there was a 66 percent rise in just one year in the number of Kentucky students getting a 3 or higher score on the AP exams. I knew that couldn’t be right and immediately contacted the KDE to correct their news item. They did so within two hours, but failed to correct a sentence in the body of the report for the new numbers.

The KDE finally got it all right several days later (see the corrected version here) which is good, but how did such an obvious error slip by the KDE to begin with?

Now, the KDE has put out another troubled news release, this one supposedly carrying education commissioner Jon Draud’s assessment of educational progress in Kentucky.

There is a ton of stuff wrong with this release (starting with the fact that at present it says it was last updated on October 3, 2007 though it is dated September 2, 2008), but I’ll just cover a few, easily disproved items.

The KDE news release says:

“According to the National Assessment of Educational Progress (NAEP, or the “Nation’s Report Card”), Kentucky children are performing better in comparison to their peers nationally than they ever have before,” and “The 2007 NAEP in reading and mathematics showed that Kentucky’s 4th- and 8th-graders made gains when compared to the state’s performance in previous NAEP assessments.”

Not true.

Kentucky’s eighth grade NAEP reading scale score was 266 in 2003 but dropped to only 262 in 2007. That 2007 score is statistically significantly lower than the 2003 score. During the same time frame, the national score stayed flat, so we are not doing better than ever before as of the latest NAEP eighth grade reading results (Don’t believe it? Check out Table 11 in the 2007 NAEP Report Card for Reading).

Furthermore, Kentucky excludes learning disabled students at rates above the national average on both fourth and eighth grade NAEP reading tests, which means the state’s reading scores, as published, are inflated (Check it out in Table A-4 in the NAEP Report Card for Reading). The report card says exclusion and other issues must be considered before making claims about performance changes (see page 7), a caution the KDE habitually ignores.

In fourth grade NAEP math, in 1992 Kentucky’s scale score was 4 points below the national average. In 2007, Kentucky’s score was also 4 points below the national average. We improved, but so did everyone else, so there has been no improvement relative to peers in other states (Check that out in Table 6 in the NAEP Math Report Card for 2007).

Kentucky did narrow the gap for eighth grade math, but at a cost of a doubled rate of exclusion for students with learning disabilities (See Table A-4), throwing the published score improvement into limbo.

Why did we suddenly exclude twice as many disabled kids in 2007 given that NAEP now allows all sorts of testing accommodations?

Why does the KDE put out such easily disproved material when the correct information is readily available in the NAEP math and reading report cards?

In fact, why does the KDE often overstate the facts (there are plenty of other problems with this news release such as continuing to cite graduation and dropout rate statistics that have been officially audited and found in significant error)? Trying to stretch small improvement into something more just undermines KDE credibility. The facts are that Kentucky’s NAEP picture isn’t nearly as rosy as the KDE claims. In fact, the notable decline in recent eighth grade reading performance should be setting off alarm bells, not sounds of celebration, at the KDE.

Here’s another problem -- why do educators stand silent while the KDE continues to pull this sort of stuff? When will the education community realize that this sort of nonsense just undermines the credibility of all?