“Our NAEP scores are in question”

It came up during today’s Kentucky Board of Education meeting in a discussion about extra help, called testing accommodations, given to students with learning disabilities.

Kentucky Commissioner of Education Terry Holliday pointed out something I have been saying since 1999 – Kentucky’s testing policies for learning disabled students impact the validity of the state’s reading scores, both on state reading assessments and on the National Assessment of Educational Progress (NAEP).

Holliday specifically told the board:

“Our NAEP scores are in question.”

He’s right.

Click the “Read more” link to learn the shocking story about how Kentucky’s educators have been reading kids the so-called reading tests in this state.



From the early days of KERA, it actually has been permissible for Kentucky’s teachers to read all parts of the state assessments, including the so-called reading assessments, to a notable proportion of the students in this state.

The kids involved are identified as having learning disabilities. So long as a learning disabled child’s Individual Education Plan (IEP) calls for tests to be read to that student, then all tests, including state reading tests, are indeed read to the student. It’s been that way since the early days of Kentucky’s education reform.

One serious consequence of this very bad policy is that schools can escape all responsibility to try to teach the child to read – ever. The school system can just create IEPs with the reading accommodation for the student’s entire school experience – elementary school through high school graduation.

Of course, state tests indicate all is well, because the read-to student’s scores are reported and counted just the same as other students’ scores. Schools look good, parents and students are lulled into a sense of well-being, but the child winds up graduating totally illiterate.

I don’t know precisely how many children have been denied the opportunity to learn to read because of this horrible policy, but I suspect over the past two decades this runs into many thousands. I’d bet more than a few of these poorly served individuals are currently found in Kentucky’s burgeoning penal system.

As Commissioner Holliday pointed out, Kentucky’s extraordinarily bad testing policy also set up a collision with the validity of Kentucky’s NAEP reading scores.

The federally operated NAEP always intended its “reading” test to be a real test of printed text reading and comprehension, not just a spoken word comprehension test. Thus, NAEP test administration guidelines specifically prohibit reading the NAEP Reading Assessment to any student. When that NAEP rule conflicts with the student’s IEP, this federal test’s directions say the student is to be excluded from testing all together.

But, no-one at the NAEP anticipated how many students were going to get the reading accommodation in their Kentucky IEPs.

The conflict in rules came to a head with the 1998 NAEP Grade 4 Reading Assessment. Thus, Kentucky’s exclusion of students with learning disabilities on NAEP skyrocketed by that year, increasing from 4 percent of the raw sample of all students NAEP wanted to test in 1994 to a whopping 10 percent exclusion in 1998. That’s right; one out of 10 of all students the NAEP wanted to test for fourth grade reading got excluded in 1998.

Even today, with NAEP now allowing some other accommodations – but not the reading nonsense, Kentucky still excludes learning disabled students on the NAEP reading assessments at rates notably higher than the national average.

You can’t exclude so many of your most challenged students without inflating your test scores. After fighting this fact for several years, the folks who administer the NAEP finally had to agree.

NAEP’s administrators also were hearing from other states crying foul about Kentucky’s supposed nation-leading improvements in NAEP reading. Other states, you see, don’t allow such testing shenanigans on their reading assessments. They want their educators to at least make the attempt to teach all kids to read.

It’s now 2011. KERA has been on the books for more than two decades, and kids in Kentucky have been getting short-changed on reading instruction the whole time. State and NAEP reading tests results have both been getting corrupted all of this time as well.

Finally, it looks like the Kentucky Board of Education may act to fix this problem. The board is considering new testing rules that will bring us in line with NAEP standards and the normal standards in other states, where reading means READING, not listening.

Of course, there are some inevitable rumbles from some who would prefer for this deceptive nonsense to continue, along with some more reasonable questions about implementation impacts on a very small number of severely disabled students.

So, stay tuned on this, but I think the board is leaning towards doing the right thing.

Too bad it’s taken 21 years.