Friday, October 3, 2008

Testing Expert Slams Former Kentucky Test Gains as “Bogus”

“If you’re a parent in Kentucky, what you care about is whether your kids can read, not how well they can do on the state test,” Mr. Koretz said. “And the national assessment told us that, in fact, the gains in the state test were bogus.”

[Nationally-renown testing expert Dr. Daniel Koretz, Harvard University, speaking at an American Enterprise Institute panel discussion on September 22, 2008, As reported in Education Week (subscription may be required)]

To illustrate his comments on illusions of progress from state tests, Koretz used for an example the great similarity between the Kentucky Instructional Results Information System (KIRIS) and the National Assessment of Educational Progress (NAEP). Koretz says that due to the similarity between the two tests, the results for fourth grade reading should have moved, “…more or less in lock step.” Instead, while scores rose sharply on the KIRIS, they slid slightly on the NAEP.

Koretz also noted that KIRIS resembled an NCLB prototype test, but he didn’t make it clear that Kentucky’s current CATS assessment actually did grow out of the KIRIS “prototype” and is indeed used for NCLB today.

KIRIS is now a failed history, but, could Koretz’s strong words apply to CATS?

To answer that question, take a look at “CATS in Decline: Federal Yardstick Reveals Kentucky’s Testing Program Continues to Deteriorate.”

In this report you will see evidence that scoring on the CATS, as in the past with KIRIS, has also been getting more and more out of step with the NAEP over time.

This figure, taken from the "CATS in Decline" report, shows how the rigor of scoring for fourth grade reading on Kentucky's assessments has steadily gotten easier over time during the transition from KIRIS to CATS. On this figure, a score of 100% indicates perfect alignment between Kentucky's score of "Proficient" and the NAEP score of "Proficient." In contrast, a score of zero indicates what Kentucky scores as "Proficient" the NAEP only scores as "Basic" performance, meaning a student has only made partial mastery of reading for his grade level.


The latest data for 2007, shown as a red bar, indicates that what Kentucky now grades as "Proficient" reading is actually considerably below what the NAEP considers only a "Basic" performance.



If you want more details, check out the full report. In any event, it appears that Koretz's strong concerns about the KIRIS “NCLB prototype” remain valid for Kentucky’s current NCLB test, as well.

4 comments:

Anonymous said...

Kind of related, but not really: What are your thoughts on SBDM councils? I just got back from the PTA's holding of an SBDM election for parent representatives. Out of a school of roughly 715 students, only 27 parents showed up to vote (attendance at the meeting was a mandatory requirement). The election was not publicized very well and two current PTA officers were elected as the parent representatives.

Does this sound strange to you, or is it just me?

David Adams said...

It's about par for the course. SBDM elections usually don't get much attention and neither do parent members of the SBDM, usually. So no, it's not just you.

Anonymous said...

Then why have SBDM? I like the idea of local control of schools, but this just seems like a waste of time if all they are going to do is serve as a rubber stamp for the principal or the PTA.

Anonymous said...

Anonymous (October 7, 2008 1:05 AM) asks, “Then why have SBDM?”

The answer is that KERA probably would not have been enacted without this mock program that local control of schools was being maintained.

The truth is that KERA ripped local control away from both local school boards and parents. Local boards get into lots of trouble today if they try to impact what happens in schools.

In truth, parents also lost out. By law parents are always a voting minority on the SBDM. The law strictly requires professional educators in the school to outnumber parent reps.

So, real control of schools is in the hands of professional educators. Professional educators generally have a closer philosophical alignment with the Kentucky Department of Education than with local parents and parent issues. Furthermore, professional educators rely on Frankfort for the continued good standing status of their teaching credentials, and therefore, their jobs. Teachers don’t want Frankfort mad at them.

So, SBDM were really a cover for the shift real control of schools away from the local community and the parents and into the hands of professional educators, who in turn are beholden to Frankfort. The clever guise was that local control still existed and that parents had a powerful voice in their schools.

Parents today have figured this out, hence the limited interest in running for SBDM and the almost complete absence of parents on SBDM voting nights.