Kentucky’s school test results will look different on Wednesday

As I mentioned earlier, the Kentucky Department of Education will release the 2009 No Child Left Behind test results in two days.

Jim Warren at the Lexington Herald-Leader has a nice article that covers some of the changes in this year’s report. For anyone interested in our schools and their accountability program, Warren’s article is worth reading.

Warren probably gets one item incorrect, however. He writes, “No Child Left Behind data will look the same as in previous years.”

In fact, the department of education indicated in a School Curriculum Assessment and Accountability Committee meeting some time back that due to complaints from a professor at the University of Kentucky, the test accountability data for each school this year will not include a breakout of the number of students in each student subgroup that were tested. The professor claims it is possible to reconstruct scores for student subgroups too small to normally get score reports when the counts of students are presented. He supposedly created a study on this; but, so far, promises from the department to send me a copy of that study have not been fulfilled.

I suspect the professor’s work overlooks a key point. Creating meaningful scores for small student subgroups is impossible with Kentucky’s current testing system. Kentucky’s assessment scores are not valid for individual students because of something called test matrixing. That fancy term basically means that each student only answers a subset of questions on the test. As a result, no individual Kentucky student is completely tested on the subject matter. Only when large numbers of student responses are averaged together does something approaching a valid performance picture for the entire student group in a school start to appear.

So, there isn’t any reason to try to compute scores for small student subgroups with the current testing program in Kentucky. It is mathematically possible to generate scores, but the results are meaningless.

However, eliminating information on the number of students tested is an important omission. It will hamper independent research on the real performance of Kentucky schools and eliminates the ability of independent researchers to confirm the validity of information in the Kentucky Department of Education’s reports.

Aside from the Bluegrass Institute, I am informed that even the Prichard Committee for Academic Excellence holds an unfavorable view on this reduction in government transparency.

Anyway, the department of education has heard our complaints, so they may have figured out that there really isn’t a good reason to drop reporting the student numbers in the “No Child” reports. We’ll see what they decided on Wednesday.

Warren also totally misses another key point in his report. These scores are coming out way too late to comply with No Child Left Behind requirements. Last year, we got “No Child” scores on August fifth. We’ll want to learn more about that, as well.