Once again, major media reporters were absent, so this may be the only eye-witness account you will see of second meeting of the CATS Assessment and Accountability Task Force in Frankfort.
Education Commissioner Jon Draud opened the meeting by emphasizing – again – that he was not looking for this group to suggest dramatic changes in CATS. He wants to put off any substantial changes until after 2014.
Draud’s position clearly didn’t sit well with Kentucky State Senator Dan Kelly. Kelly challenged Draud’s statement that putting off changes until after 2014 was a consensus position. Kelly correctly pointed out that the committee so far has taken no votes on any positions and has not reached any consensus about direction.
The meeting’s biggest surprise – a disappointing note – was Jim Applegate’s announcement that he is leaving his position as Vice President for Academic Affairs at the Kentucky Council on Postsecondary Education. As a consequence, Jim is also resigning from the CATS task force.
Applegate took a few minutes to share departure comments about what the task force needs to do regarding the CATS assessments. In a key part of his statement, Applegate said:
“If out of all of this we don’t end up with an assessment system that allows us at every step of the way to understand where the individual child is on the road to the next step after high school, on to college, on to the skilled workplace, whether they’re behind, they’re ahead, they’re on track – and, it’ll help us understand how to intervene with that child to do the right thing and then allows us longitudinally to reassess at a point in the future to know whether our interventions work or not – then, I don’t know why we’re even bothering to assess. You know, uh, I don’t know what the point is.
It’s all well and good to say this school’s doing good by some standard, and that school’s doing good by some standard, but if the students aren’t moving on to the next levels successfully – if that’s not what our focus is – then I don’t know why we even bother. I don’t care that school X is hitting better at some measure than school Y. I don’t. I care whether the kids that come out of that school are ready to move to the next level.”
I couldn’t have said it better, and Applegate’s well-directed focus will be missed both on the task force and throughout Kentucky. He has been remarkably forthright with the citizens of the commonwealth, a quality too often lacking in Frankfort.
Applegate’s departure leaves one lone postsecondary education voice on the task force, rendering the group more dominated by K to 12 professionals than ever. That will make it harder still for this group to overcome a self-interested inward focus that might not lead to the best answers for CATS and, ultimately, for the children of Kentucky – a situation that might not be lost on Applegate who summarily left the meeting right after his short presentation.
There was a considerable amount of discussion about writing portfolios in CATS. While there is wide agreement that portfolios are a good instructional aid to teaching writing, there is tremendous controversy within the group about whether the portfolios should stay in the CATS and whether teachers will drop portfolios – even abandon writing instruction in general – if portfolios are taken out of the assessment.
One surprisingly candid comment came from a somewhat unexpected source, Jon Draud’s hand-picked testing expert Doris Redfield, the only testing expert in this entire group.
Dr. Redfield said:
“If you are going to do an assessment of learning – an accountability assessment, an achievement assessment – what you want are the students’ very best possible products – that’s probably measured on-demand because of the reliability and validity factors.”
In other words, measuring writing on an assessment is most properly done with on-demand writing prompts such as those already given during the CATS tests. In contrast, writing portfolios do not provide the same level of reliable and valid scores.
There isn’t anything new in Redfield’s statement, but it was refreshing to hear her echo this, anyway.
Unfortunately, one of the best questions placed before the committee regarding portfolios came at the very end of the meeting. That’s when Steve Stevens, the President of the Northern Kentucky Chamber of Commerce, asked how we can be so sure portfolios belong in our assessment when Tennessee outscored us on the recent National Assessment of Educational Progress on-demand writing assessment and that state does not use them in its assessment program
(If you want to know how Tennessee did on writing [they ‘whupped’ us], just click here).Tennessee’s writing performance was added to the agenda for the next task force meeting, and it will be interesting to see what develops. I hope there is also a discussion of how Tennessee beat us without having any open-response questions on their other academic tests, as well.
I also hope, as Jim Applegate implored, that the committee keeps their eye on the real ball. The CATS is a hugely expensive program. We should get a lot more for our money from this exam, but as Applegate hinted, we don’t get anything reliable about individual students and their performance over time. We don’t really get much we can trust about teachers and the education programs they are using, either.
If we can get away from a lot of 20-year-old, out-of-date thinking about testing, we probably can run CATS more efficiently and effectively while we do a much better job of helping students.
And, I don’t think Kentuckians want to wait until after 2014 before making that happen.
