The Bluegrass Institute – from its earliest days – has sharply criticized Kentucky’s CATS school assessments and their underlying standards. While CATS has been telling Kentucky that our school system is doing just fine, more trustworthy evidence shows otherwise.
Now, with mediocre test results piling up, especially for Kentucky’s racial minorities, it looks like more and more legislators on both sides of the aisle are starting to ‘get it.’
This was particularly evident during today’s (November 17, 2008) meeting of the Interim Joint Committee on Education. The Kentucky Department of Education presented the latest results from CATS and No Child Left Behind, and legislators were quick to cite unsatisfactory performance gaps for whites and other racial groups, which persist today despite nearly 19 years of education reform.
The bad news triggered the following comments about general flaws in CATS from Representative Derrick Graham of Frankfort. Said Graham about a primary need to revise the underlying standards and curriculum driving CATS,
“Number one is when we talk to our universities and our colleges and our technical people that we get their input as to say what is it that you all want our kids – where do you want the kids to be by the time they graduate from high school? We need to get their input in terms of development of a curriculum, not that’s based on the CATS testing and evaluation of the CATS testing, but is based upon preparing these kids for the next level. And, I don’t think we’ve done that.”
Right on, Representative Graham. The high proportion of college freshmen who need remedial courses makes it abundantly clear that the CATS isn’t related to what kids need next. But, it needs to be.
As legislators in the meeting reacted in dismay to more data in the department’s report, including results from ACT testing that show very few kids are on track to be ready for college, sentiment certainly seems to be growing that after all the time we’ve been doing reform, making few if any changes in CATS for another six years simply isn’t going to pass muster.
Monday, November 17, 2008
CATS Deficiencies – More and More Legislators Get It
Subscribe to:
Post Comments (Atom)
2 comments:
OK, this is from one of those libertarian educators who reads your blog. I work in the belly of the beast...I confess I actually have been a principal and now work as a district administrator. I agree there are serious flaws with CATS, but we need to be really careful about where the problem in education actually lies. First point: we shouldn't scrap the test altogether because the alternatives are norm-referenced, not criterion referenced instruments. We need to know that our students are progressing against a clear, steady standard of proficiency in core subject areas. Norm-reference tests like the old standby, the CTBS, simply compare one kid to the total population. In terms of trying to measure individual student progress or a whole school's progress over time, you can't use a normed test because the target moves every time the population does. Likewise, the ACT is designed to separate students, to predict college performance, NOT to measure progress or assess a clear standard. I don't mind using the ACT as one component of our school accountability system, but again it only measures one kid against all others, and it is meant to identify winners and losers, not assess a student's performance against a stable standard. So, reform CATS if we must, but please don't throw out standards in the process.
My ultimate point is this: there are many problems with public education. CATS is actually pretty low on the list. The problem is that schools are structured for an industrial era. The structure and professional culture of schools is one that puts teachers in isolation from one another, depriving them of opportunities to improve and refine their professional practice in collaborative, self-reflective ways that are geared toward improving student achievement. It is my greatest frustration as an educator. We need to TOTALLY rethink school structure and find new, creative ways of delivering rigorous content to kids and measuring their progress. I am an advocate for school choice because I think market incentives are the best way to get at this kind of transformation in schools. Tinkering with the CATS assessment won't get it done.
Anonymous November 19, 2008 7:42 AM raises some interesting points.
If we really were able to accurately set cut scores for properly designed criterion referenced tests, those tests might indeed be better for some purposes than norm-referenced assessments. But, that is a really big IF. CATS shows exactly how big.
For one thing, there have always been questions about the questions – in CATS. Are they properly designed to get at the real knowledge and skills kids need next in life? And, is the score for what passes for “Proficient” work really accurate?
There has been a ton of discussion about inadequate coordination and sometimes rather contentious relationships between the public school community and the postsecondary community about what kids really need. So, 18 years after KERA’s enactment, we find ourselves with a set of math standards and test questions which most people realize are “a mile wide and an inch deep.” You cannot fabricate a decent criterion objective test from such bad standards.
Scoring is another issue. Even a casual look at our freshman college remedial course requirements and our high school graduation rates say CATS scores provide a clearly inaccurate picture.
The obvious further inflation that was designed into CATS scores following the 2007 standards resetting just adds to the overall impression that Kentucky lacks the will, and perhaps even the Wisdom of Solomon ability, to craft a reasonably accurate, properly graded criterion referenced test. We’ve been trying for almost 19 years, and we still have not been able to get this right.
That brings us to another issue – is there a change in process for norm-referenced testing which makes them more like criterion-referenced tests? The ACT’s addition of its Benchmark Scores several years ago certainly opens a new dialog in this area. Is testing nationwide changing all around us while Kentucky languishes with 20-year old testing philosophies about criterion versus norm-referenced testing? The addition of the Benchmarks now gives ACT some characteristics of both types of tests. Is it a new hybrid?
One thing is certain, the questions on the ACT are most definitely related to what most of our kids really need next, while there is strong evidence (see many reports in the Bluegrass Web site) that CATS can’t make similar claims. If the test questions are inadequate, no amount of scoring manipulation is likely to recover much, if any, useful information.
And, it’s the lack of trustworthy and useful performance information about KERA, nearly 19 years later, and the excessive expense, that make CATS a notable part of the problem, not a minor issue.
Post a Comment