Tuesday, January 13, 2009

CATS audit mess – no one knows the cost!

State auditors told the Kentucky Legislature’s Government Contract Review Committee today that Kentucky’s CATS school assessment program cost at least $18.6 million in 2008 – FAR higher than any figure I have seen previously – but the auditors were quick to point out that this figure does not include the total cost. The shocking reason is there simply isn’t any way to determine how much local school districts pay to support CATS testing.

The news article doesn’t mention why those costs are unavailable, but the answer is that the state’s MUNIS education finance system remains a mess more than two years after Kentucky Legislative Research Commission Report 338 identified major problems with this fiscal accounting program.

After almost 19 years of KERA, the idea that we still cannot accurately account for education spending, not even on big items like our assessment program, is simply an outrage.

So, hats off to State Representative Brad Montell R-Shelbyville, for calling this situation, “an unbelievably convoluted, mixed-up mess.” Representative Montell, that it surely is!


The Principal said...

This morning the Senate A&R was told that it would cost $5.5 million per year to move forward with the Kentucky Instructional Data System (KIDS) which would "warehouse" financial data, among other. I'm not sure that would solve the problem with the chart of accounts that you are referring to but would go a long way toward providing transparency once the chart of accounts problem is rectified.

Is BIPPS in favor of funding for the KIDS system?


Richard Innes said...

RE: The Principal’s Questions

BIPPS hasn’t taken a position on KIDS.

However, here are some issues to consider:

Right now, some of the key problems with MUNIS involve poor understanding and management of the individual accounting codes. The result is expenditures cannot be accurately extracted in ways that can accurately support efficiency studies and other management needs. The discussion in the main blog about the auditor’s inability to extract local school and district support costs for CATS is a great example. I’m not a KIDS expert, though I have attended some briefings, and I don’t see how KIDS could fix that. Warehoused erroneous data is just erroneous.

One of the main purposes of KIDS, and a challenging feature to implement (read expensive), is linking KDE databases to other agency databases. That costly plan doesn’t seem to offer much promise to improve education, making KIDS look like one program that could at least be delayed given the current financial situation.

Some are also expressing concerns about KIDS privacy issues for students and parents. Should other state agencies have access to a lot of data maintained by KDE? For that matter, should KDE have free access to all the other agency databases? How will KIDS manage approved, and unapproved, cross-database access? Of course, any expansion of access generally brings increased risk of hacking, as well. With KIDS, a successful hack, say into a public safety database, might also expose some or all KDE data. Of course, a sophisticated IT professional working in the system might be able to access all sorts of material that could be very valuable in the black market of identity theft, and so forth.

Finally, I don’t think KIDS is likely to improve transparency. It seems more focused on internal governmental communication rather than a link to the public.

Again, these are just some thoughts at this time.

Brushfire said...

MUNIS is a mess.

In terms of calculating district expenditure on CATS, though, the problem is more complex. Here's the thing: CATS preparation and administration is suffused throughout the work of administrators. You've got staff members who serve as Building Assessment Coordinators as an essential part of their work (though its not the sum total of their job responsibilities). Tracking work that they do specifically around test administration would be nigh impossible. What about test preparation activities? All classroom and interim level assessments should be aligned with CATS. Do we count that as a part of the testing costs?

It's just not something you can extract from the rest of our work as educators.

Richard Innes said...

RE: Brushfire’s Assertion Local CATS Costs Can’t Be Extracted

I don’t buy that – we haven’t even tried.

Industrial Engineering is all about doing this sort of fairly involved, but doable sort of thing. You have to do some time/motion studies to determine some of the costs, but a lot of the local district and school support for CATS could be identified and tabulated – if someone really wanted to do it.

For example, the audit talks about how local districts had to pony up extra money to cover the third grade NRT, money that should have fully come from Frankfort. That money is an easily identified added cost directly tied to CATS.

All of the local teacher time and related costs to grade writing portfolios are also direct CATS expenses. Ditto for substitute teachers hired so the regular teacher can grade the portfolios.

The opportunity costs of teacher time and student time to administer the time-consuming CATS is another direct cost.

Kids and teachers know what CATS “scrimmage” tests are. Time spent administering and grading these are also CATS costs.

Can we lock this up perfectly? I doubt it.

But, can we get a much better idea of how much real cost associated with CATS is borne at the local level – you bet we can. And, we need to do this. Those local costs are considerable, as a 2005 OEA report (LRC Research Report 328) indicates, where local costs in Fiscal Year 2004 were estimated to roughly equal the state level expenses ( $16.4 million for total local school and district costs and $20.8 million at the state level).