– What's it going to take to get accurate data?
I blogged yesterday about the misleading presentation of the 2008 EXPLORE and PLAN results in the Kentucky Department of Education’s (KDE) News Release 09-017 dated March 11, 2009.
There is more to add, but first a quick review.
I pointed out in yesterday’s Blog that the KDE’s tables create a false impression that national average scores on EXPLORE and PLAN exist for each year from 2006 and 2008 and that the national scores remained flat during this time.
The KDE’s news release actually reinforces that incorrect impression by specifically stating, “For the 2008 administration, scores nationwide remained flat, while Kentucky’s overall scores increased in nearly every subject area.”
Kentuckians deserve to be told that no one knows what the national trend in EXPLORE and PLAN has been since 2005. Only one fairly recent set of national comparison scores is available, and that was assembled back in the fall of 2005. More recent National trend data does not exist, despite the very false impression created by the KDE’s news release.
Now, however, I have done some more research, and the situation looks even worse.
I checked my old Blogs (now scrolled off our active site, unfortunately) and the KDE made the same, misleading presentation of national data last year, implying separate national scores had been collected in both 2006 and 2007. Here is the link to that earlier misleading KDE News Release 08-007 from January 30, 2008.
Still worse, I challenged the KDE on their erroneous data presentation last year. That didn’t stop the KDE from playing the same game with Kentucky’s citizens again.
In reality there still is only one recent set of national comparison scores for EXPLORE and PLAN, and that national score set was created in 2005. Despite the deceptive indication in the KDE’s tables, there is no national trend data available for EXPLORE or PLAN for 2006 through 2008.
Never the less, the KDE persistently parades this single set of 2005 national data as though it was from 2006, 2007 and now from 2008, as well. And, this year the KDE embellished the deception by specifically stating that the national scores were flat in 2008 when no data actually exist to support this claim.
The KDE’s misleading score reporting is unacceptable, and the fact that the department has now pulled the same stunt two years in a row is more egregious.
The public clearly deserves better.
Thursday, March 12, 2009
KDE Messes Up EXPLORE and PLAN Score Release – Part 2
Subscribe to:
Post Comments (Atom)
6 comments:
Richard,
I can't argue with you on this one.
It is misleading when folks skew data or overstate their claims. The department ought to walk it back. As you say, Kentuckians deserve that.
For much the same reason, the Bluegrass Institute ought take this one back as well: http://theprincipal.blogspot.com/2008/12/bad-science-its-not-study-its-spin.html
If we can just get the pros and the cons to quit messing with the data, we'd all be better off.
What's it going to take?
Richard
Richard,
I am glad you agree with me that the department’s news release is out of line.
However, I am disappointed that you would try to draw a parallel between the quality of the Hoyt report and the blatant score fabrication in KDE’s news release.
I reread your critique of the Hoyt report found at the link you provided. I would prefer to have Professor Hoyt respond to most of that. However, nothing you allege, even if it were correct, begins to approach the level of out and out fabrication of data that the KDE engaged in.
Actually, I think most of your comments are based on misconceptions of the Hoyt report, maybe shaded by a little outright bias. Let me provide one example of the problems I see in your critique using an example where you took a direct swipe at me.
You write,
“For example, the researchers cautiously state their terms, like this:
"What evidence there is suggests that KERA may have....."
But the Bluegrass Institute, being on a mission, has no apparent interest in caution. Innes claims,
"the UK scholars indicate that the way tax supports for schools are divided among local, state and federal sources has acted to reduce local taxpayers’ control over their schools in an inequitable way."
Very definite in his mind.
Well, my copy of The Merriam-Webster Dictionary lists “suggest” as a synonym for “indicate.” My “World Book Encyclopedia Dictionary” shows that “indicate” can mean “to give a sign, or hint of.” My “Roget’s Thesaurus” shows that “indicate” can mean “hint,” as well.
In other words, a variety of English language references make it clear that “indicate” isn’t the sort of “definite” term you attempt to portray it to be. What I wrote is a fair summation of what Hoyt and his co-authors originally said, and it isn’t as rigidly definitive as you allege.
Richard,
Yes, I do agree with you on KDE. Anytime a person (or group) with an ounce of integrity misleads, it ought to be retracted and KDE did that yesterday.
No, I am not drawing a parallel between KDE and Bill Hoyt’s study, but between KDE and Bluegrass Institute’s reporting of Hoyt’s study. That’s where the misleading occurs.
As for the level of fabrication - misleading is pretty much misleading. It comes down to a matter of intent. It’s slanting the evidence to color folk’s perceptions. KDE did it. You called them on it. They retracted.
BIPPS did it (in three cases I’m aware of). You’ve been called on it. And so far, you appear to cling to your spin as though it were true. That’s disappointing, indeed.
I remain unconvinced by Hoyt’s speculation on the impact sources of funding have on school district governance, but when Hoyt speculates, the reader knows it. That’s clearly not the case with BIPPS.
I can’t believe you want to parse the word “indicates” as a defense against admitting error. What’s next? ‘…that depends on what the meaning of the word “is” is?’
Just because a word has multiple meanings, does not mean that any of those meanings can apply to any use of the word. Syntax is determined by the context, and there, the words “has acted” give your intentions away.
In your sentence:
"the UK scholars indicate that the way tax supports for schools are divided among local, state and federal sources has acted to reduce local taxpayers’ control over their schools in an inequitable way."
X indicates that Y has acted to produce Z. If we substitute “hints” for “indicates” (and no one was trying to mislead) the sentence should have read: X hints that Y may have acted to produce Z. In your explanation, the verb confusion is ambiguous and misleading.
I don’t think there’s any question that the BIPPS spin is misleading. The only question in my mind is whether it was intentional; which I confess, I suspect. If I’m wrong, and it was not meant to mislead, somebody at BIPPS ought to say so - unambiguously.
…and I only mentioned you by name because, to your credit, you sign your own pieces.
Richard,
You need to read the latest evidence in support of the comments about the Hoyt report linked from here:
http://bluegrasspolicy-blog.blogspot.com/2009/03/school-boards-ducking-transparency.html
Your opinions may vary from ours, but you haven't produced any factual data to indicate we are wrong. Most definitely, nothing you allege that we did approaches anything like the same plane that we are talking about when the KDE fabricated scores, and a scoring trend, that simply did not exist. If a kid did that on a paper, he would be in real trouble, and rightfully so.
If you have factual data that shows we are wrong, that we will certainly examine, as always, and issue a retraction when appropriate. Our kids deserve no less.
But, as the latest example in the Enquirer shows, you haven't come close to making your case regarding the Hoyt report. In fact, front page news articles, among other evidence, back us up.
Malarkey.
The fact is, you continue to claim things Hoyt's report doesn't say.
...and this "latest evidence" is another perfect example of what's wrong.
Just where is it that the Enquirer piece connects the problems it notes to the sources of school funds? They don't. They don't because they can't. They can't because it would violate journalistic integrity.
As for Enquirer readers agreeing that the comments were "right on," where'd you get that?
I grant you some literary license but the readers are readers and you have no idea what they think. The only place to go for that might be the 7 comments made on the Enquirer's website related to the article - as if that were determinitive. So who told you they get it?
Was it the one who wrote, "I am a parent in the Fairfield School District. I have had no problems contacting ALL board members at this district. ALL but 1 have email address. The one has a phone number and I know for a FACT that if you call him and leave a message he WILL contact you back..."
Was it the commenter who said, "On the website is a complete minutes of all the board meetings and when meetings will take place. Not sure why this guy has a problem with them..."
Or perhaps it was the one who wrote, "It doesn't matter if you can contact them or not. If you do get in touch with them they will tell you they only handle the monetary issues with the school system..."
Then there was, "IF YOU VOTE FOR A SCHOOL TAX TOU ARE STUPID." You've got something there.
Or, "Oh well - I guess each time I vote NO, I can blame it on public school educationz, since I'mz dumbz?"
And this, "Right on the homepage were meeting dates, board members and contact info, and great links like one to a list of all the staff with their postition and contact info. Check it out: http://www.norwoodschools.org/boardEducation.aspx. I was pleasantly surprised to find Norwood Board of Education's website so exemplary. WOW - great job Norwood. Kudos to the new webmaster."
Or finally, "Biographical information is relevant when a candidate is running for election to the board. Once they are elected I see no need to have their biography posted on a website..."
Anything y'all think you can string together, no matter how flimsily, is apparently fair game in your shop. Your Enquirer post
should carry warning label. "Warning! This post may be total horse hockey."
Obviously, this is my problem, not yours. I keep expecting BIPPS to perform to a higher standard. But there seems to be little interest in credibility; just politics. I have known you to do better work than this. This is disappointing, but I'm done trying.
Richard,
It's interesting that most of the favorable comments you mention are from the North side of the Ohio River.
Anyway, I guess you and I are not going to agree on this. You don't seem to be able to see the connections that seem so apparent to me, so I'll move on, as well.
Post a Comment