Monday, November 30, 2009

More on the Every1Reads deception in Jefferson County Schools

Jim Waters just posted an excellent commentary on the deceptive reading test results that the Jefferson County Public School system has been handing out through the Every1Reads program.

Jefferson County turned true reading scores from the Kentucky Core Content Tests (KCCT) part of the state’s CATS assessments on their ear. Instead, Jefferson County invented its own scoring system, saying that any student who scored anywhere above “Novice” on the KCCT was reading “At or Above Grade Level.”

Let’s explore what the real reading performance in Jefferson County looks like. We’ll look at claims from the Every1Reads Web site and compare them to proficiency rates for the district and the entire state on the KCCT. We’ll also make a comparison to the far more credible National Assessment of Educational Progress (NAEP) reading results for Kentucky.


The red bar shows that Every1Reads claimed on average that 91.5% (rounded in the figure) of all tested students in Jefferson County scored “At or Above Grade Level” for reading (see the Every1Reads Web site for this number).

Those tested grades included three to eight plus grade 10.

While Every1Reads didn’t report a by-grade breakout, I calculated the eighth grade percent of students in Jefferson County who scored above Novice from the district’s Kentucky Performance Report for 2007.

The eighth grade percentage of students who scored above Novice was also 92 percent, equaling the overall average for all grades as shown in the figure.

Now, let’s see how Jefferson County really did against the real state standards, which do not have anything in them about “At or Above Grade Level” performance.

Jefferson County’s eighth grade students only scored 59 percent proficient on the KCCT reading assessment, far below the 92 percent Every1Reads fiction. Jefferson County also scored notably lower than the statewide eighth grade KCCT reading proficiency rate of 65 percent.

But, let’s really put this in perspective.

We know that the KCCT is an inflated assessment. That is one reason why the legislature shut down CATS and is requiring the KCCT to be revised once new state standards are in place.

How inflated is the KCCT? Look at the last bar in the graph.

On the 2007 National Assessment of Educational Progress (NAEP), only 28 percent of Kentucky’s eighth grade students scored Proficient or Above. That’s all. Thus, eighth grade reading proficiency rates reported by the KCCT in 2007 are more than double those from a more credible test.

And, while there are no NAEP scores for Jefferson County, don’t forget that the district does have a lower KCCT reading proficiency rate than the overall state rate.

Draw your own conclusions about the real reading proficiency rate in Jefferson County.

It's possible to do a similar sort of analysis using the ACT EXPLORE test results for the eighth grade. If you want to see that, make a comment and I’ll add this additional evidence that Every1Reads is seriously misleading Louisville about the performance of its school system.

2 comments:

Anonymous said...

Percent at or above NAEP Proficient (B+/A- and above on classroom performance) is not the same as percent at or above NCLB proficient (C-/C and above on classroom performance).

Richard Innes said...

RE: Anonymous Nov 30 at 3:09

Percent above NAEP Proficient most definitely is not equal to the proficiency rate reported for NCLB, but who says the current NCLB "Proficient" standard in Kentucky means very much?

The NCLB rate in Kentucky is the same as the KCCT proficiency definition. That KCCT definition does not relate to anything relevant in the real world, and the proficiency standards in the KCCT have steadily been inflated over time, as well (I've done considerable research on that with the NAEP Ruler approach).

However, since you raised this point, I will show you some more about Jefferson County, the NAEP, and the ACT EXPLORE scores in a day or two.

That will show that the NAEP Proficiency rate agrees rather well with the EXPLORE rates. And, those EXPLORE rates can be related to something very worthwhile.

Also, I've never seen a report that discusses the B+/A- smf C-/C "stuff." Anonymous, can you provide references, or is this just your opinion?