Logic left behind, or no logic at all?

Another educator attack on No Child Left Behind has just been published in the Courier-Journal.

Disputing what NCLB shows, Louisville high school teacher Noah Cooksey claims the Long Term Trend data from the National Assessment of Educational Progress (NAEP) show “Today's students outperform their counterparts from 40 years ago at every age level in both reading and math.”

Well, that just isn’t so.

In fact, on page 2 the “NAEP 2008 Trends in Academic Progress (NCES 2009–479)” report card clearly and plainly says:

“The average reading score (in 2008) for 17-year olds was not significantly different from that in 1971.”

“The average mathematics score (in 2008) for 17-year-olds was not significantly different from that in 1973.”

When it comes to the Age 17 data, Cooksey doesn’t know what he is talking about.

It gets worse. Click the “Read More” link to see that.



The NAEP report includes graphs of the Long Term Trend data for the entire period Mr. Cooksey discusses. Here is the reading graph, cut and pasted directly from the NAEP report card (click on the graph to enlarge, if necessary).


Notice that the Long Term Trend NAEP saw some changes in 2004, but the people who run the NAEP consider the more recent scores (shown with solid lines) to be generally comparable to the earlier data (dashed lines) and report all the reading data together on this graph.

Look at the “pesky” asterisk footnote (Cooksey obviously didn’t). The asterisks identify scores that are considered statistically significantly different from the 2008 score. Any test score for each age group prior to 2008 that is not statistically significantly different from the 2008 score lacks an asterisk.

In particular, you don’t see an asterisk on the 1971 score for Age 17 students because the one-point score difference is not statistically significant. This tiny difference is meaningless and could be entirely due to sampling errors; so, the NAEP report refuses to say the 2008 score is significantly higher than the score in 1971.

It gets worse.

Cooksey fusses about people who think our schools are in crisis. Well, look at the Age 17 scores for 1988, 1990, and 1992. Scores for those years ARE statistically significantly higher than the 2008 Age 17 reading score. So, high schools in the United States did indeed LOOSE ground in reading over the past two decades. With a large proportion of our students arriving at college with inadequate reading skills, the last thing we need is for our high schools to show deterioration in this critical area. Is this a crisis? Maybe.

Side note: Cooksey says the Long Term Trend NAEP is given every four years, obviously another error on his part. In fact, this assessment has been given at highly irregular intervals throughout its history. You begin to wonder if Cooksey knows anything about this federal testing program.

Now, look at the Age 13 reading data. The 2008 score of 260 is the same as the score back in 1992. So, reading performance for middle school students on this NAEP series has been flat for almost two decades, too.

Here is the same sort of depiction, this time for the NAEP Long Term Trend math scores.


Once again, there is no statistically significant difference in the Age 17 scores between 1973 and 2008. In fact, the high school level math scores have been flat since 1990. That helps explain how the US is sliding in international rankings for mathematics and science performance at the high school level. That is a serious situation as more and more high tech jobs are leaving the US for foreign shores.

We have made some progress in the lower grades in math, but not that much. Sadly, we have not been able to capitalize on that in our high schools, and what really matters the most is the final product of our school system.

Certainly, the stagnation in our high school performance is a serious situation that warrants some very aggressive attention – given the significant foreign education competition, a situation serious enough that some might feel inclined to call it a crisis. By the way, the TIMSS testing that Cooksey mentions only covers grades 4 and 8, so it won’t show this crisis. How clever of him to overlook the high school results from PISA, the “Program for International Student Assessment,” which does look at high school math and science for Age 15 students (Primarily 10th graders).

Here is how we did on that according to the National Center for Education Statistics (Note: OECD is the Organisation for Economic Co-operation and Development):

“In PISA 2006, U.S. 15-year-old students' average mathematics literacy score of 474 was lower than the OECD average of 498, and placed U.S. 15-year-olds in the bottom quarter of participating OECD nations, a relative position unchanged from 2003.”

“In PISA 2006, U.S. 15-year-old students' average science literacy score of 489 was lower than the OECD average of 500, and placed U.S. 15-year-olds in the bottom third of participating OECD nations.”

To sum up, when you look at the real situation for the end-product performance of our students on the Long Term Trend NAEP, it certainly seems like education nay-sayers – and maybe even some of the bad news from NCLB – have it a lot more accurately than teacher Cooksey.