Saturday, February 28, 2009

The Real ACT Deal

Back last August when the overall average ACT scores for all 2008 high school graduates came out, I cautioned that the overall scores might be impacted by the performance of students who didn’t attend public schools.

I was right.

Finally, through the good offices of State Senator Jack Westwood (Thank You to this supporter of government transparency), I have obtained a data disk with 2008 ACT scores and participation numbers for all the public high schools in Kentucky. When this data is combined with the overall averages and participation numbers for the state, it is possible to determine how the public schools did separately from the performance of other-than-public-school graduates.

The graph tells the tale.


As you can see, the public and other-than-public-school ACT Composite scores in Kentucky actually became equal in 1998. This was a time when many parents were “bailing out” of Kentucky’s KERA-driven public schools. Those poorly prepared transfer students dragged down the other-than-public-school average scores dramatically.

Times change. The greater flexibility in private and home school systems allowed these non-public alternatives to adjust to their vastly increased student load (which nearly doubled during the years covered in the figure), and the results since 1998 are obvious, and unmistakable.

Starting from the same point in 1998, Kentucky’s public school alternatives have significantly outperformed public schools on the ACT Composite.

Furthermore, because there are now notable numbers of Kentucky kids in other-than-public-school programs (which includes home schoolers), the non-public students' much higher ACT averages pulled up the overall average score notably in 2008.

As the graph shows, by themselves, Kentucky’s public schools only increased their ACT composite by 0.4 points since 1998.

In very sharp contrast, the other than public Composite skyrocketed from 20.2 to 22.2 in the same time interval. In fact, that 2008 Composite score for private schools was 0.4 points higher than the other-than-public-school Composite way back in 1993 (the earliest data I have). However, only 2,668 other-than-public-school students took the 1993 tests while 5,257 did in 2008, an increase in other-than-public-school enrollment of almost 100 percent. Often, as ACT participation increases, scores go down, but that obviously didn’t happen in Kentucky.

Well done, schools of choice!

13 comments:

Susan Weston said...

Richard,

Could you publish a table listing the numbers that go with the dots?

Hempy said...

The graph does not account for the economic demographics of the respective school populations. Nor do private schools accommodate many, if any at all, of students with disabilities.

Hence, the scores are of no particular significance other than a score.

Richard Innes said...

RE: Susan’s Request

I will put up the source spreadsheet that calculates the other-than-public school ACT scores in Kentucky after the new Bluegrass Institute Web site and supporting FTP site are fully up and running. Unfortunately, I got caught by the transition in Web servers. The spreadsheet would be unreadable if I tried to put it up as a graphic in Blogger. If you really want it quickly, just e-mail a request to innes@bipps.org.

Keep in mind, the public school ACT data that are needed to calculate that spreadsheet must be obtained from Senator Westwood, as the Kentucky Office of Education Accountability has not chosen to put them on line. The all-student scores can be obtained from the ACT Web site or the Kentucky Department of Education.

RE: Hempy’s Demographic Comment

A lot gets said about private schools, but unfortunately there is little really trustworthy data to say anything one way or the other. Private and home school populations tend to avoid data collection of this nature. In fact, because home schooling parents don’t have to report to the department of education (just to their local school district), even the count of such students in Kentucky is suspect.

Here are some questions to consider, however. Do you think the students in private schools got dramatically more well off in recent years versus public schools? Did the other-than-public-school student group get less diverse after 1998? Did the other-than-public-group enroll fewer students with learning disabilities after 1998?

The answers to those questions would have to be yes to help explain the very different ACT performance trends in these two school systems.

Anonymous said...

Its no great secret that the privite schools do a better job. their teachers have to produce in order to keep their jobs. where public schools teachers are not held to the same high degree of sucess. I have been in the FCPS system since 1983 not including my own education at public school. Too many students in a class and the students that can not keep up with the regular work load are left behind, my youngest son is in high school now, he can not keep up at the pace at which they teach.
we can not help him at home do you our education short falls and we can not afford for some one to teach him after school. He brings home Fs in three of his core classes. He does not want Fs but this is all he can get. in one class his last mid term score was "23". we arranged a meeting and they put him is several classes where the teacher just sits and reads to the class (is this teaching?) he is still failing his core class. I would rather take the money we spend in taxes and send him to privite school. Im sure there is a privite school out there that would take him in for the same amount that we spend per student per year, the government would only have to give us a coupon that would be good at any privite school and the government would have to pay for the coupon. (good luck getting that money, see the kentucy retirement system)

The government started a good thing with government school and i beleave they are still a value to our education system but our government schools do not fit every child.

Susan Weston said...

Richard,

Thank you for the e-mailed spread sheet. It let me check what I thought I saw in your graph before commenting on the substance.

Over 16 years, public school ACT composite rose 0.7 points. The scores held basically steady for from 1993 to 2002, and then made the entire 0.7 point improvement from 2002 to 2008.

Over the same 16 years, other-than-public-school scores rose 0.5 points, with a decline and then a recovery. 2007 was the first year that sector did better than 1993.

Anonymous said...

It should be noted that private schools do not have to educate students with various categories of learning disabilities.

Since the late 1970s and early 1980s,public schools have been required to serve all children ages 2-21 regardless of disability.

Richard Innes said...

Susan,

Your comments focus too much on the scores while ignoring the phenomenal changes in ACT participation that occurred in private schools between 1993 and 2008. I guess I need to more completely discuss that, which I will do in a new blog.

Also, I don't think it would be accurate to call the public school scores on ACT between 1993 and 2002 steady. There was a slow but consistent increase, followed by a slow but consistent decline during this period. Finally, can you tell me where you got the 0.5 point score change for private schools between 1993 and 2008? Is that a typo?

Anyway, look for more in a new blog item.

Susan Weston said...

Richard,

On the other-than-public improvement, it's a rounding issue.

If I just look at the table you sent, the composite score went from 21.8 to 22.2, which certainly looks like 0.4 change.

If you push it out to three decimal points, though, you see the composite going from 21.753 to 22.228, which means a 0.475 change.

I know this is a rounding error, but which version is the error?

Do you think it's better to call the improvement 0.4 or 0.5?

Susan Weston said...

Richard,

You also asked about public school results from 1993 to 2002. The data you sent me showed these ACT composites:

1993 19.9
1994 19.9
1995 19.9
1996 20.0
1997 20.0
1998 20.2
1999 20.0
2000 20.0
2001 20.0
2002 19.9

1998 is definitely the peak year there, but it also had a drop in the number of students tested. Looking at it on my own, I wasn't inclined to call it important.

But maybe I was wrong.

Do you think Kentucky public schools delivered a meaningful improvement in composite ACT scores between 1993 and 1998?

Richard Innes said...

Susan writes:

"Do you think Kentucky public schools delivered a meaningful improvement in composite ACT scores between 1993 and 1998?"

As I said, this was a slow, but consistent change in one direction.

A 0.3 point ACT change isn't particularly notable, of course. In fact, even a 0.7 point change isn't enough, considering how poorly prepared our students continue to be upon arrival at college.

However, the private school decline from 21.8 down to 20.2 is very notable. And, the private school rise from 1998 to 2008 is also notable. Two full points of change on ACT will probably wake anyone up.

Again, more on this later in a new blog.

Hempy said...

For a student to consistently fail to perform or not show any improvement would suggest that there may be other problems.

Has he been tested for any learning disabilities? There are a whole host of other conditions that can adversely affect a student's performance.

There are visual, hearing, dietary (too many soft drinks), hyperactivity, deficit disorders, drugs and alcohol use, and emotional-social problems that are often hard to detect.

If your child hasn't been tested, you ought to ask the district to test him. Unless your son is a behavior problem, the school itself is not likely to take the initiative and ask for testing.

Too often, educators not trained in learning problems, often dismiss such performance with statement such as, "He doesn't try." "He doesn't bring pencil and paper to class." "He won't bring his books." "he never turns in any assignments or his homework." "He just doesn't try."

These can all be correct observations, but they don't get at the problem.

Anonymous said...

Hello. And Bye.

Hempy said...

If we'd legalize all things hemp we wouldn't need all these pharmaceutical compounds with all their adverse side effects. Of course, that would be much to the chagrin of the pharmaceutical drug cartel.