Tuesday, March 10, 2009

Measurement of Growth (Models)

Only three short years ago, our nation's students were measured as either/ or: either they met the "proficiency" standard for academic achievement, or they didn't. Two states, Tennessee and North Carolina, began a pilot program measuring student growth from year to year. No longer were students measured against an arbitrary benchmark; instead they would be measured against themselves and their progress followed year-to-year.

In the 2006, a total of seven schools nationwide met No Child Left Behind's Adequate Yearly Progress standard due to counting student growth. As more states began measuring individual student growth as an alternative, that number grew, to 353 in 2007 and 1,571 last year.

As this trend continues, there has not been much study of what this means for accountability. Until now. Education Sector has just released a report from Charles Barone, director of federal policy for Democrats for Education Reform (and a blogger extraordinaire), looking at how the choices states make in implementing growth models impact accountability. It found some real downsides to growth models like Tennessee's:
  • By setting an interim goal short of proficiency, in a state judged by the U.S. Department of Education to have among the lowest standards of any state, it may be setting the bar so low as to evoke fairly small gains in student achievement.
  • While the “expected score” system estimates a student’s path to proficiency in three years, in fact, many students will not make it to proficient in three years or ever because of a statistical phenomenon known as Zeno’s paradox.
  • Finally, because this model relies on multiple regression analysis, one must be a statistician to understand it. Although complexity may be a necessary trade-off for more accuracy, there is a loss of simplicity and transparency for parents and the general public.
Zeno's paradox is troublesome but interesting to think about. Picture a loaf of bread. Cut it in half. Then cut it in half again. And one more time. The loaf of bread has been cut in half three times, but there's still bread to cut. In fact, you could, with a very sharp knife, cut the bread in half an infinite amount of times. That is Zeno's paradox: you can make progress towards something indefinitely without ever erasing the original. For a student short of meeting proficiency thresholds, they could theoretically make "progress" every year without ever completing the path to proficiency.

Go here to read the entire report.

2 comments:

Anonymous said...

No, that's not Zeno's Paradox. Zeno wrote about distance, which as far as Zeno knew, was infinitely divisible. Bread is not, because as soon as you're down below the molecular level, there's no bread any more.

Also, Zeno's Paradox for distance was resolved in the 19th century, or maybe earlier, by the theory of limits. Invoking it in this context doesn't make even metaphorical sense.

Unknown said...

Having spent a fair amount of time looking at this issue, I found it curious that Barone didn't bother to cite the plethora of research around the issues surrounding value-added models. Which is what the Tennessee "model" is based on. For those who have any interest in getting into the nitty gritty of this (which it looks like Barone doesn't want to do), The Wisconsin Center for Education Research hosted a conference last spring on this topic, and there's more than enough research in the papers to raise more questions than answer them. See http://www.wcer.wisc.edu/news/events/natConf_papers.php