Friday, June 20, 2008

Education, Citizenship and Need

I spent the middle of this week in Colorado at an Aspen Institute conference on higher education. It was great (the conference and Aspen both) and a couple of ideas really stuck with me.

One came from economist (and ES non-resident senior fellow) Tony Carnevale, who made the point that in America, work is the essential obligation of citizenship. Not (as is often said) voting--lots of people don't vote and there are few policies or social norms that penalize non-voting. Work, on the other hand--you're expected to work, and the consquences of not working are severe, more so (often much more so) than in other societies. And given that the ability to work productively is increasingly tied to education, preparation for work--education--is increasingly a non-negotiable obligation of citizenship too.

People understand this, which is why the percentage of beginning high schools students (and their parents) who expect to go to college now hovers near 90 percent, with similar numbers for students of differing race/ethnicity. As public opinion researcher Daniel Yankelovich noted, this puts consumers in an interesting relationship with higher education. They generally admire colleges and universities, for good reasons. But they also understand that they need higher education, because they can't go anywhere else for the training and credentials that open the door to economic opportunity. They feel like they have a right to be able to enroll in an affordable college that meets their needs. They're increasingly frustrated by the fact that college costs more and more--and they see those rising costs as stemming from the fact that higher education essentially has them over a barrel. This creates a danger of growing public resentment that could undermine the good will on which so much of higher education's elevated standing is based.

Yankelovich drew a parallel to research he conducted for AT&T back in the 70s, when it was Ma Bell. People would get enraged over problems with their phone service, he said, not because the problems were so terrible in an absolute sense, but because they couldn't not have phone service and they couldn't buy phone service from anyone else. They felt powerless, and people really don't like to feel powerless. While I grew up in the post-Bell breakup era, I've certainly had the same reaction to various cable television providers over the years--those %#%&*!s made me wait five hours to turn my service on and they still jack up prices by eight percent every year. In the grand scheme of things it didn't matter much--what's a few hours and a few bucks a month? But--particularly in the pre-satellite dish era--it was the fact that I had no choice that got my goat.

All of which makes me think that if higher education doesn't come to grips with the problem of runaway cost increases, a combination of growing public resentment and new organizations clamoring to get into the market will change the post-secondary landscape more quickly than some might imagine, in ways that are unforseeable and with consequences that society would do well to avoid.

Thursday, June 19, 2008

Don't Be Hating

NYTimes education reporter Sam Dillon has a nice profile of Teach for America founder Wendy Kopp and KIPP CEO Richard Barth, who lead the nation's two most successful and high-profile education entrepreneurial organization while also being married to one another.

Of course, no TFA article would be complete without the requisite disparaging quote from Stanford professor and Obama advisor Linda Darling-Hammond, who has elevated TFA haterism into something of a fine art over the years. TFA hating is an interesting phenomenon and worth exploring in more depth.

I think there are basically two issues at work here. The first is privilege. Teaching is very much a middle-class profession in this country. Teachers tend to be a lot like my aunt, who has lived in the same mid-sized southeastern Pennsylvania town for the last 30 years, raising three kids, marrying twice, and generally living a prosaic American life while making a career as a middle school math teacher. She's never been paid very much, but she takes a lot of pride in her work and enjoys respect in her community.

Many (though certainly not all) TFA corp members, by contrast, come from the privileged backgrounds inherent to Yale and other elite colleges where the organization likes to recruit. I think there's a sense among some that TFAers are parachuting into the teaching profession for a little while, grabbing a piece of moral authority, and then using it to further their already-privileged lives. A teacher like my aunt reading about state dinners for Prince Charles and limousines lined up outside the Waldorf-Astoria might wonder, not unreasonably, why it never occurred to all those rich and famous people to recognize or support her lifetime of service.

The second issue is professionalism. There is a robust strain of thought (of which Darling-Hammond is a leading proponent) which holds that teaching needs to be elevated into the ranks of respected, well-compensated professions like medicine and law. As NCATE president Art Wise recently said, "Professions normally have common programs of preparation and extended terms of practice. TFA does not fit the professional model of teaching; other professions do not assign novices primary responsibility. "

Here's the problem with this argument: most people—like me, for example—know virtually nothing about what it takes to be a doctor. My knowledge of diagnosis and treatment is pretty much limited to "if it's bleeding, make it stop." That's because I've had little exposure to the practice of medicine—I'm not related to or friends with any doctors, nurses, or other health professionals, and I've been lucky enough to avoid seeing very much doctoring from a treatment perspective.

Yet despite the fact that I've never been a teacher or taken an education class, I know a fair amount about teaching—far more than I know about medicine. Why? Because like everyone else who goes to school and then college, I've spent thousands upon thousand of hours in my life being taught. I've observed a range of teaching practitioners in action, sitting in their classrooms day after day for months at a time, unconsciously absorbing their methods, making judgments about which approaches work and which don't.

This is not to say that I wouldn't benefit from conscious exploration of pedagogical theory and explicit instruction in teaching methods. And I'd probably be better off without some of what I learned from my teachers--they weren't all good. But the fact that college graduates come directly from an educational milieu--combined with TFA's rigorous screening process and the tendency of elite colleges to graduate students with the kind of exceptional verbal abilities that research suggests are associated with classroom effectiveness--appears to be enough to make up for what TFA corp members lack in formal experience and training. That's why, on average, they're as effective or more effective than the traditionally-prepared teachers with whom they work.

Regular teachers, who won't be and shouldn't be displaced by the TFA model, deserve a lot more recognition and privilege than they receive. But unless the revolution is coming sooner than I think, we're stuck with class-based society that distributes privilege disproportionately, and all else equal I'd rather Ivy League graduates looked for their status signifiers in low-income classrooms on the Mississippi Delta than in hedge funds and investment banking houses on Wall Street. And while it may be frustrating to advocates for the professionalization agenda that TFA complicates their narrative, that's no excuse for wasting valuable time and energy trying to tear down a program that unambiguously makes the world a better place.

Update: Sherman Dorn turns in a lengthy post describing the above as a "slur against the apparently evil Linda Darling-Hammond." I think this stems from some confusion on Sherman's part as to the meaning of the word "hater" and thus "haterism." I'm using the word in the modern colloquial sense--see the Urban Dictionary here--which denotes a level of sustained, unreasonable animosity that's substantially milder than "hatred" in the traditional sense. 

Those tricky charts

The Fordham Foundation's report comparing high and low achievers under No Child Left Behind had me initially convinced. I wrote an entire blog post claiming that it showed the merits of NCLB, that the law could be credited for closing the achievement gap, and pushing for a future system of accountability that held schools responsible for the education of all children from all backgrounds with all abilities.

In my post I reiterated the gains published by Fordham. They say:

Low-achieving students made solid progress on the National Assessment of EducationalProgress (NAEP) from 2000 to 2007 (an accomplishment surely worth celebrating, even though these students are still far, far behind). Meanwhile, however, the progress of our top students has been modest at best.

Their charts showed me graphically what I had just read. While the chart above purported to present how low achieving students increased in the years following the enactment of the law, the one at left supposedly showed no major closing of the achievement gap before NCLB.

But these aren't the charts we should have seen. I took these same two Fordham charts and combined them below. The solid line represents where Fordham drew the cut-off for NCLB, the year 2000. But No Child wasn't signed into law until January 2002. The first NAEP tests measuring its true impacts could not have been until 2003, represented by the dotted line.

When we make this correction, the claims in the report do not seem to stand up as well. The lower tenth of performers made gains throughout the chart, but especially from 2000 to 2003, where they gained 13 points to their high achieving peers' six. Notably, this accounts for almost all the gain claimed in the Fordham report.
















Fordham's argument, that we've focused too heavily on equity issues while neglecting excellence, could have been made without the misdirection. Along with this analysis of NAEP data, the report also featured a teacher survey. It supported their earlier conclusions, and could have stood on its own merits. Instead, they published both together, and the data doesn't show exactly what they claim. All we're left with, in the end, is debating tough choices.

Update: Mike Petrilli responds, citing the difficulties in defining the "era of NCLB." This is indeed a tough task. It's even tougher to put statements like these:
The fairest approach is to point out the large gains in NAEP scores in the period around 1998-2003 and acknowledge that NCLB’s association with these gains is unknown.
into charts.

My broader point is that if you compare scores across a longer spectrum, from 1992 in reading and 1990 in math (as Fordham did), the lowest ten percent of achievers narrowed the achievement gap from their top ten percent peers by 9, 2, 1, and 4 points, respectively, for 4th and 8th grade math and 4th and 8th grade reading. Those changes are in scale scores, not percentages. That's not exactly a sign of crisis.

Wednesday, June 18, 2008

Disruptive Assessment

The new book, Disrupting Class, written by Clayton Christensen, author of Silicon Valley bible The Innovator's Dilemma, is getting a lot of attention for its projections that half of all high school courses could be online by 2019. (Education Next has an article summarizing the book.) Beyond this headline, though, is a much more fundamental and interesting prediction.

Christensen and his co-authors Michael Horn and Curtis Johnson suggest a two-step process for the disruptive change they foresee. The first step is the current version of online learning, in which they highlight companies like Apex Learning and public providers such as Florida Virtual School. But, the eventual focus of the disruptive cycle is a wholesale change to "student centric learning," highly personalized learning facilitated by powerful cognitive tutors and technology applications. In the book, the authors are wise to make the not necessarily obvious connection with assessment.

With the change to student-centric learning, assessment--the art and science of testing children to determine what they have learned--can and should change, as well. Student-centric learning should, over time, obviate the need for examinations as we have known them. Alternative means of comparison, when necessary, will emerge.

They don't use these terms, but essentially what they describe is a very powerful, technology-aided formative assessment and instructional approach, under girded by well-defined cognitive models and learning progressions. It's the place where digital media offers the most opportunity and the truly disruptive approach.

Assessment offers a critical connection in Christensen's disruptive cycle between the first phase, online learning, and the second phase, student-centric learning. Since online learning already takes place in a digital environment, and, perhaps more importantly, it is not constrained to a particular pace or time, it provides an immense opportunity to embed technology-enabled formative assessment throughout instruction. New types of assessment tools, simulations, and games that capture a wide variety of descriptive formative data are many times a distraction or add-on in the traditional classroom environment. But, they can and should play a central and critical role in the online learning environment.

Disruptive assessment, driven by continued research and applications of cognitive science and technology, is the big opportunity.

Tuesday, June 17, 2008

Dueling Manifestos

The extremes in school-reform debates always seem to conspire against the middle, making change a lot tougher to achieve.

It happened again last week when two coalitions of pro-public-school educators and policymakers published reform manifestos. We’re not likely to see really significant, sustained improvement in the educational achievement of disadvantaged students unless we embrace the core recommendations of both documents. But from the way the two camps have framed their arguments—and caricatured each other’s—one would think that reform is a distinctly either-or proposition.

The first statement—A Broader, Bolder Approach to Education—is sponsored by the Economic Policy Institute, a Washington, DC, think tank. EPI president Lawrence Mishel and researcher Richard Rothstein enlisted Duke economist Helen Ladd, New York University professor Pedro Noguera, and Tom Payzant, a former Boston schools superintendent and U.S. assistant secretary of education, to help make an argument that Rothstein has made for several years—that “attempting to rely on school improvement alone to raise the achievement of disadvantaged children” is a mistake. And that “to be fully effective,” school reform “must be complemented by a broader definition of schooling and by improvements in the social and economic circumstances of disadvantaged youth.”

The key words here are “to be fully effective.” There’s ample evidence that millions of students from disadvantaged families suffer in the nation’s classrooms because they come to school ill-fed, ill-housed, lacking adequate heath care, and without sufficient exposure to language. And a wide range of notable educators, civil rights advocates and policymakers signed the EPI statement for just that reason, from NAACP Chairman Julian Bond to African American scholars such as Glen Lowry and William Julius Williams, present and past urban school superintendents Payzant, Arne Duncan, Rudy Crew, and Beverly Hall, and former federal education officials Payzant, and Diane Ravitch, and Marshall Smith (who sits on Education Sector’s board).

The problem is that many public educators have used the disadvantages that poverty inflicts on many students as an excuse for failing to even try to educate them. It’s an impossible task, they argue, we can’t overcome the baggage that our students bring with them to school. I’ve heard that lame lament over and over in hundreds of public schools in every corner of America. Educators declare that schools can’t make any meaningful difference in the educational lives of disadvantaged students absent the elimination of poverty’s debilitating effects.

They’re wrong. Schools can make a significant difference, even if they can’t be “fully effective” in overcoming the consequences of poverty.

And so advocates of educational accountability, those who say that the only way to give disadvantaged students a chance at a decent education is to hold excuse-inclined educators responsible for their students’ achievement, brought us state testing systems and, in 2002, the federal No Child Left Behind Act. This camp is behind the second of last week’s manifestos, which was organized by NYC Chancellor Joel Klein and Al Sharpton. Not surprisingly, past or present officials of the organizations that pushed Congress hardest for NCLB accountability, including The Education Trust, the Citizens Commission on Civil Rights, and the Progressive Policy Institute (former PPI education director Andy Rotherham now co-directs Education Sector with me), were signers of Klein’s manifesto.

A fundamental failing of NCLB, however, is that it doesn’t account for the reality that some schools are dealt much tougher educational hands than others, by virtue of the fact that their students suffer from the ravages of poverty that the Broader, Bolder manifesto addresses. What we need to do is find ways to give schools credit for successfully improving the educational performance of the kids they have, by using so-called value-added measures of student performance, and by capturing more than just how well schools teach basic reading and math skills. Both are heavy lifts, but they would liberate schools to expand the scope of their mission and encourage them to work hard regardless of their students’ backgrounds.

So both camps are right, and wrong:

Yes, we should find ways to reduce the effects of poverty on students. Doing so will allow them to achieve at higher levels. But no, we shouldn’t assume that schools can’t make a difference on their own.

Yes, we need to hold schools and teachers accountable for their performance. Too many of them simply haven’t embraced high expectations on their own. But no, we shouldn’t pretend that poverty has no impact on students. No accountability system can work unless it is credible, and NCLB, as currently crafted, is not.

Monday, June 16, 2008

Will Data Save D.C. Vouchers?

The Year 2 evaluation of the D.C. voucher program was released today. Given the current debate over whether the program should be funded for another year, I'm guessing this study will see a lot of spin in the next few weeks, since there's a little something for everyone.

Those who'd like to end the program can point out that the results were, all-in-all, underwhelming, but supporters of the voucher program can point to the positive results among certain subgroups. Most notably, students from the first cohort who used the voucher scored significantly higher in reading - supporters might use this to convince lawmakers to hold out for another year or two in order to see if the effects continue for subsequent cohorts. But will a few positive results be enough to save D.C. vouchers?

Update: As Chad points out below, voucher supporters are already grabbing on to the subgroup analysis to support the voucher program. And I've already gotten a couple of news releases with headlines touting "academic gains" and "early successes". For some reason, though, none of these releases mention the fact that the overall analysis of reading results did reach the 91 percent cutoff for statistical significance, but just didn't quite make it to 95 percent. Maybe that's just too stats-nerdy for a press release.

False discoveries: Interpret with caution

Despite my best efforts to persuade, the folks at Fordham's Flypaper blog are again trumpeting the results of the DC voucher program. While they admit the study found no significant difference in academic gains between students who received a voucher and those that did not, they cherry-pick this quote:
However, being offered a scholarship may have improved reading test scores among three subgroups of relatively more advantaged students: those who had not attended a School in Need of Improvement (SINI) school when they applied to the program, those who had relatively higher pre-program academic performance, and those who applied in the first year of program implementation.
despite the Institute of Education Sciences explicitly warning these findings "were no longer statistically significant when subjected to a reliability test" and "the results may be 'false discoveries' and should therefore be used and interpreted with caution."

These quotes are not exactly buried; you can find them in the Executive Summary.

A New Kind of Competition

Last week the University of Phoenix, the biggest private for-profit university in the nation, released its first "Academic Annual Report," which compares the scores of Phoenix seniors to Phoenix freshmen on the Measure of Academic Proficiency and Progress (MAPP) test, which is administered by ETS, the same folks who administer the SAT, AP, and GRE. Phoenix says their freshmen enter with lower scores than typical college students but leave at about the same level, evidence (they contend) that a Phoenix education is sound.

Now, one can dispute the methodology here, and point to the fact that MAPP can only capture a sliver of what colleges aspire to teach their students in the best of circumstance, and these would be fair points to make. But they also really miss the forest for the trees. Of far greater import is that here we have an individual university actually voluntarily providing public evidence of how much its students learn while they're in college. This rarely happens. And the fact that a for-profit university is leading the charge is wholly unsurprising.

The public and non-profit university sector has traditionally held an iron grip on an incredibly good market to have in the information age: credentialing higher learning. Elite institutions in particular are essentially selling branded intellectual property; just as corporations like Microsoft and Eli Lilly have made vast fortunes in this business, so too has it worked out exceedingly well for the likes of Stanford and Duke. Given significant financial and regulatory barriers to entry, along with burgeoning demand driven by demographics, public policy, and changing economic conditions, traditional colleges have been able to stay relatively inefficient and complacent in the way they provide educational services while transferring education-generated revenues internally to pay for other things like faculty scholarship, sports teams, etc.

For-profits like Phoenix have historically built their businesses by going after the parts of the market that traditional colleges were either unable or unwilling to serve: adult students, part-timers, occupational training, etc. But it was only a matter of time before they started to turn toward the huge profit center that is the traditional four-year college student. That they've made inroads here despite competing on un-level financial playing field (lacking the direct and indirect tax-related public subsidies non-profits enjoy) just goes to show how much financial slack is built into the cost of educating a traditional undergraduate.

So it stands to reason that for-profits would be much more aggressive in marketing their learning results, in competing for students on educational terms, rather than via brochures featuring five demographically balanced students sitting in a circle on a leaf-strewn academic quad. In the end, education is all the for-profits have to sell. Some traditional colleges and universities see the handwriting on the wall and have pushed to create new venues for self-reporting data, e.g. the Voluntary System of Accountability sponsored by public and land-grant institutions. The Phoenix report just emphasizes that the clock is ticking: either traditional colleges and universities will determine how institutional success in teaching students will be measured and defined, or somebody else will.

Everyone's favorite sound bite

Eduwonkette's post this morning on the long-term effects of teachers needs explanation. In attempting to report on a new study, the crux of her argument is thus:
It's everyone's favorite sound bite: good teachers alone can close racial and socioeconomic achievement gaps. But if the entire teacher effect doesn't persist from year-to-year - that is, a student only retains some fraction of the learning advantage they get from having a highly effective teacher - these claims simply don't hold up.
There are two problems with this argument. One, the entire teacher effect need not persist for it to be real, and two, the evidence shows time and again that placing a student in the hands of an excellent teacher for even one year has lasting impacts.

Let's look at the study itself. Eduwonkette repeats its finding that, "only about one-fifth of the test score gain from a high value-added teacher remains after a single year." If we dig a little deeper, we see that the researchers found a one-year long-term learning coefficient of .66, with the teacher the student had a year beforehand contributing a full one-third of this effect.

If we look two years out--that is, the student had been under the tutelage of two different educators for two additional years of schooling--that original teacher still contributed about one-fourth of the student's long-term learning gains. These gains attributable to teachers come after controlling for incoming student achievement scores, gender, race, age, income, and learner characteristics (disability and limited English proficiency). These findings in no way challenge previous studies indicating teacher effects accumulate over time.

Update: Eduwonkette responds, but she clearly did not read the study carefully. Unlike other value-added studies that look at teacher effects over time with multiple teachers, this one looked at only the learning gains attributable to one teacher in one year. Then, it asked the question of how much, for example, we can attribute a 5th or 6th grader's math and reading scores with his or her 4th grade teacher. In finding that the 4th grade teacher alone accounted for one third and one quarter, respectively, of a student's achievement scores one and two years later, the study gave us more evidence of lasting teacher effects. True, they diminish slightly over time, but without controlling for future teachers, these findings in no way dampen the lessons from previous studies.