There’s a great deal of attention being given to using test scores to measure teacher performance these days, recent announcements from the Gates foundation ensure this will be high on the national agenda in coming years. But recent studies show that the value-added measures contain significant amount of error. Which raises questions: how can imperfect measures be incorporated into high-stakes decisions like teacher pay? How good is good enough?
Reformers have been waiting for longitudinal data systems to be implemented to provide value-added data to support improvements to compensation and retention decisions. The data is now there in several states, but the quality of the new information may not be as good as many of us had hoped. Just before Thanksgiving, two new studies were released that show the lack of stability of value-added measures of teacher effectiveness over time. The first by Dan Goldhaber looks at North Carolina data to see if pre-tenure teacher effectiveness (measured by the value-added gain of a teacher’s students) is a good predictor of their effectiveness post-tenure (here). The study showed that a teacher ranked in the bottom quintile of teacher effectiveness has a 32 percent chance of being in the bottom quintile post-tenure. While this is better than random (random would be around 20 percent), it is not much better than random. At the same time, 11 percent of the poor performers pre-tenure (bottom quintile) end up being in the top quintile post-tenure. The measure is a little more consistent at identifying top performers – 46 percent of top performers are top performers post-tenure (see Table 1 for all measures of pre and post value-added effectiveness). Goldhaber also looked at using the first 3-years of data to predict outcomes, and the predictive power does not change much.
Table_1a.pdf
A second paper by Tim Sass shows similar results from California and Florida studies. (here)
This paper focuses on whether value added measures of teacher quality are stable enough to use for compensation decisions. It shows similar results as the Goldhaber study over time. The lack of stability over time may not be surprising given the group of students a teacher gets each year is random. The data can not measure whether a teacher had a particularly disruptive class in the first year, and a better group of students the next. So, the randomness of classroom make-up may have a lot to do with these results. The Sass study shows that while measurable, student characteristics explain some of the differences in value-added effectiveness, but most of the differences across time are unexplained (See Table 2 for complete effectiveness measures).
figure_2.pdf
The part of the Sass study that caused me the greatest concern was how inconsistent these value-added measures are across tests. Students in Florida take two tests annually. They take a low stakes norm-referenced test and a high stakes standard-aligned test. Sass looks at how stable these value-added measures are across these two tests. So for this comparison, the random draw of students is the same for any given teacher. While these results look a little more stable (43% of bottom quintile teachers remain in the bottom quintile on the other exam), they are not as stable as you would hope. If just switching the exam moves 5 percent of teachers from the bottom of the distribution to the top, it would likely make teachers question the validity of the measure reflecting true effectiveness.
These papers and a few others suggest that value-added measures are not very consistent over time, and may not be the panacea for which some reformer have been hoping.
How Good is Good Enough? Now you would think that the bar for improving teacher compensation and tenure decisions would be pretty low. The current compensation structure is based almost exclusively on a teacher’s years of experience and college credits/advance degrees. Advance degrees have been consistently shown to have no impact of teacher effectiveness. For experience, teachers appear to improve their craft slightly over the first two to three years, but additional experience does not seem to have any impact. Clearly moving to value-added compensation could more accurately reward effective teachers than the current system. However, if a compensation system were based partially on these value added measure, I think that teachers would perceive the outcomes above as too arbitrary. It also makes me think that principals and mentor teachers could do a better job of predicting effectiveness than last year’s test results. (See Brian Jacobs on this question – principals seem to do pretty well on the identifying teachers at the top and bottom of the distribution, but their measurement is less predictive than prior year’s value-added (here). Of course this is not an either or choice. Can principals armed with value-added test results do an even better job than either one alone? What about a combination of principal evaluations, mentor teacher evaluations and value-added? Are there more rigorous evaluation methods like those of the Teacher Advancement Program or others better predictors than the value-added measures? (See Ed Sector Report on Teacher Evaluation here) As with all good research, it leads to more research. And with Gates interested in these topics, more research is likely to be on its way.
Friday, December 05, 2008
Thursday, December 04, 2008
Two Steps Back
Two pieces of bad news today for those working to build a quality supply of public schools:
First, the 1st District Court of Appeals in Florida ruled that a 2006 law creating the Florida Schools of Excellence Commission conflicts with the Florida state constitution. According to the court, the Commission, which would be an independent, statewide office established to approve and support charter schools, was unconstitutional because the Florida constitution limits oversight of charter schools to local school districts.
Well, it looks like it's time to change the Florida constitution. A growing body of research shows that having, in addition to local school districts, one or more professional authorizers whose sole focus is approving and overseeing charter schools makes for a healthier and higher quality charter school sector. In fact, a 2006 ES report on Florida charter schools stated that the proposed Schools of Excellence Commission, "will likely reduce the number of appeals to the State Board and relieve unwilling sponsors of their chartering responsibilities while significantly improving the quality and transparency of authorizing across the state."
And the second piece of bad news is the bailout bill passed in the House Education Committee in Michigan which allowed Detroit to keep it's "first class school district" status (reducing the enrollment threshold from 100,000 students to 60,000 students) and thereby limiting the opening of new charter schools in the district. As we've reported, maintenance is required among Michigan's charter schools, but a blunt limit on opening new charter schools does nothing to improve the quality of charter schools or the quality of Detroit Public Schools.
First, the 1st District Court of Appeals in Florida ruled that a 2006 law creating the Florida Schools of Excellence Commission conflicts with the Florida state constitution. According to the court, the Commission, which would be an independent, statewide office established to approve and support charter schools, was unconstitutional because the Florida constitution limits oversight of charter schools to local school districts.
Well, it looks like it's time to change the Florida constitution. A growing body of research shows that having, in addition to local school districts, one or more professional authorizers whose sole focus is approving and overseeing charter schools makes for a healthier and higher quality charter school sector. In fact, a 2006 ES report on Florida charter schools stated that the proposed Schools of Excellence Commission, "will likely reduce the number of appeals to the State Board and relieve unwilling sponsors of their chartering responsibilities while significantly improving the quality and transparency of authorizing across the state."
And the second piece of bad news is the bailout bill passed in the House Education Committee in Michigan which allowed Detroit to keep it's "first class school district" status (reducing the enrollment threshold from 100,000 students to 60,000 students) and thereby limiting the opening of new charter schools in the district. As we've reported, maintenance is required among Michigan's charter schools, but a blunt limit on opening new charter schools does nothing to improve the quality of charter schools or the quality of Detroit Public Schools.
Darling-Hammond Unbound
Score one for the KAPPAN magazine. The edu-magazine has a very timely piece on school accountability in its just-mailed December issue by top Obama policy advisor Linda Darling-Hammond. Anonymous "reformers," some of whom also have ties to the Obama administration-in-waiting, have been taking shots at the Stanford professor during the transition, in part because she has been tough on the quality of state testing under NCLB. They have declared her to be "anti-accountability."
The KAPPAN piece provides a valuable window on her thinking. She's indeed not a fan of NCLB-brand multiple-choice testing. "NCLB reinforced using test-based accountability to raise achievement, yet the US has fallen further behind on international assessments of student learing since the law was passed in 2001," she declares at the top of the article.
Darling-Hammond has spent a lot of time studying the teaching and testing systems of high achieving industrialized countries and likes them better than ours. Among other things, she says, they teach fewer topics in greater depth; focus more on reasoning skills and applications of knowledge rather than on coverage of content; and rely heavily on open-ended questions "that require students to analyze, apply knowledge, and write extensively," in contrast to US tests that "rely primarily on multiple-choice items that evalute recall and recognition of discreet facts." She's right about that.
Darling-Hammond points approvingly to a "growing emphasis" in high-performing countries on "project-based, inquiry-oriented learning" that has led "to an increasing prominence for school-based tasks, which include research projects, science investigations, development of products and reports or presentations about these efforts"--so-called performance tests. The bulk of the article (written with co-author Laura McClosky) describes approvingly locally administered peformance assessment in countries ranging from Finland to Australia, Hong Kong, Sweden, and the UK.
There's little doubt that LDH would push to introduce these kinds of assessments into US public education if she were to have a senior role in the Obama administration: We need, she writes, "a new vision of assessment" in American education. To the extent that testing drives teaching, that would be a good thing.
The question is whether she would push to incorporate performance tests into NCLB-style statewide testing systems, or try to move testing down to the local level.
In writing that "the policy community has little understanding about how systems of assessment for learning might be constructed and managed at scale," she is acknowledging the challenges of using performance testing under the NCLB system, cost and scoring reliability chief among them. One thing she might do if she goes to work for Obama would be to have the federal government sponsor an effort to address the difficulties of doing performance testing at scale, and give states financial incentives for using such tests. Given the growing consensus that well-crafted performance assessments would represent a big step towards teaching students the higher-order thinking skills that they need today (Darling-Hammond points out that US students score lower on problem-solving that their international counterparts), this would be a smart investment--and a refreshing change from the Bush administration's hear-no-evil, see-no-evil stance on test quality.
Another possible policy solution, she implies elsewhere in the article, would be to include performance-based local assessments into "overall examination scoring systems." That's what several of the countries she has studied do.
The big question for supporters of NCLB's statewide standardized testing systems is whether performance assessments would be used for holding educators accountable for student achievement. Much edu-blood was spilled over that question a year ago, when Rep. George Miller included the notion of local assessments in a draft NCLB reauthorization bill.The combatants eventually withdrew from the field and the Miller draft was decommissioned.
But it's clear that Darling-Hammond is ambivalent about using performance testing to hold educators accountable for student achievment. She notes that the countries she has studied "do not use their examination systems to rank or punish schools or to deny diplomas to students." Finland, she writes, "has no external standardized tests to rank students or schools." Instead, she writes approvingly, the testing systems in Finland and other countries are closely linked to efforts to develop teachers' ability to teach higher-level skills to their students; they are part of the countries' human capital strategies.
So, if Barack Obama gives Linda Darling Hammond a major role in his administration, we're going to have a big policy debate over testing in American education and whether we should move beyond NCLB accountability to something potentially very different. Such a debate wouldn't be a bad thing.
The KAPPAN piece provides a valuable window on her thinking. She's indeed not a fan of NCLB-brand multiple-choice testing. "NCLB reinforced using test-based accountability to raise achievement, yet the US has fallen further behind on international assessments of student learing since the law was passed in 2001," she declares at the top of the article.
Darling-Hammond has spent a lot of time studying the teaching and testing systems of high achieving industrialized countries and likes them better than ours. Among other things, she says, they teach fewer topics in greater depth; focus more on reasoning skills and applications of knowledge rather than on coverage of content; and rely heavily on open-ended questions "that require students to analyze, apply knowledge, and write extensively," in contrast to US tests that "rely primarily on multiple-choice items that evalute recall and recognition of discreet facts." She's right about that.
Darling-Hammond points approvingly to a "growing emphasis" in high-performing countries on "project-based, inquiry-oriented learning" that has led "to an increasing prominence for school-based tasks, which include research projects, science investigations, development of products and reports or presentations about these efforts"--so-called performance tests. The bulk of the article (written with co-author Laura McClosky) describes approvingly locally administered peformance assessment in countries ranging from Finland to Australia, Hong Kong, Sweden, and the UK.
There's little doubt that LDH would push to introduce these kinds of assessments into US public education if she were to have a senior role in the Obama administration: We need, she writes, "a new vision of assessment" in American education. To the extent that testing drives teaching, that would be a good thing.
The question is whether she would push to incorporate performance tests into NCLB-style statewide testing systems, or try to move testing down to the local level.
In writing that "the policy community has little understanding about how systems of assessment for learning might be constructed and managed at scale," she is acknowledging the challenges of using performance testing under the NCLB system, cost and scoring reliability chief among them. One thing she might do if she goes to work for Obama would be to have the federal government sponsor an effort to address the difficulties of doing performance testing at scale, and give states financial incentives for using such tests. Given the growing consensus that well-crafted performance assessments would represent a big step towards teaching students the higher-order thinking skills that they need today (Darling-Hammond points out that US students score lower on problem-solving that their international counterparts), this would be a smart investment--and a refreshing change from the Bush administration's hear-no-evil, see-no-evil stance on test quality.
Another possible policy solution, she implies elsewhere in the article, would be to include performance-based local assessments into "overall examination scoring systems." That's what several of the countries she has studied do.
The big question for supporters of NCLB's statewide standardized testing systems is whether performance assessments would be used for holding educators accountable for student achievement. Much edu-blood was spilled over that question a year ago, when Rep. George Miller included the notion of local assessments in a draft NCLB reauthorization bill.The combatants eventually withdrew from the field and the Miller draft was decommissioned.
But it's clear that Darling-Hammond is ambivalent about using performance testing to hold educators accountable for student achievment. She notes that the countries she has studied "do not use their examination systems to rank or punish schools or to deny diplomas to students." Finland, she writes, "has no external standardized tests to rank students or schools." Instead, she writes approvingly, the testing systems in Finland and other countries are closely linked to efforts to develop teachers' ability to teach higher-level skills to their students; they are part of the countries' human capital strategies.
So, if Barack Obama gives Linda Darling Hammond a major role in his administration, we're going to have a big policy debate over testing in American education and whether we should move beyond NCLB accountability to something potentially very different. Such a debate wouldn't be a bad thing.
Harvard's Endowment Falls to $29 Billion
Even after large stock market losses, if Harvard paid out five percent of its endowment--a requirement for all private foundations except those of colleges and universities--it would increase the school's budget by $214 million.
They've suffered a large financial loss on paper, but so have the rest of us, and lawmakers shouldn't let the economic downturn curtail efforts for endowment sanity.
They've suffered a large financial loss on paper, but so have the rest of us, and lawmakers shouldn't let the economic downturn curtail efforts for endowment sanity.
The Rachel Maddow Show
I'll be on Rachel Maddow's Air America radio show today at 1:15 6:00PM, talking about why college keeps getting more expensive. Rest assured, state legislatures will get their share of blame. But I'll also be pointing the finger at people like SUNY-Buffalo president John Simpson, who apparently sees the current economic crisis as the perfect opportunity to raise student tuition in order to fund a grand agenda of local economic development and institutional status-promotion. “It’s easier to push a conversation about this kind of substantive change today than it was a year or two ago, because the world wasn’t in such crisis,” Simpson said earlier this week. Call it the "shock doctrine" theory of making college less affordable.
Some might say that SUNY would still be relatively cheap even if Simpson's plan to raise tuition by 63 percent over the next decade were implemented. But that depends on the student. A new report from the National Center for Public Policy and Higher Education includes the following statistics about how the net cost of attending a four-year public university has changed over time as a percentage of the median income of families in the lowest and highest income quintiles.
Lowest quintile 1999 - 00 : 39%
Lowest quintile 2007 - 08 : 55%
Highest quintile 1999 - 00: 7%
Highest quintile 2007 - 08: 9%
Over the last eight years--and for quite a while before that--the rich have gotten much richer in America at the same time that college has gotten much more expensive. The two trends roughly cancel out. And since people in the highest quintile run things, they don't really see a huge problem. For low-income families, by contrast, college has always been a stretch, even after taking into account higher levels of financial aid. Middle- and upper-income families will probably be able to absorb tuition hikes. Low-income students at the margin, by contrast, will be increasingly priced out of the four-year sector, or have their studies compromised by the need to work, or be shouldered with unmanageable debt. I'm not saying there's never cause to raise tuition, but it would d be nice if students actually got something back in return--better support services and more well-paid instructors, for example. Instead they're being asked to pay for research and other things that help everyone but those paying the bills.
Tuesday, December 02, 2008
Oxygen
A co-worker and I were discussing today the oft-repeated education reformer line that schools should be for students and not for adults. See Joel Klein use a version here and Michelle Rhee's iterations here, here, here, or here. It's a good line, if for nothing else than it puts traditional powers in education policy (read: teachers unions) on the defensive as if they do not consider student concerns, and it does it without even naming them.
My co-worker and I are talking about this trend, and she suggested a metaphor that might contrast the line. Whenever you board a plane, she points out, they go through a long safety ritual. "Please note your nearest exit rows" and "fasten your belt by inserting the buckle and pulling tight on the remaining cord." They also insist that, in the case of an emergency, adults secure their own oxygen masks before attending to their children. Despite the best instincts of parents, this policy actually makes some sense. There's no point in having a bunch of adults trying to help out their children first and fainting in the process. Better to secure their own safety in order to be in a position to help those who need it.
It's an extreme metaphor to be sure, but it actually makes some sense in the context to long-struggling urban education systems. They're bad, they've been bad, and they're crashing for whole segments of the population. But it isn't just the students that need help--district finance, curricula, infrastructure, technology, etc. are all suffering--and adults who try to rush in without fixing some of these problems first will just faint and flounder. They'll have no air.
I'm generally sympathetic to what Rhee is trying to do: she's going after the adults in the system who have long settled for complacency and demographics to explain why DC's public schools have been so bad. But in an effort to test the all-publicity-is-good-publicity theory, she's lobbing fireballs like this one, from last week's Time piece:
Kevin doesn't like the magazine cover's title ("How to Fix America's Schools"). But it's not just the title. The cover itself is Rhee looking stern in a classroom, dressed in black, holding a broom, suggesting she'll sweep away problems. Quotes like the one above and the cover photo--two things Rhee had complete control over--are the things that test the publicity theory. Like the plane crash metaphor,"interests of children" advocates need to be careful how much they say and do, or else they may find they're lacking air.
My co-worker and I are talking about this trend, and she suggested a metaphor that might contrast the line. Whenever you board a plane, she points out, they go through a long safety ritual. "Please note your nearest exit rows" and "fasten your belt by inserting the buckle and pulling tight on the remaining cord." They also insist that, in the case of an emergency, adults secure their own oxygen masks before attending to their children. Despite the best instincts of parents, this policy actually makes some sense. There's no point in having a bunch of adults trying to help out their children first and fainting in the process. Better to secure their own safety in order to be in a position to help those who need it.
It's an extreme metaphor to be sure, but it actually makes some sense in the context to long-struggling urban education systems. They're bad, they've been bad, and they're crashing for whole segments of the population. But it isn't just the students that need help--district finance, curricula, infrastructure, technology, etc. are all suffering--and adults who try to rush in without fixing some of these problems first will just faint and flounder. They'll have no air.
I'm generally sympathetic to what Rhee is trying to do: she's going after the adults in the system who have long settled for complacency and demographics to explain why DC's public schools have been so bad. But in an effort to test the all-publicity-is-good-publicity theory, she's lobbing fireballs like this one, from last week's Time piece:
Rhee is, as a rule, far nicer to students than to most adults. In many private encounters with officials, bureaucrats and even fundraisers--who have committed millions of dollars to help her reform the schools--she doesn't smile or nod or do any of the things most people do to put others at ease. She reads her BlackBerry when people talk to her. I have seen her walk out of small meetings held for her benefit without a word of explanation. She says things most superintendents would not. "The thing that kills me about education is that it's so touchy-feely," she tells me one afternoon in her office. Then she raises her chin and does what I come to recognize as her standard imitation of people she doesn't respect. Sometimes she uses this voice to imitate teachers; other times, politicians or parents. Never students. "People say, 'Well, you know, test scores don't take into account creativity and the love of learning,'" she says with a drippy, grating voice, lowering her eyelids halfway. Then she snaps back to herself. "I'm like, 'You know what? I don't give a crap.' Don't get me wrong. Creativity is good and whatever. But if the children don't know how to read, I don't care how creative you are. You're not doing your job."
Kevin doesn't like the magazine cover's title ("How to Fix America's Schools"). But it's not just the title. The cover itself is Rhee looking stern in a classroom, dressed in black, holding a broom, suggesting she'll sweep away problems. Quotes like the one above and the cover photo--two things Rhee had complete control over--are the things that test the publicity theory. Like the plane crash metaphor,"interests of children" advocates need to be careful how much they say and do, or else they may find they're lacking air.
Competition on Quality, Service, and Price
A couple of weeks ago I was at a meeting where a higher education spokesperson flat-out stated there was no market demand for student learning data. His group had done focus groups, he said, and it just wasn't as high on their list as other things. His point would be fair, even if true, if there was real, meaningful data available on student outcomes in higher education. Instead, his argument is a chicke
n-and-egg conundrum. We have no meaningful data, so students and parents don't request it. Students and parents don't request it, so colleges don't provide it.
Luckily, consumer preferences can change. Automobile companies told Ralph Nader that safety didn't sell. He shamed them into understanding it can, and now car commercials are as much about side-impact safety beams as they are about horsepower. The automotive industry wasn't about fuel economy when gas prices were cheap, but all of a sudden we're seeing advertisements extolling a vehicle's miles per gallon. Ford went to Congress today hat in hand promising to develop electric vehicles and sell the Hummer brand.
All this is to say that we don't always have to accept $6 million buildings only for tutoring athletes or $55 million dorms complete with Coldstone Creamery, 7/11, state-of-the-art gym (pictured), grocery delivery, room cleaning, and laundry service as the sole basis on which colleges compete for students. They could, you know, compete on quality, service, and price.
For more on this topic and many others, listen to today's Education Sector event, "Is Technology the Answer to Rising College Costs," below:
n-and-egg conundrum. We have no meaningful data, so students and parents don't request it. Students and parents don't request it, so colleges don't provide it.Luckily, consumer preferences can change. Automobile companies told Ralph Nader that safety didn't sell. He shamed them into understanding it can, and now car commercials are as much about side-impact safety beams as they are about horsepower. The automotive industry wasn't about fuel economy when gas prices were cheap, but all of a sudden we're seeing advertisements extolling a vehicle's miles per gallon. Ford went to Congress today hat in hand promising to develop electric vehicles and sell the Hummer brand.
All this is to say that we don't always have to accept $6 million buildings only for tutoring athletes or $55 million dorms complete with Coldstone Creamery, 7/11, state-of-the-art gym (pictured), grocery delivery, room cleaning, and laundry service as the sole basis on which colleges compete for students. They could, you know, compete on quality, service, and price.For more on this topic and many others, listen to today's Education Sector event, "Is Technology the Answer to Rising College Costs," below:
Generalizability
The Michelle Rhee story continues to percolate ever-upward through layers of media, landing on the cover of Time this week. While there are many sound policy-based reasons for supporting her reform efforts, I have to admit one of the things I like is that she talks the way I talk. Not just about education, but in general. Which isn't surprising; we're the same age and both graduated from upstate New York universities in the same year, 1992. (Mine, SUNY-Binghamton, was where people tended to enroll when they couldn't get into hers, Cornell). To wit: "What I'm finding is that our principals are ridiculously--like ridiculously--conflict-averse," Rhee says. When I was in high school one English teacher went on at length about how the ubiquitous improper use of the word "like" as a means of emphasis was a clear sign of the linguistic apocalypse. Now famous Ivy League-educated Time-cover-gracing people talk this way. (Although there's still some loss in translation--I suspect the second "ridiculously" should have been italicized.)
The story is well-written and worth reading. My biggest qualm--and it's amazing how often this happens--is on the magazine cover. Inside, the piece is titled "Rhee tackles classrooom challenge," which is fair enough. But the cover title is "How to Fix America's Schools." And that's wrong. It would be a mistake to over-generalize about the lessons of DCPS, which is (thankfully) unusual. Most school districts haven't been systematically degraded by three decades of often corrupt one-party rule. Most districts don't employ significant numbers of truly incompetent teachers. Most districts are not unequivocally the worst in the nation when compared to similar districts. Most districts get much less funding per student. Most high-poverty districts aren't funded at levels similar to the surrounding wealthier suburbs. And so on.
The educational challenges in DC are unusual and, compared to most districts, extreme. The needed changes are of commensurate severity. Seeing DC as the definitive proving ground for larger questions about tenure, management style, etc. is not going to serve anyone's interests in the long run. The issues themselves will become over-politicized and thus harder to solve. And inferences drawn about what makes sense for other districts will be distorted by the differences with DC.
Monday, December 01, 2008
Brainstorm
Hope everyone had a great Thanksgiving break. Mine was excellent, including a trip to New York to see The Seagull, right up until the part that involved being on the southbound New Jersey Turnpike at 11PM in the middle of a rainstorm, stop-and-go-traffic, etc. Then, not so much.
Announcement: Starting today I'm going to be a regular contributor to Brainstorm, the group blog at the Chronicle for Higher Education. It will include a fair amounting of cross-posting from Q&E along with new material. You'll also be seeing my byline in the print edition, starting with with this column about the biennial "Measuring Up" state report cards and how they were right all along.
Edison-Go-Round
EdisonLearning, the school management company founded by Chris Whittle in the early 1990s, has fired it CEO, Terry Stecz, two years after he replaced Whittle. In a memo to Edison staffers, Michael Stakias, the president of Liberty Partners, a New York-based private equity firm that is Edison’s majority owner, announced that Jeff Wahl, Edison’s chief operating officer since 2007, would replace Stecz immediately. Before joining Edison, Wahl spent 15 years in various management roles at General Electric. Maybe Liberty's hoping he's got the right background to illuminate a path to profitability at a company named for the inventor of the lightbulb.
Stecz apparently didn't. He was working in management at Pharmacia, the $14-billion heathcare company with products that included Celebrex and Nicorette, when Liberty recruited him to be Edison’s COO in 2004. He became Edison’s CEO when Liberty pushed Whittle out of the company’s management in early 2007. Stecz struggled, with little apparent success, to shake off Edison troubled legacy as a controversial, unprofitable school management company, going so far as to change the company’s name earlier this year from Edison Schools, Inc, to EdisonLearning and announce a move into on-line education.
Whittle hasn’t fared much better that his successor. The high-living entrepreneur set out to launch an international network of high-end for-profit private schools when he left Edison in early 2007, only to depart his new company, Nations Academy, last summer in the wake of a falling out with his major investor, Sunny Varkey of Dubai.
Whittle is reportedly planning a new for-profit private school business. And he’s trying to raise some cash. He’s put up for sale for $27 million a guest house and a third of the property on his 11 acre estate on Georgia Pond in the Hamptons. Six years ago, when Edison’s stock crashed and the company nearly went under, Whittle sought to sell the entire estate, where the neighbors include Steven Spielberg and Martha Stewart, for $45 million, before taking the property off the market when Liberty bought Edison.
Stecz apparently didn't. He was working in management at Pharmacia, the $14-billion heathcare company with products that included Celebrex and Nicorette, when Liberty recruited him to be Edison’s COO in 2004. He became Edison’s CEO when Liberty pushed Whittle out of the company’s management in early 2007. Stecz struggled, with little apparent success, to shake off Edison troubled legacy as a controversial, unprofitable school management company, going so far as to change the company’s name earlier this year from Edison Schools, Inc, to EdisonLearning and announce a move into on-line education.
Whittle hasn’t fared much better that his successor. The high-living entrepreneur set out to launch an international network of high-end for-profit private schools when he left Edison in early 2007, only to depart his new company, Nations Academy, last summer in the wake of a falling out with his major investor, Sunny Varkey of Dubai.
Whittle is reportedly planning a new for-profit private school business. And he’s trying to raise some cash. He’s put up for sale for $27 million a guest house and a third of the property on his 11 acre estate on Georgia Pond in the Hamptons. Six years ago, when Edison’s stock crashed and the company nearly went under, Whittle sought to sell the entire estate, where the neighbors include Steven Spielberg and Martha Stewart, for $45 million, before taking the property off the market when Liberty bought Edison.
18th Century Skills
Before there were “21st Century Skills,” there were “18th Century Skills.” None other than Benjamin Franklin identified 13 virtues to which he aspired. But Franklin knew that simply trying to embody his virtues without keeping track of his performance wouldn’t be enough. So he created a series of tables to record daily transgressions.
Quantifying his data this way made it possible for Franklin to track his progress over time. While he never achieved perfection on the scale he created, he “was by the endeavor a better and happier man” and “had the satisfaction of seeing [marks representing transgressions] diminish” over time.
Not unlike Mr. Franklin, teachers gather quasi-quantitative data about students every day. Doing so can be as simple as a system of behavior checks and minuses. But unless that data is captured consistently over time and communicated to all of the adults who work with that student, we miss opportunities to identify trends and correlations that can help us serve students better and assess the impact of our interventions. Technology has the potential to make such information easier to capture and quantify as well as to provide improved tools of analysis and communication. Wireless Generation, for example, develops simple ways for teachers, students and others to capture quasi-quantitative information about academic and social indicators on hand held devices. Having the information digitized makes it easy to identify trends and correlations.
Imagine if digital tools that could highlight a correlation between lapses in Temperance and Chastity had been available to Mr. Franklin.
Quantifying his data this way made it possible for Franklin to track his progress over time. While he never achieved perfection on the scale he created, he “was by the endeavor a better and happier man” and “had the satisfaction of seeing [marks representing transgressions] diminish” over time.Not unlike Mr. Franklin, teachers gather quasi-quantitative data about students every day. Doing so can be as simple as a system of behavior checks and minuses. But unless that data is captured consistently over time and communicated to all of the adults who work with that student, we miss opportunities to identify trends and correlations that can help us serve students better and assess the impact of our interventions. Technology has the potential to make such information easier to capture and quantify as well as to provide improved tools of analysis and communication. Wireless Generation, for example, develops simple ways for teachers, students and others to capture quasi-quantitative information about academic and social indicators on hand held devices. Having the information digitized makes it easy to identify trends and correlations.
Imagine if digital tools that could highlight a correlation between lapses in Temperance and Chastity had been available to Mr. Franklin.
Wednesday, November 26, 2008
Malcom Gladwell's Outliers, Part 2
Continuing from yesterday's post about Gladwell's new book, which is basically an inventory of all the things besides individual talent and initiative that lead to unusual success. Put another way, it's a book about unequal opportunity--how individual success and failure are a product of external circumstances, much more so than people like to believe, and how those circumstances vary profoundly among people in complex, inequitable, and often random ways.
The book also has a whole chapter about KIPP, and this is where the flaws in Gladwell's way of writing are most glaring. He cites Karl Alexander's well-known "summer learning loss" research of students in the Baltimore public schools. Low-income students actually gained more during schools than their well-off peers, Alexander found, but fell back over the summer while the rich kids moved ahead. It's an important point, and has led to a lot of discussions about the use of time in school. (See an ES report on the subject here.) But this leads Gladwell to say the following:
The weird thing is that Gladwell goes directly from here to a description of the KIPP Bronx Academy, which indeed provides students with more time. But, as Gladwell himself notes, it also has a culture that demands a lot of effort from students. The KIPP teachers emphasize discipline, self-control and respect for authority, peers, and oneself. They explicitly teach students how to listen and respond in class. They consider every student, no matter how disadvantaged, as bound for college. Does Gladwell really believe that most inner city schools work this way? That the only difference between KIPP and regular public schools is time? Long hours are far from the only "lessons from the rice paddy," as Gladwell more than adequately describes.
In the chapter on expertise--a concept that has (justifiably) gotten a lot of attention from cognitive and education researchers--Gladwell asserts that there's essentially no such thing as great accomplishment without long, hard work. You need roughly 10,000 hours of practice at something to be really good at it, he says, and what distinguishes people like Bill Gates from others isn't fantastic intellect--although Gates certainly has that--but rather the fact that he had an exceedingly rare opportunity to accumulate those hours early in his career, at exactly the right moment in history. Which leads to another of Gladwell's main themes: timing. The 20th century's software titans were all born around 1955, he notes, just like the 19th century's industrial giants were born near 1834 and the most successful Manhattan lawyers were born in a few years bracketing 1931. All of these people had the intelligence and courage to take advantage of a rare opportunity created when the circumstances of their business fundamentally changed. But the fact that the opportunity presented itself in the first place was just luck. Men (and women) need their moment in order to succeed.
From there Gladwell moves on to family background and culture, with a side conversation about IQ. He notes the lack of correlation between extremely high IQ scores and success in life, and offers the sad example of one Chris Langan, IQ of 190+, who grew up with an abusive father in a broken home, dropped out of college due (he claims) to some truly incompetent and uncaring financial aid officers, and currently lives on a horse farm in rural Missouri, working in solitude on a theory of the universe that nobody will ever read:
He knew he needed to do a better job of navigating the world, but he didn't know how...These were things that others, with lesser minds, could master easily. But that's because those others had had help along the way, and Chris Langan never had. It wasn't an excuse. It was a fact. He'd had to make his way alone, and no one--not rock stars, not professional athletes, not software billionaires, and not even geniuses--ever makes it alone.Gladwell devotes most of the rest of the book to culture. Appalachian blood feuds, Korean airline crashes, successful Jewish lawyers, and the tendency of Asians to be good at math, he says, can all be explained by the particular values, attitudes, and inclinations of different cultures. Asian agricultural societies are built around rice cultivation, he observes, which requires entrepreneurial ism, attention to detail, complex management, perseverance, and phenomenal amounts of work, all year long. European-style wheat cultivation, by contrast, is much simpler, amenable to mechanization and unskilled labor, and lets you take half the year off. The things you need to do to cultivate rice are the same things you need to do to learn math, he says, noting:
When students sit down to take the TIMMS exam, they also have to fill out a questionnaire. It ask them all kinds of things, such as what their parents' level of education is, and what their views about math are, and their friends are like. It's not a trivial exercise. It's about 120 questions long. In fact, it's so tedious that many students leave as many as ten or twenty questions blank. Now here's the interesting part. As it turns out, the average number of items answered on that questionnaire varies from country to country. It is possible, in fact, to rank all the participating countries according to how many items their students answer on the questionnaire. Now, what do you think happens if you compare the questionnaire rankings with the math rankings on TIMMS? They are exactly the same. In other words, countries whose students are willing to concentrate and sit still long enough and focus on answering every single question in an endless questionnaire are the same countries whose students do the best job of solving math problems.
The book also has a whole chapter about KIPP, and this is where the flaws in Gladwell's way of writing are most glaring. He cites Karl Alexander's well-known "summer learning loss" research of students in the Baltimore public schools. Low-income students actually gained more during schools than their well-off peers, Alexander found, but fell back over the summer while the rich kids moved ahead. It's an important point, and has led to a lot of discussions about the use of time in school. (See an ES report on the subject here.) But this leads Gladwell to say the following:
What Alexander's work suggests is that the way in which education has been discussed in the United States is backwards. An enormous amount of time is spent talking about reducing class size, rewriting curricula, buying every student a shiny new laptop, and increasing school funding--all of which assumes that there is something fundamentally wrong with the job schools are doing. But look back at [Alexander's data]. Schools work. The only problem with school, for the kids who aren't achieving, is that there isn't enough of it...For it's poorest students, America doesn't have a school problem. It has a summer vacation problem, and that's the problem the KIPP schools set out to solve. They decided to bring the lessons of the rice paddy to the American inner city.
The weird thing is that Gladwell goes directly from here to a description of the KIPP Bronx Academy, which indeed provides students with more time. But, as Gladwell himself notes, it also has a culture that demands a lot of effort from students. The KIPP teachers emphasize discipline, self-control and respect for authority, peers, and oneself. They explicitly teach students how to listen and respond in class. They consider every student, no matter how disadvantaged, as bound for college. Does Gladwell really believe that most inner city schools work this way? That the only difference between KIPP and regular public schools is time? Long hours are far from the only "lessons from the rice paddy," as Gladwell more than adequately describes.
Gladwell is often characterized as public intellectual, and he's clearly a very smart guy. But his real genius lies in explanation. His 10,000 hours of expertise came not in sociology but journalism, at the Washington Post. He's a master of marrying compelling anecdotes with important ideas. The world is a complicated, confusing place, and Malcolm Gladwell makes it understandable by telling stories that are fun to read. This is an extremely difficult thing to do, and nobody does it better. But there's a temptation in this kind of writing to sand down the edges of things to a bright polish and summarize complex issues in a pithy phrase. It's a comforting, attractive message--things are simpler than you think. But it's not always true. Saying that school time matters is one thing--saying it's the only thing that matters is something else entirely.
That said, I admire the spirit of Outliers and really enjoyed reading it. It's rooted in compassion and a belief in the possibility of a better world. Gladwell thinks that if people really appreciated all the dimensions of opportunity, they'd be more likely to support ideas and policies designed to expand opportunity to more people. Gladwell himself is an outlier, and the book is a worthy way to use the influence that position affords.
Tuesday, November 25, 2008
Malcom Gladwell's Outliers,, Part 1
Malcom Gladwell's new book, Outliers, was released last week. I read it over the weekend, on the theory that I had roughly 60 days--90 at the outside--before I'd heard it referenced at so many conferences that mere mention of the central anecdotes would cause me to reach for a hotel pen and stab myself in the eye as a distraction from the pain. I believe the medical term for this is "Thomas Friedman Syndrome."
Outliers is a good book in many ways, and says a lot about education. It's a critique of the standard narrative of extraordinary success, those inspiring tales of hard work and gumption that are often used to explain the achievements of sports stars and CEOs. The Bill Gates's of the world, in other words--and the Malcolm Gladwells. These stories are badly incomplete, Gladwell says, because they ignore context. Success is not just a matter of who people are, but where they come from, along with countless instances of plain luck, for better and for worse.
The first chapter deals with the fairly well-known phenomenon of birthdays and Canadian hockey players. A hugely disproportionate number of elite hockey players have birthdays in January, February and March. That's because the age cutoff for entry into junior hockey leagues is January 1. A five-year old hockey player born on that day (they start early in Canada) could be as much as 25 percent older than his youngest competitors, giving him a major advantage in size, strength, and coordination. Since junior league hockey is a ruthless meritocracy, that success would lead to selection in more elite leagues, and thus more opportunities for practice, better coaching, more success, even more elite leagues, and so on. What begins as an arbitrary age-based difference evolves into actual differences of skill and technique, masking the original injustice.
Gladwell notes that this problematic in two ways. First, it's patently unfair to children born at the end of the year, who have virtually no chance of advancing to the highest hockey ranks. Second, it's really inefficient as a means of matching training with talent. Canada has effectively cut its pool of potential hockey stars in half.
The parallels with higher education are obvious. Earlier this week, David Brooks wrote:
Jan. 20, 2009, will be a historic day. Barack Obama (Columbia, Harvard Law) will take the oath of office as his wife, Michelle (Princeton, Harvard Law), looks on proudly. Nearby, his foreign policy advisers will stand beaming, including perhaps Hillary Clinton (Wellesley, Yale Law), Jim Steinberg (Harvard, Yale Law) and Susan Rice (Stanford, Oxford D. Phil.).As a Democrat, and as someone who thinks smart people should be running the country, I'm thrilled. But the assumption that a true valedictocracy would naturally lead to an administration staffed exclusively by the graduates of a tiny handful of private universities is wrong. The process by which students end up at Harvard and then the upper reaches of government is nothing close to a pure merit-based sorting process. First, all kinds of educational resource misallocations at the K-12 level give greater opportunities to upper-income students. Then elite colleges add admissions preferences for legacies, donors, the children of the rich, famous and powerful, and graduates of long-established networks of elite private secondary schools. From there it's a short distance--via alumni, social and recruiting networks--to the seats of power in finance and government, which, like many elite colleges, are on the East Coast.
The domestic policy team will be there, too, including Jason Furman (Harvard, Harvard Ph.D.), Austan Goolsbee (Yale, M.I.T. Ph.D.), Blair Levin (Yale, Yale Law), Peter Orszag (Princeton, London School of Economics Ph.D.) and, of course, the White House Counsel Greg Craig (Harvard, Yale Law).
This truly will be an administration that looks like America, or at least that slice of America that got double 800s on their SATs. Even more than past administrations, this will be a valedictocracy — rule by those who graduate first in their high school classes. If a foreign enemy attacks the United States during the Harvard-Yale game any time over the next four years, we’re screwed
So while the President-elect himself comes from famously humble origins, the family and educational backgrounds of the advisers Brooks mentions contain a vastly disproportionate number of elite university professors and administrators, wealthy families, and exclusive private high schools, when compared to the population at large. That doesn't mean those people aren't brilliant, talented and hard-working. They are, in the same way that NHL hockey players are. But in both cases they're products of an inefficient and unfair system. In any given year, the 50 smartest public high school graduates from the state of Florida will surely have more raw intellectual ability than the 50 smartest graduates of Phillips Exeter Academy, but members of the latter group are much more likely to end up in the West Wing someday.
More on Outliers, KIPP, why Asian students are good at math, and K-12 education generally tomorrow.
Time Matters
Last month Seattle parents released an analysis of student learning time (defined as time students actually spend in class) at the city's ten high schools. They found large discrepancies:
At Garfield High in Seattle, classes last at least 55 minutes. At Nathan Hale High, they're often 50. Garfield has eight short days when students arrive late or leave early so teachers can collaborate and train. Nathan Hale has 40, and schedules 100 minutes each week for students to simply read.
At the end of the year, that means students at Garfield spend about 23 more hours in each academic class — roughly the equivalent of four more weeks of instruction, according to an analysis done by a parents group in West Seattle.
The article follows with a dramatic "does it matter" without ever answering the question. Like money, education commentators tend to look to the research without thinking about common sense. Do time and money matter in education? The logical answer is of course they do, but there might not be a direct linear relationship between the two inputs and educational outcomes. There are always confounding variables--things like central office inefficiencies cutting down on the impact of money or a teacher not using her time effectively.
That lost time matters. Nathan Hale has lower percentages of white and low-income students than Garfield, yet it has lower test results in reading, writing, math, and science. Despite having greater diversity, Garfield has lower achievement gaps in reading between men and women, whites and blacks, students on free and reduced lunch and those who are not, and students in special education. Nathan Hale is a good school, but Garfield outperforms it across the board despite a harder assignment.
State and district officials don't seem to see the link between Garfield's success and its additional time. The article quotes Kathe Taylor, policy director at the State Board of Education, wondering if counting minutes is like missing the forest for the trees, saying "If one school district can do in two hours what it takes another six hours to do, and the students achieve equally well, then you have to ask what difference does it make." Nathan Hale's principal chimes in to say, "Raw minutes is nowhere near the whole story."
Not the whole story, maybe. But not one to ignore either.
Hat tip to NCTQ.
Student Loan Shenanigans
Sherman Dorn blogged today about the difficulties his wife has had getting Nelnet (a loan company that has engaged in nefarious practices in the past) to process her paperwork for the federal loan forgiveness program. The program provides loan forgiveness up to $17,500 for teachers who teach in hard to staff subject areas, like special education.
Dorn wants to know if anyone else has had similar troubles getting their loan forgiveness paperwork processed. My guess is that yes, there are others out there with the same problem.
Dorn wants to know if anyone else has had similar troubles getting their loan forgiveness paperwork processed. My guess is that yes, there are others out there with the same problem.
High School Seniors Are Like Opilio Crab
I love the Deadliest Catch, an action filled Discovery Channel show about Alaskan crab fishermen. I might just eat crab instead of turkey this year because Captain Phil asked me to. But I've never been able to find the education connection I needed to write about the Deadliest Catch on Quick & Ed--until now (not that that stops some of us). And I'd like to say thanks to Inside Higher Ed for giving me the opportunity.Kent Barnds writes in a column for IHE today about how the tough, hard bitten job of a crab fisherman is like, well, a college admissions officer. Ignoring, of course, the admissions officer's climate controlled office and cushy chair.
Here are a few of the parallels Barnds draws: the unwanted commentary both admissions officers and the now famous fishermen receive from those who haven't actually done the job; the need to catch just the right amount of crab, or students, to meet the quota; and the anxiety of watching the pot come out of the water wondering if it'll be full of crab or just contain a dead fish--much like the anxiety admissions officers face as they get replies from admitted students. Hauling a 700-pound pot filled with crab and risking a paper cut: it all has drama.
Barnds says, "Jonathan Hillstrand, who captains the Time Bandit, once declared “We’d rather be lucky than good any day.” Let’s be candid — it takes quite a bit of luck to predict and “know” where the crab will be and takes just as much to predict and “know” what 17-years olds will think and do." So the end lesson is that, like opilio crab, 17-year old high school seniors are running in herds along the ocean floor and admissions officers are in boats, developing a strategy to find them and figuring out the right bait to haul them in.
At least the admissions officers don't have to toss the females back into the ocean anymore.
Monday, November 24, 2008
Admired, Not Read
Last year, I was invited to Dickinson College in southern Pennsylvania to debate the meaning of success in higher education. My counterpart in the discussion was Christopher Nelson, president of St. John’s College, a small liberal arts school in Annapolis, Maryland. We were introduced by our host, and Nelson went first. His speech was erudite, passionate, and replete with classical references. He waxed eloquent about the meaning of knowledge, and how teaching as an enterprise was central to the St. John’s philosophy. I found myself glancing uneasily at my own notes, which had always served me well in the past but suddenly seemed paltry by comparison.
I didn’t realize it then, but I had run headlong into the Great Books of the Western World, the subject of a smart, engaging new book by Alex Beam, a columnist for the Boston Globe. In A Great Idea at the Time: The Rise, Fall, and Curious Afterlife of the Great Books, Beam traces the history of a peculiar moment in the development of America’s striving middlebrow culture, when hundreds of thousands of families across the nation decided to spend a lot of money on ancient texts that few would ever read. It’s also a story of higher education, and a 140-year-old argument about the responsibilities that colleges have to students.
Beam’s narrative begins in 1869, with the appointment of Charles Eliot to the presidency of Harvard University...[for more, in which I argue that the great books matter and liberals need to reclaim the liberal arts, click here]
EVENT: Computers, Professors, and the Cost of Higher Education
Note: Pushing this back to the top as a reminder to sign up, and also because the magazine article in question is now live to the world, here. Read it! Right now!
In fact, this is wrong. Colleges are perfectly capable of becoming more productive in the same way that lots of other industries have--by substituting capital for labor and replacing lots of expensive employees with less-expensive computers. It's not a theoretical idea--hundreds of colleges are doing this right now and more are jumping on board every day. Given that colleges are about to be mightily screwed by recession-induced budget cuts, this trend is likely to accelerate. But you haven't heard about it, in part because the significant cost-savings aren't being passed on to consumers in the form of lower prices. This is the subject of new article I've written for Washington Monthly which you can--nay, must--read here.
And to see a sure-to-be-fascinating discussion of these issues, live and in person, featuring a panel full of experts, plus myself, sign up for and attend this Education Sector Event, to be held on the morning of December 2nd here in Washington, DC. Ask questions, heckle, throw things, get free magazines and related materials--it's sure to be a great time.
Friday, November 21, 2008
Philadelphia Public Schools Gain Market Share, Blames Charters
In the private sector when an enterprise gains customers it's a good thing. Apparently that is not the case in Philadelphia School District, where their chief budget officer says charter schools are costing the district $105 million because 27% of their students were previously home-schooled or educated in a private school. Putting aside the idea that a 5% increase in market share is a bad thing, let's do some math. Stay with me here.
The city's chief budget officers claims charters, because of increased market share, are costing the city an extra $105 million. Charters educate 34,4000 students in the city and receive $320 million in reimbursements (including some state funds) for a total per-pupil expenditure (excluding private money) of $9,302.33.
The city reimburses charters $8,088 for every student in general education and $17,658 for every student in special education. Assuming charters took the same percentage of special education children as traditional public schools (13.2% in Philadelphia), how much should the city be spending on these new charter students?
13.2% of children in special ed. * $17,658 per student = $2,330.86
86.8% of children in regular education * $8,088 per student = $7,020.38
$2,330.86 + $7,020.38 = $9,351.24
So charters are actually getting less than they should. More students, less money, and the city complains?
The city's chief budget officers claims charters, because of increased market share, are costing the city an extra $105 million. Charters educate 34,4000 students in the city and receive $320 million in reimbursements (including some state funds) for a total per-pupil expenditure (excluding private money) of $9,302.33.
The city reimburses charters $8,088 for every student in general education and $17,658 for every student in special education. Assuming charters took the same percentage of special education children as traditional public schools (13.2% in Philadelphia), how much should the city be spending on these new charter students?
13.2% of children in special ed. * $17,658 per student = $2,330.86
86.8% of children in regular education * $8,088 per student = $7,020.38
$2,330.86 + $7,020.38 = $9,351.24
So charters are actually getting less than they should. More students, less money, and the city complains?
Bailout Back and Forth
Treasury Secretary Henry Paulson set off a flurry of activity when he suggested last week that some of the $700 billion bailout money might go to help companies issue private student loans. Higher Ed Watch explains in this post why Paulson's plan is a bad idea.
NASFAA (the National Association of Student Financial Aid Administrators), though, seems to think it's a great idea, saying that "stricter loan eligibility requirements and higher interest rates and fees on non-federal loans are jeopardizing educational opportunity." A letter sent Wednesday from representatives of colleges, students, and organizations like the Project on Student debt explains why NASFAA is wrong, and Higher Ed Watch explains what colleges can do to ensure educational opportunity without any government bailouts. Inside Higher Ed has all the back and forth here.
But while everyone has been talking about private loan eligibility, one of the biggest, most genuine threats to college access--reductions in public college funding--has actually been happening. The California State University system got approval on Wednesday to turn away at least 10,000 eligible students next fall because of overcrowding and underfunding.
NASFAA (the National Association of Student Financial Aid Administrators), though, seems to think it's a great idea, saying that "stricter loan eligibility requirements and higher interest rates and fees on non-federal loans are jeopardizing educational opportunity." A letter sent Wednesday from representatives of colleges, students, and organizations like the Project on Student debt explains why NASFAA is wrong, and Higher Ed Watch explains what colleges can do to ensure educational opportunity without any government bailouts. Inside Higher Ed has all the back and forth here.
But while everyone has been talking about private loan eligibility, one of the biggest, most genuine threats to college access--reductions in public college funding--has actually been happening. The California State University system got approval on Wednesday to turn away at least 10,000 eligible students next fall because of overcrowding and underfunding.
Thursday, November 20, 2008
Cash-strapped Colleges
With headlines predicting a steep recession, colleges and universities are already making budget cuts. Compare what some schools are saying about the financial crisis:
with what others are already doing:Morton Schapiro, president of Williams College in Massachusetts, which has long had a commitment to accepting students without considering their financial situation, said he doubted that all colleges with such full need-blind policies would be able to hold to them.
“The major dial you turn for most financial crises is that you admit more students who can pay, as a way of increasing revenues,” Mr. Schapiro said. “With the tremendous decline in wealth, I think fewer people will hold on to needs blind.”
In October, Cornell University in Ithaca, N.Y., increased its fund-raising campaign goal for undergraduate scholarships to $350 million from $225 million, which has "helped reinvigorate giving" specifically for this priority, says Simeon Moss, press office director at Cornell. While overall giving is down, donations directed toward undergraduate aid have soared to $63.4 million in fiscal-year 2008, from $13.7 million in fiscal-year 2007, he says.There are good and bad ways colleges and universities can manage their budgets during tough times. Tying fundraising efforts to student financial aid seems to be one of the good ones. Kudos to Cornell for taking proactive action.
Tuesday, November 18, 2008
Stalking the Iron Sheik
Last night I saw an advance screening of Darren Aronofsky's new movie, The Wrestler, starring Mickey Rourke as an aging, down-on-his-luck professional wrestler and Marisa Tomei as a local stripper who may or may not have a heart of gold. It's a very good movie that you should definitely go see when it's released in theaters next month. The Wrestler is a human drama first and foremost, but it's also about the cold reality of working life, the aesthetics of late-80s hair metal (musical and otherwise), and the strong bonds of subculture brotherhood. It was shot on a shoestring $6 million budget in hand-held documentary style, and the contrast with the precise, controlled visuals of Requiem for a Dream and the criminally-underrated The Fountain is pretty amazing.
I'm not giving much of the plot away in revealing that the movie revolves, in part, around the possibility of the protagonist, one Randy "Ram" Robinson, participating in a 20th anniversary rematch of an epic 1989 pay-per-view showdown with a wrestler named "The Ayatollah." As we all know, this is a thinly-veiled reference to the classic 1984 title bout between Hulk Hogan (Randy basically looks what Hogan would have looked like if he'd spent the last 20 years on a crystal meth bender, or living the life that Mickey Rourke actually lived) and Hossein Khosrow Ali Vaziri, a.k.a The Iron Sheik. And this, in turn, is all the excuse I need to tell my Iron Sheik story.
The Sheik, now retired, was born in Iran and had real wrestling skills, competing on the national team (Iran has some of the best wrestlers in the world) in the late 1960s. By the 1970s he had moved to the U.S., where he wrestled under names like The Great Hussein Arab (insert joke about McCain campaign rallies here). Certain contemporaneous developments in geopolitics made his status as the designated "heel" or bad guy in the pro wrestling ring more or less inevitable, and he went on to a successful career, winning the heavyweight title before losing it to Hogan while awing fans in mid-sized regional municipal auditoriums across the land with the power of his patented finishing maneuver, the "camel clutch."
Anyway, it was some time in the late 1990s, and I was in the Indianapolis airport waiting for an outbound flight. I was hungry so I went to the scary food court, grabbed something greasy and marginally edible, and went to find a table--only to see, sitting right in front of me, the Sheik himself, amiably chatting with another really large dude who was obviously a fellow wrestler. They'd been in town the previous night for a match at the since-demolished Market Square Arena (site of Elvis' last concert, fyi). I didn't want to interrupt them--the 12-year old in me still found him extremely scary--so I sat nearby to eavesdrop. Their conversation went something like this:
Another Really Large Dude: "Hey, did you hear that The Undertaker bought a place on a golf course in Myrtle Beach?"
Sheik: "Real estate is always a smart place to invest."
ARLD: "Yeah, Hacksaw Jim Duggan told me he cleared $100,000 on his place in Florida last year."
Sheik: "If Jake the Snake had at least put his money in T-bills like I told him to, he could have retired five years ago."
And so on and so forth, the most mundane conversation imaginable, except they referred to everyone, and each other, by their stage names. It was surreal. Eventually they finished their burgers and walked to airport security, at which point the security guys started jumping and down, pointing, and yelling "the Iron Sheik!" (airport security jobs were close to minimum wage gigs back then, as you'll recall). We ended up on the same plane, but the Sheik and ARLD were in first class so I didn't get a chance to hear more.
No real point to this story other than a brush with semi-greatness, and you should really go see The Wrestler.
Dear President-Elect Obama...
NCLB reauthorization may not be at the top of the next administration's to-do list, what with the economic meltdown and two wars, but President Obama will need to tackle President Bush's signature education law eventually. And he'll need all the good ideas he can get.
Today, ES releases two briefs offering ideas on how the Obama administration can reform NCLB. Title 2.0 focuses on how President Obama can revamp the federal role in human capital by focusing Title II dollars on higher impact reforms. And In Need of Improvement offers a number of steps Congress and the Obama administration can take to strengthen NCLB's choice provision.
Today, ES releases two briefs offering ideas on how the Obama administration can reform NCLB. Title 2.0 focuses on how President Obama can revamp the federal role in human capital by focusing Title II dollars on higher impact reforms. And In Need of Improvement offers a number of steps Congress and the Obama administration can take to strengthen NCLB's choice provision.
Not-So-Great Expectations
Higher education is haunted by a formula, which goes something like this: P/R=G.
The P stands for student preparation, broadly defined — the combination of innate ability and elementary-school and secondary-school preparation that students bring to college. Imagine those attributes normalized on a scale going from 0 to 1, with 1 describing the smartest, most well-educated student in the world.
The R stands for rigor, defined by individual colleges and universities — academic requirements, placement-exam cut scores, and the general difficulty of the work. R increases as standards become more rigorous, with the top values at places like the California Institute of Technology and the Massachusetts Institute of Technology.
The G stands for the odds of a student's earning a degree. An intelligent, well-prepared student attending a college with typical standards would be very likely to graduate. An ill-prepared student who enrolls somewhere with unusually tough standards would not.
The P/R=G formula dominates the way people think about college graduation rates and student success. And, not coincidentally, it puts colleges in the position of having no real responsibility or efficacy when it comes to making G higher. They can't make P higher, because raw ability is what it is, and the elementary and secondary schools are someone else's problem. And they can't make R lower, because that would betray their scholarly ideals and dumb things down for the best students. A low G is regrettable, but really, what can be done?
It's a pretty depressing conclusion. So I was glad to read a report on the Community College Survey of Student Engagement, known as Cessie, which says that the formula is all wrong.
The P stands for student preparation, broadly defined — the combination of innate ability and elementary-school and secondary-school preparation that students bring to college. Imagine those attributes normalized on a scale going from 0 to 1, with 1 describing the smartest, most well-educated student in the world.
The R stands for rigor, defined by individual colleges and universities — academic requirements, placement-exam cut scores, and the general difficulty of the work. R increases as standards become more rigorous, with the top values at places like the California Institute of Technology and the Massachusetts Institute of Technology.
The G stands for the odds of a student's earning a degree. An intelligent, well-prepared student attending a college with typical standards would be very likely to graduate. An ill-prepared student who enrolls somewhere with unusually tough standards would not.
The P/R=G formula dominates the way people think about college graduation rates and student success. And, not coincidentally, it puts colleges in the position of having no real responsibility or efficacy when it comes to making G higher. They can't make P higher, because raw ability is what it is, and the elementary and secondary schools are someone else's problem. And they can't make R lower, because that would betray their scholarly ideals and dumb things down for the best students. A low G is regrettable, but really, what can be done?
It's a pretty depressing conclusion. So I was glad to read a report on the Community College Survey of Student Engagement, known as Cessie, which says that the formula is all wrong.
Monday, November 17, 2008
Hot Boys (With Audio)
If you missed Education Sector's talk last week with author Peg Tyre you can listen to it below. The discussion features Tyre, best-selling author of The Trouble with Boys, New America Foundation Senior Analyst Sara Mead, and USA Today columnist Richard Whitmire. Topics ranged from gender differences in NAEP scores and ADHD diagnoses to boy-girl brain differences and why boys like the video game World of Warcraft.
Last Week, Next Up on Testing
The paper we released last Monday is the first in a series to explore what the next generation of assessments might look like. I received many many emails and comments over the past week, some via our online discussion. A quick recap from all of that:
First, there is a surprisingly strong reaction to the term “21st century skills.” Most wanted to say something about how important and significant these skills are to the students they teach or know. And in general there was agreement that there is a set of “21st century” skills that students need more now than before. But the term “21st century skills” seemed like an unpromising default to many, a way of avoiding specificity. One comment was particularly pessimistic but perhaps a fair point: “It’s a meaningless term—by the time we figure out what it means, it will be the 22nd century. Then what?” As an aside, I initially avoided the term but decided to take it on to see if I could push past the platitudes.
Second, there is a lot of interest at the local level in the new assessment tools and quite a few questions came in about which ones worked best, which ones should teachers/schools/districts use. My response is that there isn’t one tried and tested (sorry for pun) assessment that districts and schools should adopt and start using. The CWRA, which I profiled in the report, is one example—and I think a good one—of how schools are trying out new forms of assessment that measure reading, writing and math skills and problem-solving, inquiry and decision-making skills. The larger point is not that this is the right test for every school or district, but that this is the right direction for assessment.
Related to this is the problem of cost. There was a lot of concern about finding funds for assessments like the CWRA. “Even if my school wanted to try this,” wrote one teacher about the CWRA, there’s no way they would spend any extra money on it. This isn’t surprising, wrote Jack Beirwirth, who is the superintendent of Long Island’s Herricks Public Schools, the first public school district to use the CWRA. We’re all cutting back, he said, on programs, services and jobs. But the CWRA is well-worth the extra cost--teaching and measuring critical thinking and analytical reasoning are among our goals, he explains. Herricks also became one of four school districts participating independently in the 2006 PISA.
There were also a lot of comments about breaking down the distinction between instruction and assessment. Can we “embed” assessment in teaching, so that teachers can learn how their students are doing and improve their practice at the same time. So that assessment is not seen as a series of burdensome tests but also a tool for continuous learning. This isn’t easy—it requires teachers who know how to use assessment both for generating summative information and to inform their daily practice. But it can be done--see Paul Curtis’ description of New Tech High's approach.
Emerging technologies play a big role in this (I received several emails asking “what about technology?”). Our next paper on assessment--this one by Bill Tucker-- will examine how information technology can be used to improve assessment.
First, there is a surprisingly strong reaction to the term “21st century skills.” Most wanted to say something about how important and significant these skills are to the students they teach or know. And in general there was agreement that there is a set of “21st century” skills that students need more now than before. But the term “21st century skills” seemed like an unpromising default to many, a way of avoiding specificity. One comment was particularly pessimistic but perhaps a fair point: “It’s a meaningless term—by the time we figure out what it means, it will be the 22nd century. Then what?” As an aside, I initially avoided the term but decided to take it on to see if I could push past the platitudes.
Second, there is a lot of interest at the local level in the new assessment tools and quite a few questions came in about which ones worked best, which ones should teachers/schools/districts use. My response is that there isn’t one tried and tested (sorry for pun) assessment that districts and schools should adopt and start using. The CWRA, which I profiled in the report, is one example—and I think a good one—of how schools are trying out new forms of assessment that measure reading, writing and math skills and problem-solving, inquiry and decision-making skills. The larger point is not that this is the right test for every school or district, but that this is the right direction for assessment.
Related to this is the problem of cost. There was a lot of concern about finding funds for assessments like the CWRA. “Even if my school wanted to try this,” wrote one teacher about the CWRA, there’s no way they would spend any extra money on it. This isn’t surprising, wrote Jack Beirwirth, who is the superintendent of Long Island’s Herricks Public Schools, the first public school district to use the CWRA. We’re all cutting back, he said, on programs, services and jobs. But the CWRA is well-worth the extra cost--teaching and measuring critical thinking and analytical reasoning are among our goals, he explains. Herricks also became one of four school districts participating independently in the 2006 PISA.
There were also a lot of comments about breaking down the distinction between instruction and assessment. Can we “embed” assessment in teaching, so that teachers can learn how their students are doing and improve their practice at the same time. So that assessment is not seen as a series of burdensome tests but also a tool for continuous learning. This isn’t easy—it requires teachers who know how to use assessment both for generating summative information and to inform their daily practice. But it can be done--see Paul Curtis’ description of New Tech High's approach.
Emerging technologies play a big role in this (I received several emails asking “what about technology?”). Our next paper on assessment--this one by Bill Tucker-- will examine how information technology can be used to improve assessment.
A Financial Aid Shake Up
Last Friday, FastWeb, a free, online scholarship search service, released the results of a survey it conducted on student borrowing, showing that half of students applying for private loans, parent PLUS loans, and home equity loans were denied access to funds. If this number is accurate, it means that a large number of students will need to adjust their college choices--from a private to a public institution and from a 4-year to a 2-year institution--because of financial constraints.
There's good reason to be skeptical of FastWeb's high numbers--the survey was sent out to users of the website (people already looking for additional financial aid) and the response rate was low (1,202 responses out of 7 million invitations). It's likely that those who took the time to fill out the survey were also the ones having the most trouble finding financial aid.
But we do know that private student loans, because of the tightening of credit markets, are more difficult to get. While the actual percentages of students being denied private loans may not be as high as FastWeb reports, it's still higher than in previous years, when easy access to credit helped fuel a boom in private lending.
So what are policymakers to do?
Tuition levels have risen to the point at which federal lending limits are insufficient to cover tuition at pretty much all private colleges and even some public, 4-year institutions. As a result, some congressional members are talking about raising federal loan limits again to make up for some of the lost private loan dollars. But that would be a mistake.
Tuition levels were able to get as high as they are partly because of easy debt--in an era of loose borrowing requirements, students were able to access large amounts of private loans. This was a great situation for colleges--students took out large amounts of loans, the institutions got paid, and students were left to bear the debt burdens. While lip service was paid to the need to reduce tuition prices, there wasn't much real pressure - enrollments stayed high and tuition bills were paid. Now that private lending is more limited, there may be some real price pressure on colleges to reduce tuition rates, or at least limit increases.
Adding to the argument against raising federal loan limits is a recent admission by the University of Phoenix that it sets tuition partly based on federal loan limits. And it actually cut tuition in its two-year Axia college division after seeing that students were dropping out because they maxed out their loan eligibility. So maybe less borrowing and more pressure to lower prices isn't such a bad thing in higher education, and could lead to more reasonable tuition rates.
Even when the credit markets loosen up, we (hopefully) won't return to the wild west days of lending that led to our current problems. Just like subprime mortgates aren't really good for home ownership, subprime student lending isn't really good for college access or success. Tighter private loan markets might result in a more cost-consciousness, on the part of students and parents, and institutions.
There's good reason to be skeptical of FastWeb's high numbers--the survey was sent out to users of the website (people already looking for additional financial aid) and the response rate was low (1,202 responses out of 7 million invitations). It's likely that those who took the time to fill out the survey were also the ones having the most trouble finding financial aid.
But we do know that private student loans, because of the tightening of credit markets, are more difficult to get. While the actual percentages of students being denied private loans may not be as high as FastWeb reports, it's still higher than in previous years, when easy access to credit helped fuel a boom in private lending.
So what are policymakers to do?
Tuition levels have risen to the point at which federal lending limits are insufficient to cover tuition at pretty much all private colleges and even some public, 4-year institutions. As a result, some congressional members are talking about raising federal loan limits again to make up for some of the lost private loan dollars. But that would be a mistake.
Tuition levels were able to get as high as they are partly because of easy debt--in an era of loose borrowing requirements, students were able to access large amounts of private loans. This was a great situation for colleges--students took out large amounts of loans, the institutions got paid, and students were left to bear the debt burdens. While lip service was paid to the need to reduce tuition prices, there wasn't much real pressure - enrollments stayed high and tuition bills were paid. Now that private lending is more limited, there may be some real price pressure on colleges to reduce tuition rates, or at least limit increases.
Adding to the argument against raising federal loan limits is a recent admission by the University of Phoenix that it sets tuition partly based on federal loan limits. And it actually cut tuition in its two-year Axia college division after seeing that students were dropping out because they maxed out their loan eligibility. So maybe less borrowing and more pressure to lower prices isn't such a bad thing in higher education, and could lead to more reasonable tuition rates.
Even when the credit markets loosen up, we (hopefully) won't return to the wild west days of lending that led to our current problems. Just like subprime mortgates aren't really good for home ownership, subprime student lending isn't really good for college access or success. Tighter private loan markets might result in a more cost-consciousness, on the part of students and parents, and institutions.
Friday, November 14, 2008
Stop the Violence
Last Sunday, the Washington Post ran a story about the recent surge in school violence at
The research around school-based prevention programs suggests that more important than specific programs or curricula are the principles upon which effective strategies are based and the fidelity with which they are implemented. Schools can decrease problem behavior by organizing and managing themselves effectively, creating environments that support prosocial behavior, instituting clear rules and expectations, and creating structures and supports that help administrators, teachers, and students work together to meet those expectations. Yet these supports are often symbolic (e.g. behavioral contracts) or entirely lacking in the most troubled districts. It is encouraging to know that in some districts (like D.C., according to a draft five-year action plan), strategies are underway to bolster Student Support Teams (SSTs) to better coordinate academic and behavioral interventions for at-risk students. I’d like to see SSTs expanded to become a more universal approach, especially in urban districts where the vast majority of kids can be categorized as at-risk.
- Posted by guestblogger Sara Yonker
Dispatch from the Front Lines
Policy types debate whether school systems should try to fix failing public schools or close them and replace them with charters or other types of new schools. Answer: it's a lot easier to create a constructive school culture with a new team than having to change the hearts and minds of existing staff. But sometimes the reality on the ground, to use the cliche, makes school reform a little trickier. In Chicago, school authories have opted for trying to turn around six failing high schools rather than closing them down because moving students to new schools would cause them to cross gang boundaries.
Secretary Spellings, Blogger
Margaret Spellings, the United States Secretary of Education, is guest-blogging today at our fellow institutional blog, Eduwonk. (She must have missed the memo that bloggers are all a bunch of unshaven, pajama-wearing D&D players living in their parents' basement.) Education Sector is, of course, very pleased to host her discussion of education and technology. On a larger note related to all the current discussions of the new administration's education policies: I think the Secretary and her staff have done some important work over the last three years in the higher education realm and I hope it continues in the future.
Thursday, November 13, 2008
Perspective
Alan Odden says that Michelle Rhee's proposal to give DC teachers the option to trade job security for a lot more money, if they want to, or not, if they don't want to, "would raise eyebrows everywhere, because that would be a gargantuan change."
The only real eyebrow-raising element of this is that it's national news. Here you have someone who was brought in to reform an organization that everyone--everyone--agrees was terribly dysfunctional, and her response has been, in part, to identify the employees who aren't doing a good job and replace them with better employees. This isn't just obvious and rational; it's ordinary. It happens all the time, particularly in organizations that are labor-dependant. And that's a good thing, because we're all much worse off when low-performing organizations stay chronically unreformed and low-performing.
The only real eyebrow-raising element of this is that it's national news. Here you have someone who was brought in to reform an organization that everyone--everyone--agrees was terribly dysfunctional, and her response has been, in part, to identify the employees who aren't doing a good job and replace them with better employees. This isn't just obvious and rational; it's ordinary. It happens all the time, particularly in organizations that are labor-dependant. And that's a good thing, because we're all much worse off when low-performing organizations stay chronically unreformed and low-performing.
Which raises a broader point: from a p.r. standpoint, taking a stand here and now on this issue is a terrible decision for the American Federation of Teachers. Teachers have a very good case to make when they say that, compared to other professionals with comparable levels of education who work at similarly difficult jobs, they're underpaid, disrespected, and forced to work in conditions that most people wouldn't tolerate. Fair compensation, dignity, and a basic quality of working life are things that all people understand and have a right to expect in their jobs.
Permanent job security regardless of performance, by contrast, is enjoyed by a small and shrinking portion of the workforce. Most people can't relate to tenure. They've never had it, they never will, and they understand why. Drawing the line there is a strategy of alienation for a group that has fewer friends than it understands and fewer than it needs.
The Myth Continues
Nicholas Kristof's column today urges President-elect Obama to move education to the front of the policy line. We're all for that here at The Quick and the Ed. But Kristof commits a major sin by repeating the line, that we've twice debunked, that the current generation is the first to have less education than their parents. The line is false both in rhetoric and fact.
I understand Kristof's pain. He's trying to create a crisis where one doesn't exist so as to transfer energy into an important issue. Realistically, our schools as a whole aren't bad; they're just kind of...flat. Take some time to look at the original data where the claim comes from, and here's what you'll find:
I understand Kristof's pain. He's trying to create a crisis where one doesn't exist so as to transfer energy into an important issue. Realistically, our schools as a whole aren't bad; they're just kind of...flat. Take some time to look at the original data where the claim comes from, and here's what you'll find:
- the United States had and continues to have a very high percentage of its adults completing a higher education degree.
- while that percentage has crawled upwards here in the States, other countries have cleared it. Where once we were first, we're now tenth.
- our high school graduation rate has fluctuated between 70 and 75 since at least 1995. Some years are better, some are worse.
- many of our adults acquire GEDs over time, boosting the percentage of the population with a high school diploma. That rate hovers around 87 percent for all generations.
- college graduates in the US continue to have success finding employment for themselves, creating it for others, and capitalizing on their credentials to boost income.
Only 54% of entrants to higher education in the United States obtain a degree. Along with New Zealand, this is the lowest survival rate among OECD countries, where the average is 71% and as high as 91% as in Japan.
Our Comics-Loving President-Elect
As many others have noted, the Huffington Post reports that president-elect Obama "collects Spider-man and Conan the Barbarian comics." This is, of course, welcome news for right-thinking people everywhere. But it also raises more important questions that it answers. Which Spider-man and Conan the Barbarian comics? Some are pretty easy to guess--all Conan comics begin with the classic 1970s Roy Thomas / Barry Windsor-Smith run at Marvel, which arguably remains unequalled. Kurt Busiek's 2003-2006 stint for Dark Horse also stands out. But there were a lot of comics in between. Was Obama picking up the oversize, black and white Savage Sword of Conan back in the 80s? Is he re-reading old issues of King Conan for insights on how Conan dealt with the heavy burden of leadership after finally ascending to the throne? James Earl Jones as Thulsa Doom: inspired or ridiculous? These are important things to know. Similarly, will our future leader do the right thing and officially denounce the Spider-clone as the most egregious retcon in all of comics history? He's old enough to remember when it was big news that Peter Parker married Mary Jane (only to be un-married earlier this year in another retroactive continuity re-boot, ugh)--can he honestly say that his position on that has been consistent over the years? Did he publicly denounce the black suit, on the record, when it was first introduced post-Secret Wars, or did he only follow popular sentiment later on? Todd McFarlane goosed sales, but did he ultimately leave the character better or worse off? I assume he's pointing Malia and Sasha toward Ultimate Spider-man, if not an explanation on his thinking would be helpful. Frankly, the fact that we don't have answers here suggests a breakdown of the mainstream media's pre-election vetting process.
Wednesday, November 12, 2008
All Deliberate Speed
From a New York Times article about a small Lousiana high school that may soon cease to exist:
President-elect Obama's historic victory has been frequently described over the past week as the culmination of a long struggle for civil rights that began in the 1950s with historic court decisions like Brown v. Board. And there is great truth in this. But it's worth remembering that, 54 years and counting since the Supreme Court ordered schools to be desegregated "with all deliberate speed," we're still not finished desegregating. Promises made to children in the middle of the 20th century remain unfulfilled for their grandchildren. There's a whole division within the Justice Department full of lawyers who travel to areas, mostly in the rural South, where educational opportunity remains divided along racial lines. There is still a lot of work to be done.
This tiny, rural town in Cajun country is struggling again to find its racial equilibrium. A 43-year-old desegregation case remained unresolved on Nov. 4 as voters narrowly rejected a property tax increase to build a new Ville Platte High, which has faced decades of neglect since white flight accompanied integration in 1969.
President-elect Obama's historic victory has been frequently described over the past week as the culmination of a long struggle for civil rights that began in the 1950s with historic court decisions like Brown v. Board. And there is great truth in this. But it's worth remembering that, 54 years and counting since the Supreme Court ordered schools to be desegregated "with all deliberate speed," we're still not finished desegregating. Promises made to children in the middle of the 20th century remain unfulfilled for their grandchildren. There's a whole division within the Justice Department full of lawyers who travel to areas, mostly in the rural South, where educational opportunity remains divided along racial lines. There is still a lot of work to be done.
Magnitude
Expanding on something I alluded to in the post below about training and certifying teachers vs. attracting and selecting them: the way these issues are often discussed reveals one of the weak spots in the way research is applied to policy. Essentially, people don't properly account for differences in magnitude.
All rational people want education policy to be informed by the results of objective research and inquiry, conducted using transparent, widely-accepted methods. (In a lot of ways, this is just a restatement of the definition of "rational person.") The kind of analysis that informs policy tends to involve testing hypotheses and identifying differences. Consider randomized control trials, the "gold standard" of research, which increasingly are being used in education. You randomly assign subjects to two groups, apply a treatment to one of them, and see if some pre-defined results vary. If the variance is large enough to meet agreed-upon levels of statistical significance, you conclude that the treatment had an effect. The same basic approach, even without the randomized assignment, is used to tackle other questions--like, for example, the effect of certification on teacher effectiveness.
So let's take another look at that chart with the three overlapping bell curves, one showing the distribution of teacher effectiveness among certified teachers, one showing alt-cert teachers, and one showing uncertified teachers.
If you squint your eyes and look carefully, you'll note that the solid line--the certified teachers--is slightly to the right of (with teachers increasing in effectiveness as they move from left to right along the scale) the other two. In other words, the population of certified teachers was slightly more effective. If the populations were large enough--if, say, they represented every teacher in America--that difference might be statistically significant. It might even be significant beyond a shadow of a statistical doubt, at the .0001 level or what have you. And if that were the case, the results of the research would very likely be translated for lay readers and policymakers into something like this: "A study from [fill in the blank] found that certified teachers are more effective than non-certified teachers." And the natural policy response would be that we should make sure that all students are taught be certified teachers--they're better, after all--and we should invest in and strengthen certification programs, since they work.
The problem, of course, is that if you put your common sense hat on and think for 30 seconds about what that graph actually shows, you naturally conclude that such a policy response would be insane. The difference between certified and uncertified may indeed be statistically signifcant, but it is a tiny statistically signficant difference, one that pales in comparison to the extent to which teachers vary within those populations, and the extent to which the populations are the same. To state the obvious: the most important differences tend to be large, not small.
The vernacular of research and policymaking, in other words, is insufficiently precise about and sensitive to differences in magnitude. All significant differences are not equally signficant just because they met the same test of signficance, but they are often thought of as such.
Tuesday, November 11, 2008
A lot to Say About Testing
We have a lot of comments to the testing and 21st century post from Friday. I'm also getting a lot of emails on this--from teachers who say they are already teaching "21st century skills" and what's new about this anyway?, from district officials saying it's too expensive to assess these skills (isn't it?), from content and curriculum specialists concerned that the focus on 21st century skills will distract from efforts to teach reading and math, and many others. Tag cloud to the side is thanks to Dennis Richards, who wants to know what kinds of teachers and teaching tools it will take to teach these skills to all kids. Much more on all of this at the discussion here. Join in: ask questions, post comments.
Subscribe to:
Posts (Atom)