Stop the Violence

Last Sunday, the Washington Post ran a story about the recent surge in school violence at Hart Middle School, where, in the past several weeks, three teachers were assaulted, a 14-year-old was charged with carrying a shotgun and students discharged fire extinguishers throughout the building.  Hart, a school currently placed on “restructuring” status under NCLB due to its repeated failure to make adequate academic progress, has made little headway in its attempt to raise achievement despite the installment of a young new principal and the promise of additional interventions to support students and families.

Little headway, that is, until a few day ago, when DC schools’ Chancellor Michelle Rhee fired Hart principal Kisha Webster, supplanting her with a central office administrator. In recent coverage of this sure-to-be controversial firing, the Post gives Ms. Webster’s take on the firing.  Claiming to have been “set up,” Ms. Webster blamed her downfall on a lack of resources, second-tier teachers, and an unsupportive central office. She then went on to assert that publicity led to her demise. She is quoted here as saying: "If I had been able to keep things quiet, I'd still be [at Hart].”

Never underestimate the power of a headline. It seems, at least in the case of Hart, DCPS is more responsive to external coverage than internal warning signs and threats of disaster. From what I can gather, the problems plaguing Hart had taken root long before Webster took over in September. And DCPS knew of Hart’s struggles and should have done a better job of supporting the school and its staff from the beginning, rather than waiting for a Post exposé to prompt swift action. Regardless of central office’s failure to intervene, as school leader, Webster had a responsibility to create a safe atmosphere and climate of learning. And it is apparent that in her short tenure, neither goal was accomplished.

Hopefully, the lesson learned from Hart is that DCPS can no longer afford to reactively put out fires across the district. A change in principal may have the desired effect of ending the recent surge in violence, but unless it is coupled with better policies and practices, we’re unlikely to see systemic improvement. A well-developed and thoughtful set of policies should be crafted and put into place in each school, accompanied by a cadre of prevention programs.

Unlike a decade ago, we now have considerable evidence for successful, innovative school-based prevention programs that enhance the ability of schools to function properly, and, in turn, reduce incidences of violence and discipline considerably. See here and here for some good work being done on this front. Let’s not wait for the next Post headline to begin exploring these programs and implementing them in our schools. Instead, let’s think about some ways in which we can use the evidence base to inform both and practitioners and policy makers. 

The research around school-based prevention programs suggests that more important than specific programs or curricula are the principles upon which effective strategies are based and the fidelity with which they are implemented.  Schools can decrease problem behavior by organizing and managing themselves effectively, creating environments that support prosocial behavior, instituting clear rules and expectations, and creating structures and supports that help administrators, teachers, and students work together to meet those expectations. Yet these supports are often symbolic (e.g. behavioral contracts) or entirely lacking in the most troubled districts. It is encouraging to know that in some districts (like D.C., according to a draft five-year action plan), strategies are underway to bolster Student Support Teams (SSTs) to better coordinate academic and behavioral interventions for at-risk students. I’d like to see SSTs expanded to become a more universal approach, especially in urban districts where the vast majority of kids can be categorized as at-risk.

From a policy standpoint, more dollars should be appropriated for school-based prevention – despite the relative cost-effectiveness and proven success of school-based programs, they only receive an estimated $6-$8 per pupil per year in federal expenditures, not enough to be widely and properly implemented. Federal dollars should be targeted towards high-need districts in low-income, urban communities that exhibit a disproportionately high rate of violent and disruptive behavior and yet have the least capacity for solving such problems. And because research suggests that the quality of implementation is at least as important as the type of program, monitoring and technical assistance to districts and schools should be ramped up. 

- Posted by guestblogger Sara Yonker

Dispatch from the Front Lines

Policy types debate whether school systems should try to fix failing public schools or close them and replace them with charters or other types of new schools. Answer: it's a lot easier to create a constructive school culture with a new team than having to change the hearts and minds of existing staff. But sometimes the reality on the ground, to use the cliche, makes school reform a little trickier. In Chicago, school authories have opted for trying to turn around six failing high schools rather than closing them down because moving students to new schools would cause them to cross gang boundaries.

Secretary Spellings, Blogger

Margaret Spellings, the United States Secretary of Education, is guest-blogging today at our fellow institutional blog, Eduwonk. (She must have missed the memo that bloggers are all a bunch of unshaven, pajama-wearing D&D players living in their parents' basement.) Education Sector is, of course, very pleased to host her discussion of education and technology. On a larger note related to all the current discussions of the new administration's education policies: I think the Secretary and her staff have done some important work over the last three years in the higher education realm and I hope it continues in the future. 

Perspective

Alan Odden says that Michelle Rhee's proposal to give DC teachers the option to trade job security for a lot more money, if they want to, or not, if they don't want to, "would raise eyebrows everywhere, because that would be a gargantuan change."

The only real eyebrow-raising element of this is that it's national news. Here you have someone who was brought in to reform an organization that everyone--everyone--agrees was terribly dysfunctional, and her response has been, in part, to identify the employees who aren't doing a good job and replace them with better employees. This isn't just obvious and rational; it's ordinary. It happens all the time, particularly in organizations that are labor-dependant. And that's a good thing, because we're all much worse off when low-performing organizations stay chronically unreformed and low-performing. 

Which raises a broader point: from a p.r. standpoint, taking a stand here and now on this issue is a terrible decision for the American Federation of Teachers.  Teachers have a very good case to make when they say that, compared to other professionals with comparable levels of education who work at similarly difficult jobs, they're underpaid, disrespected, and forced to work in conditions that most people wouldn't tolerate. Fair compensation, dignity, and a basic quality of working life are things that all people understand and have a right to expect in their jobs.

Permanent job security regardless of performance, by contrast, is enjoyed by a small and shrinking portion of the workforce. Most people can't relate to tenure. They've never had it, they never will, and they understand why. Drawing the line there is a strategy of alienation for a group that has fewer friends than it understands and fewer than it needs. 

The Myth Continues

Nicholas Kristof's column today urges President-elect Obama to move education to the front of the policy line. We're all for that here at The Quick and the Ed. But Kristof commits a major sin by repeating the line, that we've twice debunked, that the current generation is the first to have less education than their parents. The line is false both in rhetoric and fact.

I understand Kristof's pain. He's trying to create a crisis where one doesn't exist so as to transfer energy into an important issue. Realistically, our schools as a whole aren't bad; they're just kind of...flat. Take some time to look at the original data where the claim comes from, and here's what you'll find:

• the United States had and continues to have a very high percentage of its adults completing a higher education degree.
• while that percentage has crawled upwards here in the States, other countries have cleared it. Where once we were first, we're now tenth.
• our high school graduation rate has fluctuated between 70 and 75 since at least 1995. Some years are better, some are worse.
• many of our adults acquire GEDs over time, boosting the percentage of the population with a high school diploma. That rate hovers around 87 percent for all generations.
• college graduates in the US continue to have success finding employment for themselves, creating it for others, and capitalizing on their credentials to boost income.

The scariest statistic is not the one Kristof repeats. It is that we have one of the highest rates of college entry, but we have yet to figure out how to get them through college:

Only 54% of entrants to higher education in the United States obtain a degree. Along with New Zealand, this is the lowest survival rate among OECD countries, where the average is 71% and as high as 91% as in Japan.

Our Comics-Loving President-Elect

As many others have noted, the Huffington Post reports that president-elect Obama "collects Spider-man and Conan the Barbarian comics." This is, of course, welcome news for right-thinking people everywhere. But it also raises more important questions that it answers. Which Spider-man and Conan the Barbarian comics? Some are pretty easy to guess--all Conan comics begin with the classic 1970s Roy Thomas / Barry Windsor-Smith run at Marvel, which arguably remains unequalled. Kurt Busiek's 2003-2006 stint for Dark Horse also stands out. But there were a lot of comics in between. Was Obama picking up the oversize, black and white Savage Sword of Conan back in the 80s? Is he re-reading old issues of King Conan for insights on how Conan dealt with the heavy burden of leadership after finally ascending to the throne? James Earl Jones as Thulsa Doom: inspired or ridiculous? These are important things to know. Similarly, will our future leader do the right thing and officially denounce the Spider-clone as the most egregious retcon in all of comics history? He's old enough to remember when it was big news that Peter Parker married Mary Jane (only to be un-married earlier this year in another retroactive continuity re-boot, ugh)--can he honestly say that his position on that has been consistent over the years? Did he publicly denounce the black suit, on the record, when it was first introduced post-Secret Wars, or did he only follow popular sentiment later on? Todd McFarlane goosed sales, but did he ultimately leave the character better or worse off? I assume he's pointing Malia and Sasha toward Ultimate Spider-man, if not an explanation on his thinking would be helpful. Frankly, the fact that we don't have answers here suggests a breakdown of the mainstream media's pre-election vetting process. 

All Deliberate Speed

From a New York Times article about a small Lousiana high school that may soon cease to exist:

This tiny, rural town in Cajun country is struggling again to find its racial equilibrium. A 43-year-old desegregation case remained unresolved on Nov. 4 as voters narrowly rejected a property tax increase to build a new Ville Platte High, which has faced decades of neglect since white flight accompanied integration in 1969.

President-elect Obama's historic victory has been frequently described over the past week as the culmination of a long struggle for civil rights that began in the 1950s with historic court decisions like Brown v. Board. And there is great truth in this. But it's worth remembering that, 54 years and counting since the Supreme Court ordered schools to be desegregated "with all deliberate speed," we're still not finished desegregating. Promises made to children in the middle of the 20th century remain unfulfilled for their grandchildren. There's a whole division within the Justice Department full of lawyers who travel to areas, mostly in the rural South, where educational opportunity remains divided along racial lines. There is still a lot of work to be done. 

Magnitude

Expanding on something I alluded to in the post below about training and certifying teachers vs. attracting and selecting them: the way these issues are often discussed reveals one of the weak spots in the way research is applied to policy. Essentially, people don't properly account for differences in magnitude.

All rational people want education policy to be informed by the results of objective research and inquiry, conducted using transparent, widely-accepted methods. (In a lot of ways, this is just a restatement of the definition of "rational person.") The kind of analysis that informs policy tends to involve testing hypotheses and identifying differences. Consider randomized control trials, the "gold standard" of research, which increasingly are being used in education. You randomly assign subjects to two groups, apply a treatment to one of them, and see if some pre-defined results vary. If the variance is large enough to meet agreed-upon levels of statistical significance, you conclude that the treatment had an effect. The same basic approach, even without the randomized assignment, is used to tackle other questions--like, for example, the effect of certification on teacher effectiveness.

So let's take another look at that chart with the three overlapping bell curves, one showing the distribution of teacher effectiveness among certified teachers, one showing alt-cert teachers, and one showing uncertified teachers.

If you squint your eyes and look carefully, you'll note that the solid line--the certified teachers--is slightly to the right of (with teachers increasing in effectiveness as they move from left to right along the scale) the other two. In other words, the population of certified teachers was slightly more effective. If the populations were large enough--if, say, they represented every teacher in America--that difference might be statistically significant. It might even be significant beyond a shadow of a statistical doubt, at the .0001 level or what have you. And if that were the case, the results of the research would very likely be translated for lay readers and policymakers into something like this: "A study from [fill in the blank] found that certified teachers are more effective than non-certified teachers." And the natural policy response would be that we should make sure that all students are taught be certified teachers--they're better, after all--and we should invest in and strengthen certification programs, since they work.

The problem, of course, is that if you put your common sense hat on and think for 30 seconds about what that graph actually shows, you naturally conclude that such a policy response would be insane. The difference between certified and uncertified may indeed be statistically signifcant, but it is a tiny statistically signficant difference, one that pales in comparison to the extent to which teachers vary within those populations, and the extent to which the populations are the same. To state the obvious: the most important differences tend to be large, not small.

The vernacular of research and policymaking, in other words, is insufficiently precise about and sensitive to differences in magnitude. All significant differences are not equally signficant just because they met the same test of signficance, but they are often thought of as such.

A lot to Say About Testing

We have a lot of comments to the testing and 21st century post from Friday. I'm also getting a lot of emails on this--from teachers who say they are already teaching "21st century skills" and what's new about this anyway?, from district officials saying it's too expensive to assess these skills (isn't it?), from content and curriculum specialists concerned that the focus on 21st century skills will distract from efforts to teach reading and math, and many others. Tag cloud to the side is thanks to Dennis Richards, who wants to know what kinds of teachers and teaching tools it will take to teach these skills to all kids. Much more on all of this at the discussion here. Join in: ask questions, post comments.

g(t)?

Writing in the Boston Globe (per Matt Yglesias), Harvard economist Edward Glaeser cites Tom Kane's research on teacher quality, saying:

The first step toward improving teacher quality is to attract more talented teachers. The second step is to improve teacher selection on the job, promoting the best and encouraging the worst to help society in some other way.

The key words are "attract" and "selection." But just as important are the missing words: "certify" and "train."

The conventional system for bringing teachers to classrooms relies on pre-service training and certification to ensure quality. Students are required to complete a state-approved training program, usually offered at a university, and then obtain state certification, which increasingly involves passing some kind of exam.

Tom Kane's research shows that the training and certification model is near-worthless as a means of effectively guaranteeing or differentiating teacher effectiveness. The paper he wrote with Robert Gordon and Douglas Staiger for the Hamilton Project in 2006 is well worth reading in full, but you can get the gist just by looking at Figure 1, on page 8.

We have three bell curves, one made of a solid line, one a dashed line, and one a dotted line. Each of the lines shows the distribution of a group of elementary teachers in Los Angeles based on classroom effectiveness, as measured by their students' math test score gains, after controlling for baseline scores, student demographics, and program participation. The solid line shows the population of traditionally certified teachers, the dashed line alternatively certified teachers, and the dotted line uncertified teachers.

The graph shows two crucial things. First, the bell curves are wide--there is a lot of variation in effectiveness among teachers within each population. Students with the most effective teachers gain ten percentile points or more in one year; students with the least effective teachers lose ten percentile points. To put that in perspective, the cumulative black - white achievement gap nationwide is about 35 percentile points.

Second, the curves are, for all intents and purposes, exactly the same. There's a tiny bit of daylight between the certified teacher curve and the other two. But it's dwarfed by vast differences in effectiveness within the populations. The long, tedious argument within the education community about the virtues of training and certification amounts to debating how many angels can dance on the head of a pin. As near as anyone can tell, there is no way to figure out ahead of time who is going to be an effective teacher and who is not via traditional training and filtering processes. The best way--arguably, the only way--to figure who is an effective teacher turns out to be letting people teach and seeing if they're effective.

The logic of this quickly leads to a very different approach toward teacher quality--not train and certify, but, as Glaeser writes, attract and select. Do whatever you can to attract smart, motivated, talented people into the profession, and then impose a ruthless screening process based on results in the first few years. That doesn't mean you shouldn't try to make those first years as supportive as possible. Yet, seen through the lens of those overlapping bell curves, recent disappointing results from a study of induction models aren't surprising. Some students will suffer in the classrooms of the least effective teachers (academically at-risk students should be protected), but that's just as true of the current system.

The interesting thing is that the Kane / Staiger / Gordon paper came out two-and-a-half years ago. I'm not aware of any serious challenges to the data, methods, or conclusions. As far as education policy papers go, it received a lot of attention, getting written up in the Wall Street Journal here and being name-checked by Nicholas Kristoff here and Bob Herbert here. And of course it garnered hyperbolic praise on this blog, way back when.

Yet the education world keeps right on going with the traditional system. There's a lesson here in human psychology: when evidence is revealed that pushes people completely outside of their established frame of reference, a little mental "na-na-na-na-na-" device turns on and they simply proceed as if it doesn't exist. I suspect the dissonance is particularly jarring for education professionals, since the implication is that, paradoxically, there are severe limits on the extent to which good teaching can be taught.

The findings--which are supported by a host of other teacher effectiveness studies--also raise an intriguing question: If variation in teacher effectiveness isn't attributable to anything we can currently observe, then what's driving it? Is there some kind of innate teaching ability, a sort of g(t)? That combined with more generalized qualities like motivation, intelligence, and the experience of having been well-taught?

I don't pretend to know the answer, but I know that the empirical justification for continuing the current way of doing business in teacher policy is quickly eroding, and that researchers would serve education better by focusing more of their attentions on exploring the source of large differences rather than small ones.

Voice Your Thoughts

Don't forget to read Education Sector Senior Policy Analyst Elena Silva's latest report, "Measuring Skills for the 21st Century." A teaser:

When ninth-graders at St. Andrew's School, a private boarding school in Middletown, Delaware, sat down last year to take the school’s College Work and Readiness Assessment (CWRA), they faced the sort of problems that often stump city officials and administrators, but rarely show up on standardized tests, such as how to manage traffic congestion caused by population growth. "I proposed a new transportation system for the city," said one student describing his answer. "It's expensive, but it will cut pollution."

Students were given research reports, budgets, and other documents to help draft their answers, and they were expected to demonstrate proficiency in subjects like reading and math as well as mastery of broader and more sophisticated skills like evaluating and analyzing information and thinking creatively about how to apply information to real-world problems.

Not many public school students take assessments like the CWRA. Instead, most students take tests that are primarily multiple-choice measures of lower-level skills in reading and math, such as the ability to recall or restate facts from reading passages and to handle arithmetic-based questions in math. These types of tests are useful for meeting the proficiency goals of the federal No Child Left Behind Act (NCLB) and state accountability systems. But leaders in business, government, and higher education are increasingly emphatic in saying that such tests don’t do enough. The intellectual demands of 21st century work, today's leaders say, require assessments that measure more advanced skills, 21st century skills. Today, they say, college students, workers, and citizens must be able to solve multifaceted problems by thinking creatively and generating original ideas from multiple sources of information—and tests must measure students' capacity to do such work....

New assessments like the CWRA, however, illustrate that the skills that really matter for the 21st century—the ability to think creatively and to evaluate and analyze information—can be measured accurately and in a common and comparable way. These emergent models also demonstrate the potential to measure these complex thinking skills at the same time that we measure a student's mastery of core content or basic skills and knowledge. There is, then, no need for more tests to measure advanced skills. Rather, there is a need for better tests that measure more of the skills students' need to succeed today.

Please join us for a week-long discussion between Silva, Eva Baker, a professor at UCLA's Graduate School of Education and Information Studies, and Paul Curtis, chief academic officer of New Technology Foundation.