More Than Checking a Box

TEACH Grants, the new grant/loan program (some have called it a ‘groan’) created in the College Cost and Reduction Act is through the rule-making process and is now open for comment (hat tip to Sara Godrick-Rab at the University of Wisconsin who has been feeding us great tips on this issue). According to a January report in the Chronicle of Higher Education, the program is proving popular among students, with 30,000 checking a box on their financial aid forms indicating an interest in the program.

But it will take a lot more than checking a box, for both students and financial aid officers, to make sure that this program works as intended and doesn’t end up saddling students with thousands of dollars in unexpected debt.

The intentions of the program are great: to provide an incentive in the form of financial aid to high-achieving students committed to teaching in a high-risk school and in a high-need subject area. And it wouldn’t work to simply give out grants and hope that students follow through—there needs to be some policy mechanism to encourage students to stick with the program. Under the TEACH Grants program students receive grants of up to $4,000 per year in exchange for promising to follow through on the required teaching commitment, but if they don’t follow through the grants are converted (permanently) to loans with interest accruing from the day the grants were provided.

With many requirements (students must teach four years out of eight after graduating, must teach in a high-need subject area and in a school serving low-income students, and students need to check-in to confirm their employment every year) to fulfill the commitment, it will take some excellent counseling from colleges to make sure that each student understands what is required of them, understands their odds of fulfilling the requirement (currently, the Department of Education expects only 20 percent of students to complete the program) and understands the consequences of failing to complete the program.

Counseling for students is required under the TEACH Grants program, but given that higher education administrators have expressed concerns about the added administrative burden of the program, there’s no guarantee students will receive the kind of comprehensive, continual counseling they will need. And without that counseling, we could see many students who thought they were receiving a grant, as the TEACH Grants name implies, but are suddenly getting a bill for additional student loan debt.

Measurement Man

Dan Koretz, a professor of education at the Harvard Graduate School of Education and a member of Education Sector's Research Advisory Board, has a new book out from Harvard University Press called Measuring Up: What Educational Testing Really Tells Us. It's a great primer, written in plain English, on the important but often opaque subject of standardized testing. Koretz illuminates the complexities of "sampling error," "statistical significance," "reliability," and a host of other tough terms. And he ends with a chapter on "the sensible uses of tests." At a time when standardized tests are the single more influential force in at every level of education, roadmaps like Measuring Up are a valuable comodity.

Law School Confidential

My wife is a lawyer, and so we get the ABA Journal in the mail every month. As a rule, I never read it, because I don't even have time to read all the magazines we get that cover issues I'm interested in knowing more about. But this month's issue features a big picture on the front cover of someone I happen to know: Bob Morse, czar of the U.S. News & World Report college rankings.

I've confined most of my rankings analyses to four-year and two-year colleges, but the law school story is similar: the rankings are very influential, which makes law school deans very anxious and upset. The piece kicks off with the story of the dean of the University of Houston law school, who managed the school through the aftermath of Tropical Storm Allison, only to resign six years later when Houston dropped five spots on the list.

There are many good reasons to criticize the U.S. News methodology for ranking four-year institutions. But it strikes me that if any segment of higher education has the least cause to complain here, it's law schools, which completely embrace the logic and justice of rankings in the way they evaluate their own students. The legal profession in general is one pecking order after another, ranking everything from schools and students to law firms and judicial clerkships in hierarchies of great consequence. The letter grades that law schools give out hardly matter; performance relative to--i.e. ranked against--other students is everything.

It was only a few years ago that my wife expended a great deal of effort on and experienced a considerable amount of anxiety about the issue of whether she would graduate in the top ten percent of her class at Georgetown Law. (She did.) She was right to worry; some judges explicitly say they won't consider clerkship applications from applicants without this credential. It's particularly important to distinguish yourself if you're coming from a school like Georgetown, which is in the lower half of what most people consider the best schools.

But while she was anxious, much like a law school dean waiting for the new rankings list, she never thought the student rankings process was unfair. Far from it--it's close to a pure meritocracy, giving every student something that at least resembles an equal shot at distinction and opportunity, regardless of where you come from or who you know. That's all she wanted, or expected, and it worked.

The point being, there's nothing wrong with rankings per se, only bad rankings. And the article actually doesn't provide a lot in the way of persuasive critiques of the U.S News law school method. Numerous law schools complain that other law schools lie and cheat on the data they submit, but that's not an argument against rankings, it's evidence that some law schools are apparently run by liars and cheats. While I'm sure the rankings aren't perfect, their focus on selectivity, employment, and reputation seems reasonably appropriate for status- and job-focused colleges of law.

The article comes back to the former Houston dean at the end, where she says their employment placement services (job placement is an element of the rankings) were disrupted after the storm, negatively affecting their score. But that's just telling the truth, isn't it? I imagine any fair ranking of America's most livable cities would have knocked New Orleans down a few spots from 2005 to 2006. You can't shoot the messenger -- or in the case of law school rankings, Bob Morse.

The Wisdom of Galactica

With the conclusion of The Wire, the mantle of "best show on television" falls on Battlestar: Galactica, which bows for its fourth and final season on the Sci-Fi channel tonight. Like The Wire, Galactica features exceptionally strong writing and acting, and has been freed by its relatively small cable audience to explore issues with a depth and sophistication that you never see on network television. Galactica is similarly sustained by focusing on a single, defining theme. On The Wire, it was the dehumanizing effect of modern institutions. On Galactica, it's the absolutely corrupting influence of war. 

There are few more dangerous illusions than the notion that people can survive war unscathed as long as the cause is just. The humans on Galactica are fighting the most just war imaginable: a race of evil cyborgs ambushes a relatively peaceful human society living on a group of planets in the distant future, annihilating billions of people with nuclear weapons and chasing a small band of survivors through space toward the only refuge left to them: the long-lost homeland of Earth. 

Yet instead of making the show about the triumph of the human spirit, Galactica's creators show how that spirit is crushed, sometimes slowly, sometimes not, under the weight of the impossible moral decisions war inevitably brings. That was the essence of Lee Adama's defense of Baltar's collaboration in the final episode of last season. "It was an impossible situation," he says. "When the Cylons arrived, what could he possibly do? What could anyone have done?" You see it in Sol Tigh, who rejected collaboration and ended up promoting suicide bombing, executing his own wife, and was left ruined as a result.  You see it in the pointed contrast between Adama's pilot-episode speech to the troops and Admiral Cain's parallel speech from the between-season movie, Razor, where she gave into the need for revenge and was ultimately undone by it. 

Like any show, the beats and themes on Galactica have started to become more obvious as time goes on; it wasn't hard to guess where Kendra Shaw came from on Razor and where she would go in the end. So while I'll miss the show, I'm glad they're wrapping it up this year. As the creators have made clear, even the best of women and men can only survive war's inhumanity for so long. In this day and age, that can't be said enough. 

Notes on SES

Newly released interim report (pdf) from the Department of Education shows that the number of kids in chronically low-performing schools who are getting free tutoring through NCLB’s Supplemental Educational Services (SES) is rising but still only reaches 17 percent of all those eligible. Same kids also have the option to switch from their school to a better performing school but only a paltry one percent of eligible students are taking advantage of this. Why such low participation rates for SES and public school choice? Mostly because parents don’t seem to know or understand that they have these options (only half of those surveyed said they had been notified about the SES option, and less than a 1/3 said they’d heard about the public school choice option). [By the way, it's increasingly unlikely that SES will survive NCLB reauthorization, at least not without big changes. It's ironic, really, that as an NCLB provision (albeit now the ugly stepsister) SES is not really held accountable in any meaningful way for student outcomes. So changes should ensure that we can at least tell whether or not SES or SES providers (which have tripled in number in the past 3 years) actually improve learning opportunities for kids. Proposals to get rid of SES or lump it together into another pot don't address the participation/access issue or the questionable quality issue.]

The A.J. Soprano Factor

This discussion over at the The Atlantic and at G-Spot about merit and college admissions seems as good a reason as any to reprint, in its (relatively short) entirety, a policy brief co-blogger Erin Dillon and I wrote a few months ago:

Tony Soprano, patriarch of the eponymous crime family on the HBO drama "The Sopranos," had a son named Anthony Jr.—A.J. for short. A.J. was a terrible high school student. None too bright to begin with, he skipped class, experimented with drugs and alcohol, vandalized school property, and eventually got expelled. Nonetheless, A.J. went to college. His mother, Carmela, devoted countless hours to monitoring his grades, scouting for colleges that might accept him, buttonholing guidance counselors, managing the application process, and constantly reminding him of the importance of higher education. For Carmela, the question was not if her son would go to college, but where. In the end, she got her wish, practically dragging A.J. to school.

A.J. Soprano is a fictional character, but the intersection of social class and college access that he illustrates is all too real. Despite a wide range of programs dedicated to increasing access to higher education for all students, college is far more accessible for upper-income students like A.J. than for students of modest means. Chart 1, which is based on newly available data from the U.S. Department of Education's ongoing Education Longitudinal Survey, illustrates this fact.


All of the students in the survey were high school sophomores in 2002 and were given standardized tests in reading and math at that time. Each bar on Chart 1 shows the percent of students with different test scores and levels of family income who had enrolled in college by 2006, two years after their expected high school graduation.

The fourth bar on the graph represents the A.J. Sopranos of the world, those who scored in the bottom 25 percent (the first achievement quartile) on standardized tests as high school sophomores and came from families earning more than $100,000 per year. Despite their academic shortcomings, 58.4 percent of these students went on to college. For high-income students in the second achievement quartile—still below the median—the college-going rate was significantly higher, 85.3 percent.

This is a higher rate than that for those directly opposite A.J.—students from the highest achievement quartile and the lowest income level, less than $20,000 per year. 80.3 percent of these meritorious poor students went to college, which means that nearly 20 percent did not. High-achieving wealthy students, in contrast, went to college at a 96.2 percent clip. In other words, high-achieving poor students are five times more likely than high-achieving rich students to skip college in the first two years after high school.

Some observers have argued that there is no real college-access problem for the brightest students. Charles Murray of the American Enterprise Institute recently wrote in The Wall Street Journal that "No evidence indicates that the nation has many children with IQs above 120 [the top 10 percent] who are not given an opportunity for higher education. The university system has also become efficient in shipping large numbers of the most talented high school graduates to the most prestigious schools."

Chart 2 suggests otherwise. Aptitude and achievement play a significant role in college-going, but class matters too. Chart 2 is based on the same group of students as Chart 1, but shows only the percent of students whose first college was a highly selective four-year institution.

The odds of a poor, low-achieving student going to a highly selective institution are 0.2 percent, practically non-existent. Among the A.J. Sopranos, however, 3.5 percent managed to sneak into an elite school, suggesting that admissions preferences for athletes, legacies, and the children of the rich are alive and well.

This contrast between students with similar test scores and different incomes persists across Chart 2. In each achievement quartile, students from the wealthiest families are far more likely than their similar-scoring peers to attend the kind of high-status college or university that often serves as a gateway to personal and professional success.

There are a host of factors contributing to this phenomenon, not all of them involving anxious mothers or college admissions committees giving a leg up to the scions of wealthy alumni. Low-income students frequently attend under-resourced high schools that don't provide good guidance counseling or college preparatory curricula. College tuition is rising much faster than available need-based financial aid, which may lead some college-ready poor students to believe that higher education is beyond their financial reach. But whatever the reasons, it's clear that equal access to college remains an unmet promise in America. When it comes to higher education, it's an advantage to be rich like A.J.

If you want to see the footnotes and standard errors, the original brief is here.

Supply-Side Education Policy, Continued

Expanding on yesterday's post: Tests can be thought of as a kind of tax on the public education system. They consume scarce resources that could be used for teaching purposes. The tens of millions of dollars a year states spend on standardized tests represent funds that could have been used to reduce class size or increase teacher pay. The same is true for the money needed to create data systems that track students who move between school districts, an essential component of calculating accurate high school graduation rates. More importantly, tests consume time, both in the actual taking and in pre-test preparation, reducing the hours available for instruction and thus, presumably, the amount that students learn.

But, as with taxes, tests are necessary because they create a resource--detailed, comparable information about school success--that serves the broad public interest. Tests are compulsory, not voluntary, because otherwise people would have an incentive to free-ride and enjoy the benefit without the tax. The trick is to strike the right balance of taxation, so the short-term negative consquences are minimized (they can't be eliminated) while the public benefit is maximized. If the public resources are used wisely, they support the overall quality of the system and thus the quality of education provided. Individually, the dry cleaner across the street from my house would benefit if he didn't have to pay taxes. But he's better off operating under a system of taxation, because it pays for the sidewalks and roads that bring customers to his door.

People don't like taxes. As a result, they're susceptible to arguments that taxes can be lowered without consequences. Yet many of loudest critiques of NCLB imply the need for more information resources, not less. Fair accountability systems, we are told, will measure value-added growth. They will take into account social studies, art, music, foreign language, critical thinking, creativity, parental satisfaction, etc., etc. It's true, accountability systems would be better if they measured these things. But they can't be measured for free. And they certainly can't be measured using fewer resources than we use today.

Arguments that we could fix this whole mess simply by having less of what people don't like (taxes) and more of what people do like (public resources) are dishonest. In the long run, they breed cynicism and distrust, because they teach people to expect a combination of taxes and services that's impossible to achieve.

Supply-Side Education Policy

As part of a Slate series of policy briefs for the next president, UVA law professor Jim Ryan offers a fix-NCLB agenda. But he fails to notice that his proposed solutions completely contradict one another.

We are told, for example, that because NCLB "requires schools to test the bejeezus out of elementary- and middle-school students in reading and math," we should "Administer fewer tests. National tests should be given less often, perhaps in only fourth, eighth, and 11th grades."

We are also told that "Current test results don't tell us all we need to know about schools" because they ignore subject like "social studies, history, literature, geography, art, and music," and that "What we can't tell from scores alone, because they don't tell us where students started or how much they progressed over the year, is the value that a particular teacher or school has added to a student's education." Therefore, "School quality should also be measured using value-added assessments, crediting schools that make exceptional progress with their students, regardless of where those students started."

Which is it? If the NCLB is to be faulted because it causes schools to "downplay if not ignore subjects not tested," how does fewer tests in the same subjects fix that problem? And you can't have value-added measures without annual testing, which Ryan wants to eliminate.

This is, to use a bloggy cliche, a classic "and a pony" policy agenda. NCLB can be improved, no doubt, but the people who wrote it weren't morons; there are some very real and difficult tradeoffs to contend with in formulating accountability policy, and one of them is the tension between the costs and burdens of assessment and the need for comprehensive information. This is the equivalent of promises to cut taxes and increase services. Call it supply-side education policy: the less we test, the more we'll know.

Madlibs!

You know you want them.

Graduation Day

As the Times reported this morning, Secretary Spellings announced a new policy today that all states must calculate high school graduation rates in the same way under NCLB. It's a good idea, and illustrates several important things about national standards and state / federal relations.

First, if you give states broad discretion to define how their educational success is measured, it's a sure bet that at least a few of them will, with a straight face, put forth definitions and standards that are so lax and self-serving that they'd be funny if they weren't so sad. For example, when you read the words "on-time high school graduation rate" you probably think it means "percent of students who start high school who graduate on time." You'd think this because you're not insane, or running a state Department of Education. But for years North Carolina has interpreted this phrase to mean "percent of students who graduate from high school who graduate on time."

To repeat: Instead of calculating graduation rates as (on-time graduates / entering freshmen), North Carolina has, until this year, calculated graduation rates as (on-time graduates / all graduates). So a high school could see two-thirds of its students drop out in the 10th grade to go work at Chick-Fil-A, but still post a 100% graduation rate as long as the remaining third graduated on time. Other states have adopted fun policies like reporting the percentage of 12th graders who graduate, etc., etc.

The second thing to consider is that this new policy is a kind of national standard. There are lots of people around town who will tell you that national educational standards are a total non-starter politically and will never happen. But that's not really true; the palatability of national standards depends, as it ought to, on what is being standardized. There's no logical reason for different states or localities to adopt different defintions of "high school graduation rate," a term that has only one logical meaning. That's why today's announcement probably won't create a massive hue and cry about federal power grabs and the homogenization of the public schools. Graduation rate means what it means.

The same logic can, and should, be applied to academic subjects. Should middle schools in Richmond, VA be teaching American history from 1860 to 1870 in the same way as their counterparts in Juneau, Alaska? Probably not. But for other subjects, particularly basic computational and language skills in the early grades, there's no earthly reason for 50 different sets of standards and assessments. I can't imagine there's ever been a parent who, upon discovering that their 2nd grader couldn''t add 2 and 3, said "no problem--we'll just move somewhere with different standards."

Empty Threats

I was having lunch with a friend last week, super-smart guy but not an education person, when he mentioned that Arizona was about to opt out of the No Child Left Behind Act. He was surprised when I said it was the first I'd heard of it, because here I'm supposed to be the person who gets paid to keep track of this stuff. So I went ahead and looked for it and sure enough: "The Arizona House of Representatives is on the verge of opting out of the controversial No Child Left Behind Act, President Bush's premier educational accomplishment."

Huh! Of course they'll have to get it passed the State Senate and the governor, but still...whoops, wait a minute...few grafs later..."Some of the support stems from a change [the author] added to the bill Wednesday. If the state would not reimburse local school districts for the amount of lost federal dollars, Arizona would stick with No Child Left Behind." Ah.

That amount being $600 million per year. In the state with the worst budget deficit in the nation.

So my policy of ignoring all state threats to boycott NCLB on the grounds that they never come to anything turned out to be solid after all; this is no different than the Arizona House passing a resolution expressing the sense of the membership that the Cardinals should stop losing so many football games.

It seems fair to say that over the past five years, the leaders of the 50 states have collectively weighed the consequences of (A) cutting school spending by 5% to 10%, (B) raising taxes or cutting spending on other things in order to raise an amount equal to 5% to 10% of school spending, or (C) implementing NCLB, and they've all decided that (C) is the best option. You may disagree, but that's what they believe.

Paying for Public School

It may seem strange to pay to attend a public school, but this article in Sunday's Washington Post reports on parents who do just that. In most public school choice arrangements additional costs are absorbed by the school district or state, but in some cases parents are willing to pay 'tuition' for their child to attend the public school of their choice. And apparently D.C. parents, who only have one, urban district to choose from, can opt to cross state borders to send their child to school in Maryland or Virginia. This option isn't cheap, though--Maryland's Bethesda-Chevy Chase High School costs $13,627 a year, a price most D.C. parents can't afford.