Poverty, Schooling, and the Urban NAEP

One of the foundational arguments in education centers on poverty and schooling. All reasonable people agree that poverty has a negative influence on education, just as all reasonable people agree that quality schooling has a positive influence. The point of argument is how much these things matter, relative to one another. Some people think the negative effects of poverty overwhelm anything schools--even good schools--can do, while others believe schools make a big difference in how much poor students learn.

I tend toward the latter camp, and I think one of the strongest pieces of evidence lies with the National Assessment of Education Progress, which in recent years has been expanded from the national and state levels to include a group of large city school districts like New York, Boston, Los Angeles, Chicago, and DC. The 2007 results were released today (math here, reading here). As in previous years, they indicate that different school districts achieve very different results for poor students.

Here, for example, is the percent of low-income (eligible for the National School Lunch program) 4th graders who are "proficient" in math:

New York City: 31%
Boston: 24%
Charlotte: 23%
Austin: 22%
Houston: 22%
San Diego: 22%
Los Angeles: 15%
Chicago: 12%
Cleveland: 10%
DC: 7%

Low-income fourth graders in New York City are more than four times as likely as low-income students in DC to be proficient in math, twice as likely as Los Angeles, and significantly better than all the rest. The NAEP proficiency standard is unusually tough, but significant differences persist when we look at the percent of students who met the much easier, "Basic" standard:

Charlotte: 77%
Houston: 77%
New York City: 77%
Boston: 75%
Austin: 74%
San Diego: 65%
Los Angeles: 55%
Chicago: 54%
Cleveland: 53%
Atlanta: 52%
DC: 43%

Now, one might reasonably speculate that poverty concentration plays a big role here, that a district with a relatively small number of poor kids would have an easier time helping those kids than a district where poverty is rampant. Except that doesn't seem to be the case; New York City and DC, which bookend these lists, have almost exactly the same percentage of students living below the poverty line, 29%. Charlotte, which is different from the rest of the cities in being a unified urban-suburban district, has by far the lowest poverty rate on the list, 14%, yet does no better than (and in some cases worse than) cities with many more poor children. This is true in other grades (8th) and subjects (reading) as well.

The real source of these large differences in performance is, pretty obviously, that some of these districts are just a lot better than others. New York City, Boston, and Houston, which are consistently in the top half of cities on the NAEP, have all won the Broad Prize for Urban Education in recent years. The cities in the bottom half haven't, and for good reason.

How do these differences stack up against the overall effects of poverty? Below, see the difference between the performance of poor and non-poor 4th graders nationwide on the 2007 NAEP math test, measured three different ways: percent proficient, percent basic, and average scale score (for an explanation of how the scale scores are calculated, and everything else you need to know about NAEP, see this recent Ed Sector "Explainer")

Percent Proficient: Non-poor (53) minus Poor (22) = 31 percentage points
Percent Basic: Non-poor (91) minus Poor (70) = 21 percentage points
Scale Score: Non-poor (249) minus Poor (227) = 22 scale score points.

By contrast, the differences between the highest- and lowest-scoring cities on those three measures were 24 percentage points, 34 percentage points, and 27 scale score points.

Very comparable, in other words. The scale score difference between poor kids in DC and poor kids in New York is bigger than the difference between poor kids nationwide and non-poor kids nationwide. There's no single reason for this; success (or lack thereof) in schooling is a function of many things--management, resources, personnel, etc. etc. It all adds up.

Moreover, I think these city NAEP numbers underestimate the effect of differences of schooling quality on poor students, because they don't represent the whole range of quality. New York City and Boston (the last two Broad Prize winners) are manifestly better school districts than DC, Cleveland, etc. But nobody thinks they're as good as they could be--many, many problems remain to be solved. NYC just got a big influx of money from a school funding lawsuit, for example, which will hopefully lead to further improvement, and there's a lot more work to be done in many other areas.

Poverty creates terrible problems, but schools can help--a lot.

Update: Matt Yglesias provides needed chartification here, while Ezra Klein weighs in here. There's also a long post on Kansas City in Ezra's comments section that's worth reading, refuting the idea that because Kansas City wasted vast amounts of money trying to help urban children, ipso facto school funding doesn't matter and other districts can't do better.

Update 2: If you you want to ask NCES Associate Commissioner Peggy Carr questions about this data, send questions to tuda2007questions@ed.gov until Monday at noon. Dr. Carr will post her answers on Nov. 20 at 3 p.m at http://nces.ed.gov/nationsreportcard/2007tudachat.asp.