Scalpel

Mike Petrilli mischaracterizes what it means for a program to be labeled "ineffective" by the Office of Management and Budget's Program Assessment Rating Tool (PART). His argument starts with the presidential debate last week with this exchange:

LEHRER: What I'm trying to get at this is this. Excuse me if I may, senator. Trying to get at that you all -- one of you is going to be the president of the United States come January. At the -- in the middle of a huge financial crisis that is yet to be resolved. And what I'm trying to get at is how this is going to affect you not in very specific -- small ways but in major ways and the approach to take as to the presidency.

MCCAIN: How about a spending freeze on everything but defense, veteran affairs and entitlement programs.

LEHRER: Spending freeze?

MCCAIN: I think we ought to seriously consider with the exceptions the caring of veterans national defense and several other vital issues.

LEHRER: Would you go for that?

OBAMA: The problem with a spending freeze is you're using a hatchet where you need a scalpel.

When Obama went on Face the Nation and elaborated that there are government programs that do not work, Mike decided to give Obama some scalpel help and used OMB PART scores to determine which Education programs should be cut. Besides PART itself possibly not making it into the next administration, here's why the scores should not be used in the way Mike suggests:

1. PART treats every program equally, so the entire Bureau of Labor Statistics is one program with a budget of $500 million. Education is split into many tiny programs, so the BLS is graded on the same curve as the $1 million B.J. Stupak Olympic Scholarship Program. The Individuals with Disabilities Education Act (IDEA) is not one program, but seven. No other agency is hit as hard as Education in this way.
2. Government programs, as Mike well knows, are often shackled with poor designs. A compromise here or there makes the original intent of the program nearly impossible to achieve. PART assesses how well the program accomplishes its goals, but if its goals are conflicting or unclear, let alone flat-out impossible, the program earns a bad score. See this review of the federal Perkins loan program, designed for needy college students:

   The program's institutional allocation formula (i.e., how much program funding is given to each school to offer Perkins aid) is designed to heavily benefit postsecondary institutions that have participated in Campus-Based programs for a long time, at the expense of more recent entrants or new applicants. Since these longstanding institutions do not have a higher proportion of needy students, this allocation formula tends to limit the program's ability to target resources the neediest beneficiaries.
   

3. Education programs are not the only ones receiving "ineffective" ratings from PART. Using this "scalpel," we would also cut Amtrak, the Earned Income Tax Credit, Americorps, Veterans Disability Compensation and Veterans Home Loans, and the Air Force Base Operations & Support. This last one alone is funded at twice the level of all the programs Mike lists. It fails because, "The overall program does not have long-term, outcome-based performance measures. Program elements do have performance measures, though they are often input or output oriented rather than focused on outcomes that directly and meaningfully support the program's purpose."
   
4. PART scores are binary, meaning OMB managers must answer either "yes" or "no" to questions about program efficacy. There's no room for flexibility whatsoever. Imagine an agency that juuuust fails on every measure. It would receive a score of 0. An agency that gets even one yes, no matter how many horrendous other failings it has, would have a higher score.
   

If you want to know more about PART scores, see here.