The national-security expert Gregory Treverton has famously made a distinction between puzzles and mysteries. Osama bin Laden's whereabouts are a puzzle. We can't find him because we don't have enough information. The key to the puzzle will probably come from someone close to bin Laden, and until we can find that source bin Laden will remain at large.
The problem of what would happen in Iraq after the toppling of Saddam Hussein was, by contrast, a mystery. It wasn't a question that had a simple, factual answer. Mysteries require judgments and the assessment of uncertainty, and the hard part is not that we have too little information but that we have too much."
The foreign intelligence community has been struggling, according to Treverton, because the nature of its job is changing. During the Cold War, it spent a lot of time solving puzzles. In this day and age, it is increasingly charged with solving mysteries, which requires a different set of skills and attitudes.
On some level, the same kind of thing is happening in public policy. There's a lot we don't know about education. But we certainly know much more know than we ever knew before. Formulating good policy, therefore, is increasingly becoming a function of making sense of the information we have, not finding new information to consider. In other words, solving mysteries instead of puzzles.
The research and think tank spheres engage in both of these activities. But their underlying value systems are different. The most important thing you can do in the research sphere is create new knowledge, add to the collective sum of human understanding. The think tank sphere does some of this, but for reasons of design and purpose it tends to put more emphasis on interpreting information and translating it into specific policies. The end product of research lends itself to puzzle-solving, while think tanks tend to focus on mysteries.
You can see how these value systems clash by observing the way people criticize think tank reports. For example, a few weeks ago, Education Sector released a new report called "Frozen Assets," which used data and findings originally published by many other researchers to do two things: (1) Identify a group of common teacher contract provisions that, according to research, have a weak or inconsistent relationship with student learning, and (2) Estimate how much those provisions costs -- $77 billion nationwide.
Teachers union critics of the report immediately said, "There is no new or original research here." Implicit in this criticism is the idea that new and original research is inherently more valuable and worthy of discussion, and that by lacking those elements the paper is unworthy of serious consideration by definition. This is clearly an argument that resonates, otherwise critics wouldn't use it. "Move along," they're saying. "Nothing new to see here."
This misses the point entirely. "Frozen Assets," like a lot of think tank work, is not about solving puzzles by finding new information. It's about finding new meaning in the information we already have. In this case, that consists of connecting research about how various school factors and policies do--or do not--improve student learning with research and analysis about how much related contract provisions cost, in a way that makes sense to policymakers and the general public. This kind of mystery-solving isn't inherently more or less valuable than puzzle-solving. It's just different, and deserves to criticized on its own terms.
The larger issue lies with the fact that as the amount of freely-available information grows ever larger and the world becomes more complex, education policy will become more like foreign intelligence. The key issues will tend to be mysteries, not puzzles, and influence will accrue to the mystery-solvers.
Many people in the research and academic communities find this troubling, for some completely valid reasons. The media, public, and policymakers don't always have the time or expertise needed to figure out which reports and information sources are truly credible. The academic community addresses this problem with strong internal controls based on credentialling and peer review. But as we learned during the recent Abramoff scandals, anyone can hang out a shingle and call themselves a think tank, all you need is a Web site and an important-sounding name. Remember the "American International Center," the "premiere international think tank" run by a lifeguard in Rehoboth Beach?
That said, there are also many really good think tanks and NGOs. They're meeting the growing demand for mystery-solving, in a way that's purposefully designed to change policy. That doesn't excuse them from any and all scrutiny of the quality or integrity of their work. But that scrutiny is more meaningful when it focuses on what their work actually is.
Update 1: Alexander Russo weighs in here.
No comments:
Post a Comment