More Than Butts in Seats

Education Sector recently completed an extensive process looking at higher education accountability systems in all 50 states, the District of Columbia, and Puerto Rico. In part, we undertook the task to be able to answer comments like this one at the Chronicle of Higher Education:

As a former dean, I was responsible for collecting and reporting “outcome” data on both students and programs to the provost, who then reported it to the appropriate accrediting bodies. I am not aware of any attempt to use the data to inform policy decisions. To use the data in that way would have been completely inappropriate, since we only tended to collect data that was required, easily collectible, and fit neatly into a file for statistical analysis : butts in seats, before and after measures on very elementary standardized exams, student perceptions of faculty, etc. I don’t think any educated person would consider using such data to assess the quality of an institution.

Note how the commenter places contemptuous quotes around "outcomes." While his experience backs up our findings that many places are not collecting enough accountability information, our report documents places where it is being done: The University of Texas System is using Collegiate Learning Assessment and National Survey of Student Engagement scores in a meaningful way. South Dakota is using Collegiate Assessment of Academic Proficiency scores to calculate learning gains for students in their first two years in college. The University of Hawaii-Hilo is using major field exams to test student knowledge across nine disciplines. Ohio is calculating expected graduation rates using student input demographics. A handful of states are using real wage data to track graduates after degree completion. These examples show that it is possible to assess student learning and outcomes across large and diverse higher education systems. States just need to follow these early leaders.

The other thing to note about the above comment is its derision of the current accountability system. Not only was his institution not collecting meaningful data, but they had no mechanism in place to use it effectively. Read our report, Ready to Assemble: A Model State Higher Education System, to see what data states are already collecting and how they are putting it to work.