Overheard: a business CEO will hire any Harvard MBA before they even begin the program. It isn't the education itself that makes them valuable employees, in this estimation, it's the screen that let them in that proves their quality. In education, it turns out all of our traditional screens, and even some untraditional ones, don't tell us much about how effective the incoming teacher will be.
In a
recent report for the National Bureau of Economic Research, Jonah Rockoff, Brian Jacob, Thomas Kane, and Douglas Staiger look at a host of teacher recruit characteristics and analyze their ability to predict the teacher's effectiveness in the classroom based on these characteristics. The characteristics include some commonly studied ones like their SAT scores, whether the teacher passed their licensure test on the first try, their undergraduate major, and the selectivity of their undergraduate college. The list also included less commonly used measures like tests of cognitive and mathematic ability, conscientousness, extraversion, and efficacy. They also included a commercial screener used by several large urban districts.
The added information did lead to better predictions. But even with all these new variables on incoming teachers, the researchers could explain only 12 percent of the variance in teacher effectiveness. As they wrote, "This underscores the difficult, perhaps impossible, task of identifying systematically the most highly effective or ineffective teachers without any data on actual performance in the classroom."
The findings do not mean this difficult task is impossible (promising
research out of Louisiana suggests that teacher preparation programs matter), but it does suggest we take the
Malcolm Gladwell /
Tom Kane theory of teacher recruitment more seriously. It means that, to get a higher quality teaching workforce, it isn't simply a matter of recruiting more talented, more efficacious, or more extraverted teachers. It means allowing more people to try their hand at the profession, intensively screening them
while they're in the classroom, and then enacting salary and personnel policies to both keep them in the profession and keep them performing at a high level.
Unfortunately, we're much more willing to allow screens (really just educated guesses) at the moment of hiring than after the teacher has been in the job. One huge obstacle in basing personnel decisions on teacher effectiveness is being confident that value-added measures reflect actual value and not the effects of one bad class. Some
excellent research by Dan Goldhaber and Michael Hansen tested the bad-apple theory in teacher value-added effectiveness. They used three years of data to rank teachers into five groups, and then asked what percent of teachers moved groups. In one dataset, 31 of 281 teachers placed in the bottom group in their
students' growth in both math and reading. If we denied these teachers tenure, would we losing great teachers? Or ineffective ones, as the data would predict?
As the chart at left shows, the estimations were by no means perfect. The screen would eliminate some fair and even some very good teachers, but mostly (and this word is not comforting to teachers or their representatives) it screened out ineffective teachers.
Mostly the teachers who were ineffective in their first three years were ineffective after.
The word "mostly" should not be reason to summarily dismiss the use of value-added teacher effectiveness scores in personnel decisions. Rather, it should be embraced as a good start, something to be combined with other evaluations, especially given new research suggesting good teaching cannot easily be screened at the hiring stage.
