False discoveries: Interpret with caution

Despite my best efforts to persuade, the folks at Fordham's Flypaper blog are again trumpeting the results of the DC voucher program. While they admit the study found no significant difference in academic gains between students who received a voucher and those that did not, they cherry-pick this quote:

However, being offered a scholarship may have improved reading test scores among three subgroups of relatively more advantaged students: those who had not attended a School in Need of Improvement (SINI) school when they applied to the program, those who had relatively higher pre-program academic performance, and those who applied in the first year of program implementation.

despite the Institute of Education Sciences explicitly warning these findings "were no longer statistically significant when subjected to a reliability test" and "the results may be 'false discoveries' and should therefore be used and interpreted with caution."

These quotes are not exactly buried; you can find them in the Executive Summary.