Performance Pay On (Randomized) Trial

Posted by on August 3, 2010

This is an exciting time for those of us who are strange enough to find research on teacher performance pay exciting. It is also, most likely, an anxious time for those with unyielding faith in its effectiveness. From all the chatter on performance incentives, and all the money we are putting into encouraging them, one might think they are a sure bet to work. But there’s actually very little good evidence on their effects in the U.S. As with a lot of education policy in fashion today, investing in performance pay is a leap of faith.

But now, just in time to be way too late, there are currently four high-quality evaluations of teacher performance pay programs in progress, and they are the first large-scale experimental studies of how these bonuses affect performance in the U.S.

Prior to these, the only contemporary, high-quality evidence on the subject came from other nations, most notably Israel, Kenya, and India. There is also a handful of older, small-scale U.S. studies (see Cohen and Murnane 1985 and a review in Podgursky and Springer 2007).

This year, preliminary reports were released for two of these programs. First, in April 2009, the National Center for Performance Incentives (NCPI), in a joint project with RAND and the University of Missouri, released its first-year report on the New York City Schoolwide Performance Bonus Program (SPBP). SPBP gives performance awards to schools (rather than individual teachers). This analysis evaluates the effects of the first year of the program only, and finds no discernible effects (yet) of the program on student test scores.

More recently, about two months ago, a study of Chicago’s Teacher Advancement Program (TAP) was released by Mathematica. The Chicago TAP program is a four-year program that provides teacher bonuses as well as an array of tools for improvement and opportunities for advancement. This was the second annual report of four, covering the 2008-09 school year (the first evaluation, covering 2007-08, was released in 2009). The researchers also find no effects on student achievement (or teacher retention).

The two remaining studies are also in progress (both by researchers affiliated with NCPI), but no results have been released yet. There is an evaluation of grade-level team incentives going on in Round Rock, TX, as well as a large, multi-year study of individual performance bonuses in Nashville. The latter study was slated for release in April of this year, but it has been delayed (I’m not sure why).

I’ve heard some people comment that the preliminary NYC and Chicago results show that these bonus programs will be a failure. That is an overreaction.

For one thing, both programs(and their evaluations) are still at midway points, and it is entirely possible that the effects will surface as the programs are in force longer. Second, and more importantly, there are still two other programs being assessed, including what is probably the most extensive evaluation of the four (in Nashville).

Moreover, if these Texas and Tennessee programs, unlike NYC and Chicago so far, do turn out to have positive effects on student test scores, the size of those effects – and whether they are persistent over time – will be very important considerations. So too will be any other undesired effects of bonuses, such as increasing turnover, narrowing of the curriculum, or depressing the supply of entrants into the profession. The programs might also have important desired effects on outcomes other than tests scores, such as higher retention. In other words, whether or not the incentives produce higher student achievement, it is also important to understand why they had these effects, and whether the reasons (and any side effects) are things we are willing to encourage.

Still, as a whole, the results of these four evaluations represent a solid foundation for assessing performance pay. They will not be the final word, of course, as a full assessment would measure the effect of different program configurations (e.g., higher bonus amounts, different support measures to help teachers improve).

But if, in the final analysis, these four evaluations overwhelmingly show that bonuses have little appreciable effect on performance, I would hope that we would strongly consider halting their spread to new states and districts, and limiting the more thorough examination of their impacts to a smaller group of existing programs.

And in the meantime, what we can say is that cash incentives for teachers are not a surefire way to improve student test scores, and that anyone still advocating for teacher performance pay without a healthy dose of skepticism is either unaware of the evidence, or ignoring it.


3 Comments posted so far

Sorry, the comment form is closed at this time.

Disclaimer

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the shankerblog.org may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.

Banner image adapted from 1975 photograph by Jennie Shanker, daughter of Albert Shanker.