Three Questions For Those Who Dismiss The Nashville Merit Pay Study

Posted by on September 27, 2010

The reaction from many performance pay advocates to the Nashville evaluation released last week has been that the study is relatively meaningless (see here and here for examples).  The general interpretation: The results show that the pay bonuses do not improve student achievement, but short-term test score gains are not the ”true purpose” of these incentive programs. What they are really supposed to improve, so the line goes, is the quality of people who pursue teaching as a career, as well as their retention rates.

While I disagree that the findings are not important (they are, if for no other reason than they discredit the idea that teachers are holding their effort hostage to more money), I am sympathetic towards the view that the study didn’t tackle the big issues. Attracting the best possible people into the profession – and keeping them there – are much more meaningful goals than short-term test score gains, and they are not addressed in this study (though some results for retention are reported).

But this argument also begs a few important questions that I hope we can answer before the Nashville study fades into evaluation oblivion.  I have three of them.

How will we ever know if merit pay has the desired effect?  It seems unlikely that we will ever get far beyond highly speculative results on performance pay programs’ effect on improving the quality of the labor supply. There would really be no way even partially to isolate merit pay’s effect from the swirling mass of factors, observed and unobserved, that influence career choices. We could assess whether retention improved, but only for all teachers, not specifically for those who opted to teach because of the bonuses. So, not only do we not know if merit pay “works,” but I see no way that we could ever know with any confidence, at least on the “attracting better people” front.

There is nothing necessarily wrong with not knowing – indeed, it is usually the case in public policy that we don’t (and “knowledge” is approximate even when we do). But let’s be clear about our standards and prospects for empirical evidence, both now and in the future. The assumption on the merit pay side is that the most talented young people will be more likely to opt for compensation systems where they might make a lot more money if their students’ scores improve more quickly. Is this assumption so safe that we can be comfortable accepting it on pure faith?

What is the evidence that current compensation systems – increased pay for years of experience and more education – haven’t improved the quality and retention of entrants into the profession?  Normally, the argument against teachers’ current experience/education-weighted pay scale is based on these factors’ weak association with short-term test score gains. But dismissing the Nashville study means that this argument is no longer particularly credible. If compensation systems are not really supposed to improve or motivate current teachers, as assessed by their effect on student test scores – only to attract “better” ones and keep them around – then we also need to reassess the utility of education and experience according to this new standard. 

Extra salary for more education (e.g., a master’s degree) would seem to promote self-selection among talented, motivated applicants who either have higher education degrees or would be willing to pursue them. It might also keep these people in the classroom after they got their degrees (by paying them more). As for experience, are we confident that the “best” applicants are more attracted to big, possibly occasional raises based on test scores, rather than to smaller, guaranteed raises over the long haul? Perhaps they are, but what reasons/evidence do we have?

Isn’t the best idea to simply increase teachers’ salaries? I’m not sure that performance incentives will attract better people, but I am certain that higher salaries would. Competition for open positions would increase, and the best applicants would get the jobs. At least that’s the way it seems to work among most of the word’s highest performing nations.

Of course, there are arguments against it. For one thing, it is expensive. It is also not necessarily the most efficient way to spend money – ideally, we would want to target spending directly at attracting and keeping talent (that would get pretty expensive as the talent accumulated and stayed). At the same time, though, incentive programs are also rather expensive (both the bonuses themselves and designing/maintaining the systems), and they (I would argue) carry some risk of not having the desired effect. So, if the true goal is to attract and keep talent, isn’t it the safest path to invest in raising teachers’ base salaries?  We could at least be confident that this would produce results.

I would add one final semantic point to these three questions. Given the new, ostensible goals of attracting/retaining talent, the term “performance incentives” is not very appropriate. If, according to the argument, they are not supposed to incentivize performance per se, but rather career decisions (which to choose and how long to stay), perhaps they should be called “career incentives.” It has a different ring, doesn’t it?


7 Comments posted so far

  • Why should smart, talented and hardworking people go into a field where the big bonuses are tied to such problematic analyses like value-added analysis?

    Why wouldn’t a smart, talented and hardworking person want a known and more predictable salary? From whence does the assumption that the best potential teachers are big risk-takers with their own financial futures?

    Comment by ceolaf
    September 28, 2010 at 12:48 PM
  • [...] [...]

    November 30, 2010 at 10:03 AM
  • [...] Three Questions For Those Who Dismiss The Nashville Merit Pay Study comes from Matthew Di Carlo at the Shanker blog. [...]

    January 15, 2011 at 8:28 AM
  • [...] was actually having the desired effect (here’s one of the first attempts I’ve seen). As I have said before, merit pay is a leap of faith. Now, don’t get me wrong – almost all research on public policy [...]

    March 29, 2011 at 8:50 AM
  • [...] when they otherwise wouldn’t (personally, I find the argument that merit pay would do so to be far less compelling). So, if Mr. Kristof (or the Equity Project principal) is arguing that we should boost teachers’ [...]

    March 30, 2011 at 11:55 PM
  • Merit pay only works if you redefine “merit” -which cannot be based merely on a test score. How is that any different from No Child Left Behind?

    Merit needs to be an objective number based on multiple criteria including testing, PLUS – years in service, certifications, degrees, performance evals, and peer reviews.

    EVERY profession on the planet is based on some level of minimum output. What on earth makes teaching so different that these particular employees are immune to being evaluated (albeit correctly), then compensated appropriately?

    Comment by Jesse Weinberger
    July 20, 2011 at 8:54 PM
  • [...] new compensation/employment structures, and the “higher risk, higher reward” idea, though unproven in education, is not without its potential if done correctly. After all, teacher pay continues to [...]

    September 29, 2011 at 5:18 PM

Sorry, the comment form is closed at this time.

Disclaimer

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the shankerblog.org may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.

Banner image adapted from 1975 photograph by Jennie Shanker, daughter of Albert Shanker.