The Year In Research On Market-Based Education Reform: 2013 Edition

Posted by on December 17, 2013

In the three most discussed and controversial areas of market-based education reform – performance pay, charter schools and the use of value-added estimates in teacher evaluations – 2013 saw the release of a couple of truly landmark reports, in addition to the normal flow of strong work coming from the education research community (see our reviews from 2010, 2011 and 2012).*

In one sense, this building body of evidence is critical and even comforting, given not only the rapid expansion of charter schools, but also and especially the ongoing design and implementation of new teacher evaluations (which, in many cases, include performance-based pay incentives). In another sense, however, there is good cause for anxiety. Although one must try policies before knowing how they work, the sheer speed of policy change in the U.S. right now means that policymakers are making important decisions on the fly, and there is great deal of uncertainty as to how this will all turn out.

Moreover, while 2013 was without question an important year for research in these three areas, it also illustrated an obvious point: Proper interpretation and application of findings is perhaps just as important as the work itself. Read More »


Teacher Leadership As A School Improvement Strategy

Posted by on February 19, 2013

Our guest author today is David B. Cohen, a National Board Certified high school English teacher in Palo Alto, CA, and the associate director of Accomplished California Teachers (ACT). His blog is at InterACT.

As we settle into 2013, I find myself increasingly optimistic about the future of the teaching profession. There are battles ahead, debates to be had and elections to be contested, but, as Sam Cooke sang, “A change is gonna come.”

The change that I’m most excited about is the potential for a shift towards teacher leadership in schools and school systems. I’m not naive enough to believe it will be a linear or rapid shift, but I’m confident in the long-term growth of teacher leadership because it provides a common ground for stakeholders to achieve their goals, because it’s replicable and scalable, and because it’s working already.

Much of my understanding of school improvement comes from my teaching career – now approaching two decades in the classroom, mostly in public high schools. However, until six years ago, I hadn’t seen teachers putting forth a compelling argument about how we might begin to transform our profession. A key transition for me was reading a Teacher Solutions report from the Center for Teaching Quality (CTQ). That 2007 report, Performance-Pay for Teachers: Designing a System that Students Deserve, showed how the concept of performance pay could be modified and improved upon with better definitions of a variety of performance, and differentiated pay based on differentiated professional practice, rather than arbitrary test score targets. I ended up joining the CTQ Teacher Leaders Network the same year, and have had the opportunity ever since to learn from exceptional teachers from around the country. Read More »


The Year In Research On Market-Based Education Reform: 2012 Edition

Posted by on December 20, 2012

** Reprinted here in the Washington Post

2012 was another busy year for market-based education reform. The rapid proliferation of charter schools continued, while states and districts went about the hard work of designing and implementing new teacher evaluations that incorporate student testing data, and, in many cases, performance pay programs to go along with them.

As in previous years (see our 2010 and 2011 reviews), much of the research on these three “core areas” – merit pay, charter schools, and the use of value-added and other growth models in teacher evaluations – appeared rather responsive to the direction of policy making, but could not always keep up with its breakneck pace.*

Some lag time is inevitable, not only because good research takes time, but also because there’s a degree to which you have to try things before you can see how they work. Nevertheless, what we don’t know about these policies far exceeds what we know, and, given the sheer scope and rapid pace of reforms over the past few years, one cannot help but get the occasional “flying blind” feeling. Moreover, as is often the case, the only unsupportable position is certainty. Read More »


Labor Market Behavior Actually Matters In Labor Market-Based Education Reform

Posted by on July 26, 2012

Economist Jesse Rothstein recently released a working paper about which I am compelled to write, as it speaks directly to so many of the issues that we have raised here over the past year or two. The purpose of Rothstein’s analysis is to move beyond the talking points about teaching quality in order to see if strategies that have been proposed for improving it might yield benefits. In particular, he examines two labor market-oriented policies: performance pay and dismissing teachers.

Both strategies are, at their cores, focused on selection (and deselection) – in other words, attracting and retaining higher-performing candidates and exiting, directly or indirectly, lower-performing incumbents. Both also take time to work and have yet to be experimented with systematically in most places; thus, there is relatively little evidence on the long-term effects of either.

Rothstein’s approach is to model this complex dynamic, specifically the labor market behavior of teachers under these policies (i.e., choosing, leaving and staying in teaching), which is often ignored or assumed away, despite the fact that it is so fundamental to the policies themselves. He then calculates what would happen under this model as a result of performance pay and dismissal policies – that is, how they would affect the teacher labor market and, ultimately, student performance.*

Of course, this is just a simulation, and must be (carefully) interpreted as such, but I think the approach and findings help shed light on three fundamental points about education reform in the U.S. Read More »


Burden Of Proof, Benefit Of Assumption

Posted by on January 20, 2012

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

Michelle Rhee, the controversial former chancellor of D.C. public schools, is a lightning rod. Her confrontational style has made her many friends as well as enemies. As is usually the case, people’s reaction to her approach in no small part depends on whether or not they support her policy positions.

I try to be open-minded toward people with whom I don’t often agree, and I can certainly accept that people operate in different ways. Honestly, I have no doubt as to Ms. Rhee’s sincere belief in what she’s doing; and, even if I think she could go about it differently, I respect her willingness to absorb so much negative reaction in order to try to get it done.

What I find disturbing is how she continues to try to build her reputation and advance her goals based on interpretations of testing results that are insulting to the public’s intelligence. Read More »


Beyond Anecdotes: The Evidence About Financial Incentives And Teacher Retention

Posted by on January 12, 2012

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

Our guest author today is Eleanor Fulbeck, who earned her Ph.D. in education policy from the University of Colorado at Boulder in 2011, and is currently a post-doctoral fellow at the University of Pennsylvania.

A couple of weeks ago, an article in the New York Times, written by reporter Sam Dillon, took a look at the new incentive program being used by the District of Columbia Public Schools (DCPS). Under this plan (called “Impact Plus”), teachers rated “highly effective” by the district’s new evaluation system are eligible for large cash bonuses and/or permanent salary increases.

Dillon notes that, “The profession is notorious for losing thousands of its brightest young teachers within a few years, which many experts attribute to low starting salaries and a traditional step-raise structure that rewards years of service and academic degrees rather than success in the classroom.” He also profiles several teachers who received the bonuses, most of whom say it played a role in their decision to remain in the classroom.

Putting aside these anecdotes and characterizations of “experts’” views, the idea that financial incentives – such as bonuses for performance or teaching in hard-to-staff schools – is a key to boosting teacher retention is a complex empirical question, and an open one at that. Read More »


The Year In Research On Market-Based Education Reform: 2011 Edition

Posted by on December 8, 2011

** Also posted here on ‘Valerie Strauss’ Answer Sheet’ in the Washington Post

If 2010 was the year of the bombshell in research in the three “major areas” of market-based education reform – charter schools, performance pay, and value-added in evaluations – then 2011 was the year of the slow, sustained march.

Last year, the landmark Race to the Top program was accompanied by a set of extremely consequential research reports, ranging from the policy-related importance of the first experimental study of teacher-level performance pay (the POINT program in Nashville) and the preliminary report of the $45 million Measures of Effective Teaching project, to the political controversy of the Los Angeles Times’ release of teachers’ scores from their commissioned analysis of Los Angeles testing data.

In 2011, on the other hand, as new schools opened and states and districts went about the hard work of designing and implementing new evaluations compensation systems, the research almost seemed to adapt to the situation. There were few (if any) “milestones,” but rather a steady flow of papers and reports focused on the finer-grained details of actual policy.*

Nevertheless, a review of this year’s research shows that one thing remained constant: Despite all the lofty rhetoric, what we don’t know about these interventions outweighs what we do know by an order of magnitude. Read More »


Has Teacher Quality Declined Over Time?

Posted by on November 29, 2011

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

One of the common assumptions lurking in the background of our education debates is that “quality” of the teaching workforce has declined a great deal over the past few decades (see here, here, here and here [slide 16]). There is a very plausible storyline supporting this assertion: Prior to the dramatic rise in female labor force participation since the 1960s, professional women were concentrated in a handful of female-dominated occupations, chief among them teaching. Since then, women’s options have changed, and many have moved into professions such as law and medicine instead of the classroom.

The result of this dynamic, so the story goes, is that the pool of candidates to the teaching profession has been “watered down.” This in turn has generated a decline in the aggregate “quality” of U.S. teachers, and, it follows, a stagnation of student achievement growth. This portrayal is often used as a set-up for a preferred set of solutions – e.g., remaking teaching in the image of the other professions into which women are moving, largely by increasing risk and rewards.

Although the argument that “teacher quality” has declined substantially is sometimes taken for granted, its empirical backing is actually quite thin, and not as clear-cut as some might believe. Read More »


Revisiting The Merits Of Merit Pay

Posted by on September 14, 2011

Al Shanker was very concerned about the need to identify and replace incompetent teachers. The first time he wrote a column about it, his wife was one of the many people who warned him that the union’s teachers would be up in arms (see here). Shanker wasn’t worried, replying that “All of my members will read that, and they’ll all agree, because not one of them will think that they are one of the bad teachers that I’m talking about.”

He was right. Most of the members were very supportive, probably for a variety of reasons. First, most teachers take their responsibilities as teachers very seriously, thus favoring the establishment and enforcement of high standards of professional practice. Second, teachers who don’t believe themselves to be effective are more likely to leave the profession – see here. And third, we know from research that most of us just believe that we are simply better than most other people. Psychologists describe this “illusory superiority” or “above average” effect as the tendency to make self-serving comparisons between oneself and others, with the consequence that an overwhelming majority of people judge themselves to be “better than average” on a variety of traits, skills, and socially desirable dimensions ( here and here).

Nevertheless, there are many teachers who support the idea of performance pay, even if they’re wary of the details of how “merit” is defined (specifically, whether or not it includes test scores).

Now, it’s no secret that I think merit pay for teachers is of limited practical utility. Nevertheless, it’s important to understand why, evidence aside, some people (including teachers) might find the policy to be attractive. These are my thoughts on the issue: Read More »


Merit Pay: The End Of Innocence?

Posted by on August 16, 2011

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

The current teacher salary scale has come under increasing fire, and for a reason. Systems where people are treated more or less the same suffer from two basic problems. First, there will always be a number of “free riders.” Second, and relatedly, some people may feel their contributions aren’t sufficiently recognized. So, what are good alternatives? I am not sure; but based on decades worth of economic and psychological research, measures such as merit pay are not it.

Although individual pay for performance (or merit pay) is a widespread practice among U.S. businesses, the research on its effectiveness shows it to be of limited utility (see here, here, here, and here), mostly because it’s easy for its benefits to be swamped by unintended consequences. Indeed, psychological research indicates that a focus on financial rewards may serve to (a) reduce intrinsic motivation, (b) heighten stress to the point that it impairs performance, and (c) promote a narrow focus reducing how well people do in all dimensions except the one being measured.

In 1971, a research psychologist named Edward Deci published a paper concluding that, while verbal reinforcement and positive feedback tends to strengthen intrinsic motivation, monetary rewards tend to weaken it. In 1999, Deci and his colleagues published a meta-analysis of 128 studies (see here), again concluding that, when people do things in exchange for external rewards, their intrinsic motivation tends to diminish. That is, once a certain activity is associated with a tangible reward, such as money, people will be less inclined to participate in the task when the reward is not present. Deci concluded that extrinsic rewards make it harder for people to sustain self-motivation. Read More »


Evaluating Individual Teachers Won’t Solve Systemic Educational Problems

Posted by on July 26, 2011

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

Our guest author today is David K. Cohen, John Dewey Collegiate Professor of Education and professor of public policy at the University of Michigan, and a member of the Shanker Institute’s board of directors.  

What are we to make of recent articles (here and here) extolling IMPACT, Washington DC’s fledging teacher evaluation system, for how many “ineffective” teachers have been identified and fired, how many “highly effective” teachers rewarded? It’s hard to say.

In a forthcoming book, Teaching and Its Predicaments (Harvard University Press, August 2011), I argue that fragmented school governance in the U.S. coupled with the lack of coherent educational infrastructure make it difficult either to broadly improve teaching and learning or to have valid knowledge of the extent of improvement. Merriam-Webster defines “infrastructure” as: “the underlying foundation or basic framework (as of a system or organization).” The term is commonly used to refer to the roads, rail systems, and other frameworks that facilitate the movement of things and people, or to the physical and electronic mechanisms that enable voice and video communication. But social systems also can have such “underlying foundations or basic frameworks”. For school systems around the world, the infrastructure commonly includes student curricula or curriculum frameworks, exams to assess students’ learning of the curricula, instruction that centers on teaching that curriculum, and teacher education that aims to help prospective teachers learn how to teach the curricula. The U.S. has had no such common and unifying infrastructure for schools, owing in part to fragmented government (including local control) and traditions of weak state guidance about curriculum and teacher education.

Like many recent reform efforts that focus on teacher performance and accountability, IMPACT does not attempt to build infrastructure, but rather assumes that weak individual teachers are the problem. There are some weak individual teachers, but the chief problem has been a non-system that offers no guidance or support for strong teaching and learning, precisely because there has been no infrastructure. IMPACT frames reform as a matter of solving individual problems when the weakness is systemic. Read More »


Attention To Pay

Posted by on November 30, 2010

The debate over how best to restructure teacher salary systems is older than I am—with good reason: Instructional salaries represent roughly 40 percent of current K-12 public school expenditures.  And some of the arguments for changing current salary structures make sense, at least in theory. 

For instance, there is a case for tying step increases (typically awarded according to years of service) to additional measures, such as strengthened evaluation systems and curriculum-linked professional development (as is the case in the recently-ratified Baltimore contract). These types of changes, if they are bargained and approved by teachers, could be of real benefit to all stakeholders.

At the same time, it’s unfortunate that some of the talking points used commonly by those who wish to overhaul teacher salary systems are rather misleading and oversimplified. Not only do they sometimes seem designed to inspire outrage against teachers, they also tend to obscure or ignore important facts about the relationship between teacher pay and teacher quality.  Three such arguments seem particularly pervasive. Read More »


Three Questions For Those Who Dismiss The Nashville Merit Pay Study

Posted by on September 27, 2010

The reaction from many performance pay advocates to the Nashville evaluation released last week has been that the study is relatively meaningless (see here and here for examples).  The general interpretation: The results show that the pay bonuses do not improve student achievement, but short-term test score gains are not the ”true purpose” of these incentive programs. What they are really supposed to improve, so the line goes, is the quality of people who pursue teaching as a career, as well as their retention rates.

While I disagree that the findings are not important (they are, if for no other reason than they discredit the idea that teachers are holding their effort hostage to more money), I am sympathetic towards the view that the study didn’t tackle the big issues. Attracting the best possible people into the profession – and keeping them there – are much more meaningful goals than short-term test score gains, and they are not addressed in this study (though some results for retention are reported).

But this argument also begs a few important questions that I hope we can answer before the Nashville study fades into evaluation oblivion.  I have three of them. Read More »


Persistently Low-Performing Incentives

Posted by on September 21, 2010

Today, the National Center on Performance Incentives (NCPI) and the RAND Corporation released a long-awaited experimental evaluation of teacher performance pay in Nashville, Tenn. It finds that performance bonuses have virtually no effect on student math test scores (there were small but significant gains by fifth graders, but only in two of the three years examined, and the gains did not last into sixth grade).

Since this is such a politically contentious issue, these findings are likely to spark a lot of posturing and debate. So it’s worth trying to put them in context. As I discussed in a prior post, we now have at least preliminary results from three randomized experimental evaluations of merit pay in the U.S., the first contemporary, high-quality evidence of its kind.  This Nashville report and the two previously-released studies – one from Chicago and one from New York City’s schoolwide bonus program – reached the same conclusion: Performance bonuses for teachers have little or no discernable effect on student test scores. 

Although the NYC and Chicago findings are preliminary (the evaluations are still in progress), the NYC program provides schoolwide and not individual bonuses, and one additional study (Round Rock, Tex.) is yet to be released, the three already-released reports do represent a fairly impressive, though still very tentative, body of evidence on merit pay’s utility as a means to improve test scores.

And at this point, it’s a good bet that, when all the evaluations are final and the smoke has cleared, we will have to conclude that performance bonuses are, at the very least, a very unpromising policy for producing short-term test score gains.  Read More »


Extra Time: More From The Magazine’s Education Poll

Posted by on September 14, 2010

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post.

A recent education poll conducted by Time Magazine has gotten a lot of attention. Many of the questions are worded so badly that the results are rather meaningless. The question on merit pay, for example, defines the practice as “paying teachers according to their effectiveness” (who would oppose that, if it could be accurately measured?). Other questions are very interesting, such as the one asking whether respondents would pay higher taxes to improve public schools (56 percent would). Or the finding that, when asked what will “improve student achievement the most,” more than twice as many people choose “more involved parents” (54 percent) over “more effective teachers” (24 percent).

But, as is sometimes the case, a few of the survey’s most interesting results were not included in the published article, which highlighted only 11 out of 40-50 or so total questions (the full set of results is available here). Here are three or four unpublished items that caught my eye (the sample size is 1,000, with a margin of error of +/- 3 percent): Read More »


Disclaimer

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the shankerblog.org may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.

Banner image adapted from 1975 photograph by Jennie Shanker, daughter of Albert Shanker.