Learning From Teach For America

Posted by on March 19, 2012

There is a small but growing body of evidence about the (usually test-based) effectiveness of teachers from Teach for America (TFA), an extremely selective program that trains and places new teachers in mostly higher needs schools and districts. Rather than review this literature paper-by-paper, which has already been done by others (see here and here), I’ll just give you the super-short summary of the higher-quality analyses, and quickly discuss what I think it means.*

The evidence on TFA teachers focuses mostly on comparing their effect on test score growth vis-à-vis other groups of teachers who entered the profession via traditional certification (or through other alternative routes). This is no easy task, and the findings do vary quite a bit by study, as well as by the group to which TFA corps members are compared (e.g., new or more experienced teachers). One can quibble endlessly over the methodological details (and I’m all for that), and this area is still underdeveloped, but a fair summary of these papers is that TFA teachers are no more or less effective than comparable peers in terms of reading tests, and sometimes but not always more effective in math (the differences, whether positive or negative, tend to be small and/or only surface after 2-3 years). Overall, the evidence thus far suggests that TFA teachers perform comparably, at least in terms of test-based outcomes.

Somewhat in contrast with these findings, TFA has been the subject of both intensive criticism and fawning praise. I don’t want to engage this debate directly, except to say that there has to be some middle ground on which a program that brings talented young people into the field of education is not such a divisive issue. I do, however, want to make a wider point specifically about the evidence on TFA teachers – what it might suggest about the current focus to “attract the best people” to the profession.

This goal – recruiting and retaining talented people into teaching – is shared by most everyone, but it is among the most central emphases of the diverse group that might be called market-based reformers. Their idea is to change compensation structures, performance evaluations and other systems in order to create the kind of environment that will be appealing to high-achieving, less risk-averse people, as well as to ensure that those who aren’t cut out for the job are compelled to leave. This will, so the argument goes, create a “dynamic profession” more in line with the high risk, high reward model common among the private sector firms competing for the same pool of young workers.

No matter your feelings on TFA, it’s more than fair to say that their corps members fit this profile perfectly. On paper, they aren’t just “top third,” but top third of the top third. TFA cohorts enter the labor market having been among the highest achievers in the best colleges and universities in the nation. Getting accepted to the program is very, very difficult. Those who make it are not only service-oriented, but also smart, hard-working and ambitious. They are exactly the kind of worker that employers crave, and market-based reformers have made it among their central purposes to attract to the profession.

Yet, at least by the standard of test-based productivity, TFA teachers really don’t do better, on average, than their peers, and when there are demonstrated differences, they are often relatively small and concentrated in math (the latter, by the way, might suggest the role of unobserved differences in content knowledge). Now, again, there is some variation in the findings, and the number and scope of these analyses are limited – we’re nowhere near some kind of research consensus on these comparisons of test-based productivity, to say nothing of other sorts of student outcomes.

(It’s also very important to note that, for all we know, TFA teachers would get better results with more extensive preparation. After all, even the most well-designed five-week training regimen would have trouble preparing teachers for placement in some of the highest-needs schools and districts in the nation.)

Still, even these admirable young people, who could probably have their choice of jobs outside education, end up being just hard-working teachers, struggling to manage classrooms, plan lessons and get as much learning as possible from their students, often under less-than-ideal conditions. This squares with the related literature showing that the majority of measurable pre-service characteristics, such as the selectivity of undergraduate institution, GPA, etc., are, at best, inconsistently predictive of future classroom performance (at least as measured by growth model estimates). The variation within these groups completely overwhelms the variation between them.

This is one of the reasons why, whenever I hear someone talk about the need to “attract the best people” to teaching, I wonder as to their conceptualization of the “best people.” In most cases, they’re talking about the kind of folks that come through TFA.

And I’m all for getting these people into teaching – we should have as many of them in classrooms as we can. Sure, in TFA’s case, they commit to only a couple of years and most do leave, but some do stay beyond that commitment, and it’s worth noting that attrition and mobility are also extremely high among traditionally-certified teachers who, like TFA’ers, work in high-needs schools and districts. Moreover, it’s not irrelevant that many of the TFA teachers who leave the classroom pursue leadership positions in education (positions which many teachers believe require classroom experience).

But, to me, one of the big, underdiscussed lessons of TFA is less about the program itself than what the test-based empirical research on its corps members suggests about the larger issue of teacher recruitment. Namely, it indicates that “talent” as typically gauged in the private sector may not make much of a difference in the classroom, at least not by itself. This doesn’t necessarily mean that market-based policies won’t lure great teachers, but it does suggest that, if we’re going to enact massive changes in personnel policy to attract a certain “type” of person to teaching, we might reexamine our assumptions on who we’re trying to attract and what they want.

- Matt Di Carlo

*****

* Probably the most rigorous study in this area, albeit one that is not easily generalized, is this Mathematica evaluation (later published in a peer-reviewed journal), which exploits random assignment of students to classrooms. A few high-quality examples of quasi-experimental treatments include: this published analysis of New York City teachers; this paper, which was also subsequently published and also used NYC data; and this recent working paper comparing alternatively- and traditionally-certified teachers in Florida. Finally, a 2009 working paper was the first to  compare TFA teachers placed in high schools.


19 Comments posted so far

  • Turn it upside down and see if you recognize it.

    The existing national cohort of K-12 teachers compares favorably to a TFA cohort of top performers recruited from top institutions.

    This tends to indicate that reformers targeting teachers are doing so with an ideological goal rather than that of improving education. That’s not surprising since the same is true of the charter school movement.

    TFA has been used shamefully by the people supporting it.

    Comment by Bob Calder
    March 19, 2012 at 11:09 AM
  • Two related critiques:

    1.Consider the recruiting practices of TFA. It’s hard to imagine that they are simply driven by a wholesale desire to recruit, as you state, “the best people.” It’s clear that different strategies are employed for different regions from year to year, such that reputations are developed for different groups of recruits from different recruitment pools across the nation. You seem to redeem TFA by re-affirming its self-proclaimed mantra that it in fact DOES recruit those that, to use an oft-repeated term, “add value” to private sector industries, without taking the time to think about the ways in which this might not be the case. TFA’s high numbers of applicants are due to aggressive and institutionally embedded recruitment practices that are sustained by the myth that being a TFA teacher is just as valuable in the marketplace as a stint as a two year analyst at any big investment bank. It is important to understand this as, in part, a marketing strategy and unpack the many ways in which recruits might deviate from this “model.”

    2.You take issue with the notion that those who would do well in the private sector would make good teachers, but is that the right place to begin your critique? You seem to elevate the position of TFA recruits, suggesting that their skills- which you spent a large portion of the piece praising- are not fit for the classroom. In doing so, you are replaying the TFA driven narrative that it’s recruits are somehow superior to all other teachers (if not to all other people!) without questioning the structures of social and economic inequalities that have perpetuated the existence of such an “elite” group.

    Comment by JB
    March 19, 2012 at 12:00 PM
  • Comment by AM
    March 19, 2012 at 3:33 PM
  • Excellent post, Matt.

    Other countries that out perform us do tend to have teachers drawn from the slice of their college graduates that looks like TFA’s talent pool. I suspect (though I don’t know, of course) that our K-12 working environment is not really a place where anyone can thrive on average. (Lots of factors reward mediocrity, labor-management relations are not great for reasons one could blame on either side depending on the place, etc.) So the fact that TFAers are only as effective as traditionally-trained educators isn’t that revealing to me.

    I also suspect that the real value TFA has provided historically is in the entrepreneurs, principals, and other folks who have come out of their pipeline, which isn’t captured in the classroom effectiveness data directly. Our experience as a charter school authorizer in Ohio has certainly borne this out–TFA is a great source of school leaders and school founders.

    Anyhow, I share your sense that this should not be as polarizing as it is!

    Comment by Chris Tessone
    March 20, 2012 at 1:07 PM
  • Matt,

    Interesting take on the TFA research. I agree that there has been too much focus on how to attract the “best and the brightest,” whatever that means. In the business literature on human resource management, Baron and Kreps (http://www.amazon.com/Strategic-Human-Resources-Frameworks-Managers/dp/0471072532) argue that different firms and industries must understand the type of labor force they should seek out based on the nature of their work and technology (methods of production).

    In some industries–such as investment banking, law, sports, and entertainment–a few “stars” often generate the majority of a firm’s profits and success. Identifying individuals who have the potential to be stars is thus important and taking some risks in hiring is not problematic. In other words, finding a few stars can more than make up for a few hiring duds. Other industries depend on consistent, reliable performance of a large number of “foot-soldiers,” and success comes from improving the average performance of workers and reducing variation in their performance via training, technology, organization and process improvements.

    In still other industries (often involving safety and security), a few mistakes can be disastrous for a firm, so firms need “guardians” and try to make safe hires even if this might mean also reducing the number of potential high performers. Certification and credentialing is thus often important. For these firms/industries, the costs of a mistake are greater than the gains of a high performer.

    In education, it seems as if we sometimes swing from the extremes of taking a guardian approach (emphasizing rigid credentialing to avoid “mistakes”) and focusing on hiring a few stars through targeted programs.

    I think, however, given that the teaching force is so huge–there are 3-4 million teachers–it’s unrealistic to take a “star” approach, but that is precisely the approach TFA promotes by arguing that they can produce enough leverage by creating a group of transformative leaders in education.

    Ultimately, however, teaching is huge profession that relies on millions of foot soldiers. This is not to say that we shouldn’t be interested in attracting some of the best and brightest. We should. However, we should focus our main efforts on policies that hold promise for improving the average quality of teachers and that reduce the variability in their performance. This points to different strategies–improving pre-service and in-service training, improving the tools that are available to teachers in the form of better curriculum, assessments, technology, etc., and reorganizing schools to better leverage teachers skills and promote teacher learning and continuous improvement.

    Comment by Ed
    March 20, 2012 at 5:58 PM
  • Very interesting post. You have appropriately asked the right question. What does teaching “talent” look like in a preservice teacher? What are the characteristics we should be after in future teachers? TFA, in general, takes the approach that high GPAs in rigorous content areas are the right ingredients. And while the overall test-based data is flat I have talked with administrators that hire TFAers and they note that in some areas the increased content knowledge separates TFA candidates from their standard applicant pool.

    An area this post reopens for me is that for years there have been “off the shelf” HR screening tools that through independent research have been predictive of higher student test performance. These tools tend to four on underlying beliefs and demeanors. Perhaps some application of those tools with TFA and traditional candidates could shed some light on this idea of identifying the proper talent set.

    Comment by CW
    March 21, 2012 at 10:22 AM
  • If TFA teachers are performing comparably to teachers with much more extensive training, that seems like a sign they’re actually doing pretty well. Nonetheless, as you state, the evidence is pretty shaky, so it’s hard to draw any conclusions.

    It is pretty obvious public education needs a big shake up, though. Making the teaching profession more attractive almost can’t be a bad way to do this, assuming we’re willing to pay for it.

    Comment by Joe
    March 21, 2012 at 10:49 AM
  • I think that there are two points that one MUST keep in mind when thinking about the “success” of TFA:

    1) If you judge the success of a group based on the measures they choose of to focus on, you are choosing the most generous basis for that judgement. TFA focuses on test scores. Whether that is the right or wrong thing to do, there can be no more generous way to judge it. That TFA teachers do NOT produce substantially better test scores is a MUCH greater condemnation of TFA that it would be of other programs that do NOT focus on test scores.

    2) TFA also points to the impact that those who leave TFA and the classroom can have, as public leaders and citizens. This is another key aspect of their program, and the reason why it TFA suggests that TFA should not be judged by its retention rates. Well, what IS the impact of the experience of being put in so many of our nation’s most dysfunctional schools, with the most challenging populations and often among the worst leadership. If one’s only experience as an adult in public education is in poorly run organization in which the challenges far surpass the local organizatonal capacities to address challenges, what is the impact? What is the impact of that non-representative/worst-case experience on future the opinions, judgements and forecasts of TFA corps members?

    I begin all consideration of TFA with these two ideas in mind. In order to learn from TFA, we need to understand what what TFA’s goals are, and where its efforts have been focused. Then we can get to model and implementation.

    Frankly, I do not see what TFA can tell about us about teacher recruitment, “at least not by itself.” I tend to think that it tells us more about teacher prepartion and training than recruitment, but all we can be sure of is what it tells us about a particular combination of recruitment and preparation.

    Comment by ceolaf
    March 21, 2012 at 11:29 AM
  • [...] will help educational performance…) A Seven Day Plan to Finally Hold Wall Street Accountable Learning From Teach For America To Save Israel, Boycott the Settlements Women’s Rights: I think that sums it up. How [...]

    March 21, 2012 at 4:39 PM
  • [...] week, Gary Rubenstein critiques Wendy’s critique. This is getting complicated. Matt DiCarlo asks what we can learn from TFA — an insightful consideration of why there must be some middle ground on the TFA debate. A TFA [...]

    March 23, 2012 at 4:07 PM
  • Matt,
    I’m curious what you thought of the Will Dobbie paper on TFA selection criteria and value-add in year 1 in NYC as it relates to your arguments?

    http://www.people.fas.harvard.edu/~dobbie/research/TeacherCharacteristics_July2011.pdf

    Comment by Heather
    March 26, 2012 at 9:22 AM
  • Hi Heather,

    Thanks for the comment. Please see: http://shankerblog.org/?p=5221

    Thanks again,
    MD

    Comment by Matthew Di Carlo
    March 26, 2012 at 9:33 AM
  • [...] a somewhat different part of the ideological spectrum than I do on education reform. He has an assessment of the literature on TFA that I recommend.  Although I don’t know this area of research very well, his [...]

    March 27, 2012 at 8:40 PM
  • @Bob Calder

    It could also mean that the current model of teaching is simply ineffective regardless of who is teaching it.

    As far as I can tell, we rely too much on books and not enough on discussion, and the basic conceptual tools necessary to participate in discussion. We do not teach skills within their historically contextual narratives, nor do we teach history. (I’ll argue that Neal Postman is right even if his list of narratives is wrong.)

    And the reason we do not, is to avoid the necessity of making value judgements during the process of educating people from different races, classes and cultures.

    ALl we have to do is allow principles to fire teachers at will, and school districts to choose their own curriculum and the problem will solve itself in a decade.

    Or we can keep falling behind the rest of the world.

    Comment by Curt Doolittle
    March 27, 2012 at 8:49 PM
  • @JB

    I don’t know the data first hand but from what I’ve seen the TFA selection process is from a cohort that is is significantly better than the pool of teachers existing or in training.

    Net is that the market will solve the problem if we let it.

    I’d rather pay fired teachers some salary for five years and get them out of the system then have more “waiting for superman” eras in education.

    Comment by Curt Doolittle
    March 27, 2012 at 8:52 PM
  • Though understandable, measuring TFA’s value in the classroom comparable to traditional teachers is besides the point. It does not lie in classroom effectiveness.

    TFA’s value comes from the few stars that emerge and take leadership positions. The prestige they bring as leaders is akin to that of high status luminaries and researchers who bring status to universities and rarely teach. TFA leaders can tap into their private sector success to shape (some say privatize) education in innovative ways.

    Comment by TES
    March 28, 2012 at 9:12 AM
  • [...] week or so ago, Matthew diCarlo of the Shanker Institute posted on the Shanker Blog a post exploring the link between teacher performance and the much-lauded, much-criticized, and thus, controversial, program Teach for [...]

    April 2, 2012 at 4:37 PM
  • [...] lately on how to interpret the studies on Teach For America (TFA) teacher effectiveness – see Matthew Di Carlo and Adam Ozimek. But neither addresses the research from a Relinquisher standpoint. Here’s what [...]

    April 26, 2012 at 2:48 PM
  • I just came upon your blog and enjoyed the thoughtful and unbiased analysis of the program. As a Teach For America alum, my experience in the corps had its share of ups and downs, and I definitely saw corps members who both outperformed their peers and underperformed their peers. Unfortunately, my teacher career was cut short my massive layoffs and budget cuts in my district.

    My biggest critique of Teach For America is the way selection is done in the program. While the “best and the brightest,” in the traditional sense, do apply and are accepted, I think it would be a stronger teaching corps if incoming TFAers had real experiences with kids before they enter the program. It is important that we get the most talented people into the profession, but not if they are there for the wrong reasons. I am ok with teachers leaving after two years because the vast majority of them will continue to positively impact education for the rest of their lives in whatever field they choose.

    I am writing a book on building the skills necessary to be a great Teach For America corps member and get accepted into the program. I cover everything from ways to get involved with students before applying to ways applicants can think about the experiences they have had. I push for interested applicants to develop the skills in college to become great teachers and leaders so that there is not as much of a learning curve when they enter the corps and can become effective teachers more quickly. If TFA can keep recruiting the best talent, but have those teachers come in with relevant experiences, TFA can continue to lead the path toward educational equity in our nation.

    Visit my website at http://www.becomingtfa.com to read my blog and pre-register your copy of the book (No obligation to purchase and $5 off when it is published if you pre-register)

    Comment by Jake
    June 21, 2012 at 11:43 AM

Sorry, the comment form is closed at this time.

Disclaimer

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the shankerblog.org may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.

Banner image adapted from 1975 photograph by Jennie Shanker, daughter of Albert Shanker.