If Gifted And Talented Programs Don't Boost Scores, Should We Eliminate Them?

In education policy debates, the phrase “what works” is sometimes used to mean “what increases test scores." Among those of us who believe that testing data have a productive role to play in education policy (even if we disagree on the details of that role), there is a constant struggle to interpret test-based evidence properly and put it in context. This effort to craft and maintain a framework for using assessment data productively is very important but, despite the careless claims of some public figures, it is also extremely difficult.

Equally important and difficult is the need to apply that framework consistently. For instance, a recent working paper from the National Bureau of Economic Research (NBER) looked at the question of whether gifted and talented (GT) programs boost student achievement. The researchers found that GT programs (and magnet schools as well) have little discernible impact on students’ test score gains. Another recent NBER paper reached the same conclusion about the highly-selective “exam schools” in New York and Boston. Now, it’s certainly true that high-quality research on the test-based effect of these programs is still somewhat scarce, and these are only two (as yet unpublished) analyses, but their conclusions are certainly worth noting.

Still, let’s speculate for a moment: Let’s say that, over the next few years, several other good studies also reached the same conclusion. Would anyone, based on this evidence, be calling for the elimination of GT programs? I doubt it. Yet, if we applied faithfully the standards by which we sometimes judge other policy interventions, we would have to make a case for getting rid of GT.

For example, there is a common argument in education circles that teachers with master’s degrees do not produce larger test score gains than those without them, and so we should therefore stop providing those who have them with a salary “bump” (the same basic argument is often applied to giving teachers raises for additional years of experience). New York City recently shut down its schoolwide bonus program in the wake of a RAND evaluation finding that the program was not associated with higher test scores. In fact, many people argue that we should close entire schools if their students consistently fail to make progress on assessments, and open new schools if their operators demonstrate an ability to make such progress.

Are these programs and policies somehow different from gifted and talented programs? Not really. GT programs (and similar programs such as magnet/exam schools) cost money, which of course also means that they leave less funding for other interventions. If a solid body of research found that they offered no test-based benefits, how could those who argue for eliminating master’s bumps and closing low-performing schools based on the same evidence remain silent when it comes to GT programs?

The answer, it seems to me, is simple: People acknowledge that GT programs, offering special services such as advanced curricula, “hands-on” experience and specially-trained teachers, provide benefits that cannot be measured in terms of test score gains. Supporters think that they’re worth paying for, even if they don’t boost scores, because they believe that these programs improve children’s educational experience in ways that are less “tangible” but perhaps just as important.

This is an enlightened, nuanced view of “what works," as distinct from the clumsy and simplistic standard by which we sometimes (but not nearly always) judge other policies. Now, it’s certainly true that plenty of people do oppose GT and similar programs, but their arguments are typically based on other concerns, such as equity and fairness, rather than test-based results.

To be clear, I’m not trying to accuse anyone of hypocrisy. I understand that policy judgments often must be made based on imperfect evidence; and it’s difficult to find the sweet spot between this uncertainty and the need to act to improve performance. Moreover, while there are a few people who feel that standardized testing results should play absolutely no role in making decisions about teachers, students and schools, I am not among them (even if I frequently disagree with the manner in which they are used).

What I am saying is that our education discourse sometimes takes too far the viewpoint that boosting test scores is a sufficient measure of “what works," and it’s not always clear why the standard applies in some instances and not others. When pressed, nobody actually agrees with the “pure” test-based standard - even staunch advocates for using assessment data in high-stakes decisions usually acknowledge that these data are not, by themselves, sufficient to judge success and failure. Nevertheless, in some circumstances, we continue to behave as though it were sufficient – e.g., opening/closing schools because of operators’ record of test score gains (or lack thereof), or ending the practice of rewarding master’s degrees. Yet, if we were to apply that standard consistently, we would be calling for shutting down programs that the majority of citizens, parents, educators, and policymakers agree have intrinsic value, whether or not they improve the test-based bottom line.

In my opinion, test-based evidence can and should play a role in education policy decisions. But in the struggle to figure out how to use these data, let’s also pay attention to when we use them.

- Matt Di Carlo

Permalink

Is there anyone without skin in the game (that is, anyone outside of the ed school world) who sees any reason athat masters' degrees in education would be of any benefit whatsoever? I've sat through tedious and irrelevant masters' ed classes, and I can't imagine why anyone would do it but for the artificial pay boost at the end.

Permalink

There's a technical reason why G&T programs should not be responsible for increasing scores:

Tests generally have much smaller standard errors in the middle of their testing range than at the extreams. As G&T students are -almost by defintion-- likely to be scoring near the high end of the range, it becomes very very very difficult to say with statistical certaintly that any program can improve the scores of these students.

Furthermore, these tests tend to have ceilings to thei scores. In practical terms, that means that for many G&T students the tests simply don't measure as far as these students have grown.

So, even if we ignore the possibility that G&T programs help students in ways that these tests to do attempt to measure, and focus on the areas that these tests do try to measure, the instrument itself is quite rarely suitable for measuring improvement in this range.

So, no. G&T programs should NOT be eliminated if they don't improve scores (unless the ultimate goal of education is to improve scores)

Permalink

uh..yeah. The reason that people would sit through Master ed. classes is that people actually enjoy it. They enjoy being stimulated, they enjoy sharing ideas, they enjoy challenging themselves because even in the most boring of classes, if you have the right attitude, you can take something out of it. Of course, if you have the wrong attitude, you complain.

Permalink

In my state (Michigan) it is required to work towards and eventually get your masters to keep your teaching certification. Teachers are required to take classes on their own time and own dime. With the reward of keeping your job and a small pay increase (not enough to cover the expenses of the Masters). So you end up with another loan to pay off.

Very frustrating when public cries out for pay cuts and that teachers get paid too much. Here I am trying to save for my own children's college education and paying off my student loans at the same time:(

Permalink

I'm kind of surprised that this blog post ignores what seems to me to be the major reason that we won't end Gifted and Talented programs, which is that they largely serve middle income and white kids, whose parents wouldn't stand for them ending and have enough political clout to make sure they don't. If you look at all the other programs and policies that have been ended because they didn't raise test scores, they all impacted primarily low income kids of color. For example, who were the teachers who got fired? which schools have the lowest base pay and will tend to lose their more qualified teachers if they are no longer paid for their qualifications? Which schools have had the most experienced teachers fired? Which schools have been most often closed under NCLB? Which schools have lost things like art and music due to low test scores? Which schools are subject to "turnaround" strategies? In every case, it's the low income schools and the low income kids, and politicians feel pretty good about ignoring their parents when they speak out. The racial/socioeconomic climate in this country makes it easy.
Has anybody noticed that this whole "waiver" issue only arose when the NCLB cutoffs started to be too high for middle income, white schools to meet? They've been totally cool with closing down low income schools for ten years. Why now? It's not too tough to figure out.