The Middle Ground Between Opt Out And All In

A couple of weeks ago, Michelle Rhee published an op-ed in the Washington Post speaking out against the so-called “opt out movement," which encourages parents to refuse to let their children take standardized tests.

Personally, I oppose the “opt-out” phenomenon, but I also think it would be a mistake not to pay attention to its proponents’ fundamental issue – that standardized tests are potentially being misused and/or overused. This concern is legitimate and important. My sense is that “opting out” reflects a rather extreme version of this mindset, a belief that we cannot right the ship – i.e., we have gone so far and moved so carelessly with test-based accountability that there is no real hope that it can or will be fixed. This strikes me as a severe overreaction, but I understand the sentiment.

That said, while most of Ms. Rhee’s op-ed is the standard, reasonable fare, some of it is also laced with precisely the kind of misconceptions that contribute to the apprehensions not only of anti-testing advocates, but also among those of us who occupy a middle ground - i.e., favor some test-based accountability, but are worried about getting it right.

For example, right at the outset, the article asserts that tests are “designed to measure how well our schools are teaching our children."

This is just not accurate. Tests are designed to permit inferences, however imperfect, about how well students know a given block of content (e.g., relative to other students).

Now, of course, we as a nation also have chosen to use these data to assess schools’ and teachers’ contributions to students’ progress. Done correctly and interpreted carefully, such analyses potentially yield useful information, even if reasonable people disagree on how and how much they should be used. Regardless, an important part of calibrating and designing that role is to understand the tests and what they can and cannot do.

Michelle Rhee is highly visible and wields vast resources. When she asserts that tests are constructed to do something they’re not, with scarce acknowledgment as to how little we know about using the data in this manner, one can understand why people feel nervous about the standardized testing enterprise.

Similarly, later in the article, Ms. Rhee goes on to offer the claim that opt-out advocates mistakenly think tests “are designed to pass judgment on students," and responds that the truth is “quite the opposite” – i.e., that tests are “an indicator of … whether schools, educators and policymakers are doing their jobs."

While “pass judgment on students” carries negative connotations (and thus strikes me as a kind of a straw man), the truth is that tests are, at least in many respects, designed for this purpose – to assess (again, imperfectly) students’ knowledge of the material. Moreover, to reiterate, using testing data to draw inferences about the performance of schools, educators and policymakers is enormously complex and difficult.

This distinction between the measurement of student versus school/educator performance is not semantic (and their conflation not at all confined to this op-ed). The flawed assumption that testing results are, by themselves, indicators of school/teacher performance is poisonous to both education policy and the debate surrounding it, It is, for example, reflected in the consistent misinterpretation of testing data in our public discourse, as well as the painfully crude, sure-to-mislead measures of NCLB.

Now, as stated above, much of the op-ed also offers the typical, fair arguments one hears in defense of testing, including an acknowledgment that the tests should be improved, that they are just one source of information, and that others should be considered as well. From my reading, however, the emphasis is very clearly on the value of testing data for accountability and personnel policy. For example, Ms. Rhee states that, in her opinion, the “conversation about standardized testing should focus on how we make sure every student, in every classroom, has an effective teacher."

Again, I agree that these could prove to be beneficial uses, even if I often disagree as to how they have been manifested in policy. And this is, of course, an opinion piece.

But the ideas that tests are specifically designed for these purposes, or that tests are an indicator of school/teacher performance rather than student performance, are cause for serious concern not only among strong opponents of test-based accountability, but also among those who, like myself, actually think it has potential if done correctly.

So, in general, when it comes to “opting out," what’s important to me is the idea that you don’t have to agree with its proponents’ solution to acknowledge that they may be correct about the existence of a problem. There are good and bad policy applications happening right now, and it’s important to address the bad ones and build on the good ones.

Doing so will require a strong understanding of testing data, and a nuanced grasp on how it should be interpreted and used. Hopefully these are the voices that will prevail.

- Matt Di Carlo

Permalink

While this blog correctly points out some of the flaws in Rhee's op-ed, it strains too much to occupy a mythical middle-ground between the opt-outers and the test-based accountability zealots. I'm glad you acknowledge that opt-outers are concerned about the over-use and misuse of standardized testing. However, your claims this reflects a view "that there is no real hope that it can or will be fixed." seems to miss the point that opting out may actually reflect a view that opting out is one way to get things fixed. This has some justification as we see what happened in Seattle's MAP test boycott and other boycotts popping up around the country where policies were changed. To claim these actions are a "severe overreaction" seems both baseless and callous.

While it's debatable whether test based accountability can be done right (whether this means "improves teaching/learning" or just "without causing harm" is unclear), it seems this effort has been big on promises and has come up short in delivering. Shouldn't we follow the motto of "first do no harm" when applying the use of standardized testing? Instead, you seem to be saying we just need to keep tweaking things and eventually we'll get it right. And, in the meantime, parents, teachers, and kids should just sit tight during this bumpy ride and hope we get things figured out. How come the ride is getting bumpier and bumpier?

I don’t think the middle ground is mythical, just that the voices of the people standing on it tend to be drowned out by the more adamant folks on either side. It’s certainly a cleaner sound bite to say test-based accountability does or doesn’t work than to venture into the potential pros, cons, and areas for improvement. Using the MAP test boycott as an example, it’s fair to point out the challenges to the validity of test results for English language learners and students that need test accommodations (this is true with any standardized test). Yet the computer-adaptive MAP actually represents an improvement over non-adaptive tests that are not especially sensitive to the growth of students not on grade level. So is it fair to say the road is getting bumpier? Or is it just that there’s more attention to the bumps in the road? That attention is not a bad thing if it serves as an impetus for people to use test data more wisely. In keeping with what Matt said, I think it’s important to focus the conversation on identifying better and worse policy applications around data use.

Permalink

@Cara - In the article, Matt acknowledges the legitimate concern about the overuse and mis-use of standardized testing - and in fact, he acknowledges "that they may be correct about the existence of a problem." In light of this, the main issue I raised w