I write often (probably too often) about the difference between measures of school performance and student performance, usually in the context of school rating systems. The basic idea is that schools cannot control the students they serve, and so absolute performance measures, such as proficiency rates, are telling you more about the students a school or district serves than how effective it is in improving outcomes (which is better-captured by growth-oriented indicators).
Recently, I was asked a simple question: Can a school with very high absolute performance levels ever actually be considered a “bad school?”
This is a good question. Read More »
In the Washington Post, Emma Brown reports on a behind the scenes decision about how to score last year’s new, more difficult tests in the District of Columbia Public Schools (DCPS) and the District’s charter schools.
To make a long story short, the choice faced by the Office of the State Superintendent of Education, or OSSE, which oversees testing in the District, was about how to convert test scores into proficiency rates. The first option, put simply, was to convert them such that the proficiency bar was more “aligned” with the Common Core, thus resulting in lower aggregate proficiency rates in math, compared with last year’s (in other states, such as Kentucky and New York, rates declined markedly). The second option was to score the tests while “holding constant” the difficulty of the questions, in order to facilitate comparisons of aggregate rates with those from previous years.
OSSE chose the latter option (according to some, in a manner that was insufficiently transparent). The end result was a modest increase in proficiency rates (which DC officials absurdly called “historic”). Read More »
Advocates of the so-called “Florida Formula,” a package of market-based reforms enacted throughout the 1990s and 2000s, some of which are now spreading rapidly in other states, traveled to Michigan this week to make their case to the state’s lawmakers, with particular emphasis on Florida’s school grading system. In addition to arguments about accessibility and parental involvement, their empirical (i.e., test-based) evidence consisted largely of the standard, invalid claims that cross-sectional NAEP increases prove the reforms’ effectiveness, along with a bonus appearance of the argument that since Florida starting grading schools, the grades have improved, even though this is largely (and demonstrably) a result of changes in the formula.
As mentioned in a previous post, I continue to be perplexed at advocates’ insistence on using this “evidence,” even though there is a decent amount of actual rigorous policy research available, much of it positive.
So, I thought it would be fun, though slightly strange, for me to try on my market-based reformer cap, and see what it would look like if this kind of testimony about the Florida reforms was actually research-based (at least the test-based evidence). Here’s a very rough outline of what I came up with: Read More »
There is currently a push to evaluate teacher preparation programs based in part on the value-added of their graduates. Predictably, this is a highly controversial issue, and the research supporting it is, to be charitable, still underdeveloped. At present, the evidence suggests that the differences in effectiveness between teachers trained by different prep programs may not be particularly large (see here, here, and here), though there may be exceptions (see this paper).
In the meantime, there’s an interesting little conflict underlying the debate about measuring preparation programs’ effectiveness, one that’s worth pointing out. For the purposes of this discussion, let’s put aside the very important issue of whether the models are able to account fully for where teaching candidates end up working (i.e., bias in the estimates based on school assignments/preferences), as well as (valid) concerns about judging teachers and preparation programs based solely on testing outcomes. All that aside, any assessment of preparation programs using the test-based effectiveness of their graduates is picking up on two separate factors: How well they prepare their candidates; and who applies to their programs in the first place.
In other words, programs that attract and enroll highly talented candidates might look good even if they don’t do a particularly good job preparing teachers for their eventual assignments. But does that really matter? Read More »
In recent months, the Common Core has come under increasing criticism from a number of different quarters.
An op-ed in the New York Times’ Week in Review is emblematic of the best of this disapproving sentiment. Yet even it mixes together fundamental misconceptions about the entire Common Core project with legitimate issues of inadequate preparation for teachers and students and poor implementation by state education departments and districts. The Common Core is described as a “radical curriculum” that was introduced with “hardly any public discussion.” We are told that it is a “one size fits all” approach, built upon a standardized script that teachers must use for instruction. Finally, it is suggested that the Common Core is a “game that has been so prearranged that many, if not most, of the players will fail.”
This is the Common Core seen through the prism of a fun house mirror. In truth, the Common Core is neither “radical” nor a “curriculum,” but a set of grade level performance standards for student achievement in the core academic disciplines of English Language Arts and Mathematics.* Indeed, one of the more telling criticisms of the implementation of the Common Core is that in all too many states, districts and schools, these standards have not been developed into curricula which teachers could readily use in their classrooms. Read More »
Our guest author today is Jose Vilson, a math educator, writer, and activist in a New York City public school. You can find more of his writing at http://thejosevilson.com and his book, This Is Not A Test, will be released in the spring of 2014.
Travis Bristol’s article on bringing more black men to the classroom has sparked a plethora of conversation around the roles of educators in our school system. If we look at the national educational landscape, educators are still treated with admiration, but our government has yet to see fit to create conditions in schools that promote truly effective teaching and learning. In fact, successful teaching in otherwise struggling environments happens in spite and not because of the policies of our current school systems.
Even as superintendents see fit to close schools that house large populations of teachers and students of color, we must observe the roles that educators of color play in their schools, whether they consider themselves “loners” or “groupers,” as Bristol describes in the aforementioned article. When the Brown vs. Board of Education decision came down in 1954, districts across the nation were determined to keep as many white educators employed as possible. While integration plays a role in assuring equitable conditions for all children and exposes them to other peoples, segregation’s silver lining was that Black educators taught Black children Black history. Racial identification plays a role in self-confidence, and having immediate role models for our children of color matters for achievement to this day. Read More »
Our guest author today is Heba F. El-Shazli. She has 25 years of experience in the promotion of democracy, independent trade unions, political and economic development. She has worked with institutions and leaders throughout the Middle East and North Africa (MENA) to challenge authoritarian regimes. Currently she is an adjunct professor at Georgetown University’s Center for Democracy and Civil Society. The views expressed here are her own.
Amid the political and social chaos that reigns in Egypt today, a semblance of normality persists: People go to work; they buy food; they try to feed their families. And, as in the past, Egyptians employers, with the active support of the Egyptian government, flagrantly violate fundamental workers rights. Workers are fired for trying to organize unions and they are not paid what they are owed, including legally mandated bonuses, profit-sharing and health care benefits or proper safety equipment.
There is a familiar political dimension to these events. Elements in the police and military are accusing workers who protest employer abuses of being “terrorists” — which in today’s Egypt means members or supporters of the Muslim Brotherhood (MB). In a recent New York Times article, two Egyptian media sources claimed that the workers striking at the Suez Steel company were infiltrated by MB activists attempting to “destabilize” the country.
This is an accusation supported neither by the facts, nor the history of blue collar unions. Read More »
Our guest author today is Morgan Polikoff, Assistant Professor in the Rossier School of Education at the University of Southern California.
A few weeks back, education policy wonks were hit with a set of opinion polls about education policy. The two most divergent of these polls were the Phi Delta Kappan/Gallup poll and the Associated Press/NORC poll.
This week a California poll conducted by Policy Analysis for California Education (PACE) and the USC Rossier School of Education (where I am an assistant professor) was released. The PACE/USC Rossier poll addresses many of the same issues as those from the PDK and AP, and I believe the three polls together can provide some valuable lessons about the education reform debate, the interpretation of poll results, and the state of popular opinion about key policy issues.
In general, the results as a whole indicate that parents and the public hold rather nuanced views on testing and evaluation. Read More »
Our guest author today is Travis Bristol, former high school English teacher in New York City public schools, who is currently a clinical teacher educator with the Boston Teacher Residency program, as well as a fifth-year Ph.D. candidate at Teachers College, Columbia University. His research interests focus on the intersection of gender and race in organizations. Travis is a 2013 National Academy of Education/Spencer Dissertation Fellow.
W.E.B. Du Bois, the preeminent American scholar, suggested that the problem of the twentieth-century is the problem of the color-line. Without question, the problem of the 21st century continues to be the “color-line,” which is to say race. And so it is understandable why Cabinet members in the Obama administration continue to address the race question head-on, through policies that attempt to decrease systemic disparities between Latino and Black Americans when compared to White Americans.
Most recently, in August 2013, U.S. Attorney General Eric Holder announced the Justice Department’s decision to reduce federal mandatory drug sentencing regulations. Holder called “shameful” the fact that “black male offenders have received sentences nearly 20 percent longer than those imposed on white males convicted of similar crimes.” Attempts, such as Holder’s, to reform the criminal justice system appear to be an acknowledgment that institutionalized racism influences how Blacks and Whites are sentenced. Read More »
Today marks the actual calendar day of the 50th Anniversary of the 1963 March on Washington for Jobs and Freedom. In honor of that day, we republish Al Shanker’s tribute to A. Philip Randolph, the director of the March, on the occasion of Randolph’s passing in 1979. One of the themes of Shanker’s comments is the distinctive place of A. Philip Randolph in the African-American freedom struggle, distinguished from Booker T. Washington, W. E. B. DuBois and Marcus Garvey, by his focus on the empowerment of African-American working people and his commitment to non-violent, mass action as the means of empowerment. One of the lesson plans the Shanker Institute has published for teaching the 1963 March focuses precisely on this distinctive contribution of Randolph. Other lesson plans look at Randolph’s close partner, Bayard Rustin, who was the organizing genius behind the March, and examine the alliance between the labor movement and civil rights movement which made the March a success. All of the Shanker Institute lesson plans can be read here.
It may be said – I think without exaggeration – that no American in this century has done more to eliminate racial discrimination in our society and to improve the condition of working people than did A. Philip Randolph, who died this week at the age of 90.
For A. Philip Randolph, a man of quiet eloquence with dignity in every gesture, freedom and justice were never granted people. They had to be fought for in struggles that were never-ending. And progress was something that had to be measured in terms of tangible improvements in people’s lives, in the condition of society generally, and in the quality of human relationships. Read More »
A couple of weeks ago, Mike Petrilli of the Fordham Institute made the case that absolute proficiency rates should not be used as measures of school effectiveness, as they are heavily dependent on where students “start out” upon entry to the school. A few days later, Fordham president Checker Finn offered a defense of proficiency rates, noting that how much students know is substantively important, and associated with meaningful outcomes later in life.
They’re both correct. This is not a debate about whether proficiency rates are at all useful (by the way, I don’t read Petrilli as saying that). It’s about how they should be used and how they should not.
Let’s keep this simple. Here is a quick, highly simplified list of how I would recommend interpreting and using absolute proficiency rates, and how I would avoid using them. Read More »
The change in New York State tests, as well as their results, has inevitably resulted in a lot of discussion of how achievement gaps have changed over the past decade or so (and what they look like using the new tests). In many cases, the gaps, and trends in the gaps, are being presented in terms of proficiency rates.
I’d like to make one quick point, which is applicable both in New York and beyond: In general, it is not a good idea to present average student performance trends in terms of proficiency rates, rather than average scores, but it is an even worse idea to use proficiency rates to measure changes in achievement gaps.
Put simply, proficiency rates have a legitimate role to play in summarizing testing data, but the rates are very sensitive to the selection of cut score, and they provide a very limited, often distorted portrayal of student performance, particularly when viewed over time. There are many ways to illustrate this distortion, but among the more vivid is the fact, which we’ve shown in previous posts, that average scores and proficiency rates often move in different directions. In other words, at the school-level, it is frequently the case that the performance of the typical student — i.e., the average score — increases while the proficiency rate decreases, or vice-versa.
Unfortunately, the situation is even worse when looking achievement gaps. To illustrate this in a simple manner, let’s take a very quick look at NAEP data (4th grade math), broken down by state, between 2009 and 2011. Read More »
Last week, the results of New York’s new Common Core-aligned assessments were national news. For months, officials throughout the state, including New York City, have been preparing the public for the release of these data.
Their basic message was that the standards, and thus the tests based upon them, are more difficult, and they represent an attempt to truly gauge whether students are prepared for college and the labor market. The inevitable consequence of raising standards, officials have been explaining, is that fewer students will be “proficient” than in previous years (which was, of course, the case) – this does not mean that students are performing worse, only that they are being held to higher expectations, and that the skills and knowledge being assessed require a new, more expansive curriculum. Therefore, interpretation of the new results versus those in previous year must be extremely cautious, and educators, parents and the public should not jump to conclusions about what they mean.
For the most part, the main points of this public information campaign are correct. It would, however, be wonderful if similar caution were evident in the roll-out of testing results in past (and, more importantly, future) years. Read More »
As reported over at Education Week, the so-called “sequester” has claimed yet another victim: The National Assessment of Educational Progress, or NAEP. As most people who follow education know, this highly respected test, which is often called the “nation’s report card,” is a very useful means of assessing student performance, both in any given year and over time.
Two of the “main assessments” – i.e., those administered in math and reading every two years to fourth and eighth graders – get most of the attention in our public debate, and these remain largely untouched by the cuts. But, last May, the National Assessment Governing Board, which oversees NAEP, decided to eliminate the 2014 NAEP exams in civics, history and geography for all but 8th graders (the exams were previously administered in grades 4, 8 and 12). Now, in its most recent announcement, the Board has decided to cancel its plans to expand the sample for 12th graders (in math, reading, and science) to make it large enough to allow state-level results. In addition, the 4th and 8th grade science samples will be cut back, making subgroup breakdowns very difficult, and the science exam will no longer be administered to individual districts. Finally, the “long-term trend NAEP,” which has tracked student performance for 40 years, has been suspended for 2016. These are substantial cutbacks.
Although its results are frequently misinterpreted, NAEP is actually among the few standardized tests in the U.S. that receives rather wide support from all “sides” of the testing debate. And one cannot help but notice the fact that federal and state governments are currently making significant investments in new tests that are used for high-stakes purposes, whereas NAEP, the primary low-stakes assessment, is being scaled back. Read More »
Recent events in Indiana and Florida have resulted in a great deal of attention to the new school rating systems that over 25 states are using to evaluate the performance of schools, often attaching high-stakes consequences and rewards to the results. We have published reviews of several states’ systems here over the past couple of years (see our posts on the systems in Florida, Indiana, Colorado, New York City and Ohio, for example).
Virtually all of these systems rely heavily, if not entirely, on standardized test results, most commonly by combining two general types of test-based measures: absolute performance (or status) measures, or how highly students score on tests (e.g., proficiency rates); and growth measures, or how quickly students make progress (e.g., value-added scores). As discussed in previous posts, absolute performance measures are best seen as gauges of student performance, since they can’t account for the fact that students enter the schooling system at vastly different levels, whereas growth-oriented indicators can be viewed as more appropriate in attempts to gauge school performance per se, as they seek (albeit imperfectly) to control for students’ starting points (and other characteristics that are known to influence achievement levels) in order to isolate the impact of schools on testing performance.*
One interesting aspect of this distinction, which we have not discussed thoroughly here, is the idea/possibility that these two measures are “in conflict.” Let me explain what I mean by that. Read More »