On Education Polls And Confirmation Bias

Posted by on September 5, 2013

Our guest author today is Morgan Polikoff, Assistant Professor in the Rossier School of Education at the University of Southern California. 

A few weeks back, education policy wonks were hit with a set of opinion polls about education policy. The two most divergent of these polls were the Phi Delta Kappan/Gallup poll and the Associated Press/NORC poll.

This week a California poll conducted by Policy Analysis for California Education (PACE) and the USC Rossier School of Education (where I am an assistant professor) was released. The PACE/USC Rossier poll addresses many of the same issues as those from the PDK and AP, and I believe the three polls together can provide some valuable lessons about the education reform debate, the interpretation of poll results, and the state of popular opinion about key policy issues.

In general, the results as a whole indicate that parents and the public hold rather nuanced views on testing and evaluation. Read More »


No Presentation Without Representation

Posted by on June 26, 2013

I tend to comment on newly-released teacher surveys, primarily because I think the surveys are important and interesting, but also because teachers’ opinions are sometimes misrepresented in our debate about education reform. So, last year, I wrote about a report by the advocacy organization Teach Plus, in which they presented results from a survey focused on identifying differences in attitudes by teacher experience (an important topic). One of my major comments was that the survey was “non-scientific” – it was voluntary, and distributed via social media, e-mail, etc. This means that the results cannot be used to draw strong conclusions about the population of teachers as a whole, since those who responded might be different from those that did not.

I also noted that, even if the sample was not representative, this did not preclude finding useful information in the results. That is, my primary criticism was that the authors did not even mention the issue, or make an effort to compare the characteristics of their survey respondents with those of teachers in general (which can give a sense of the differences between the sample and the population).

Well, they have just issued a new report, which also presents the results of a teacher survey, this time focused on teachers’ attitudes toward the evaluation system used in Memphis, Tennessee (called the “Teacher Effectiveness Measure,” or TEM). In this case, not only do they raise the issue of representativeness, but they also present a little bit of data comparing their respondents to the population (i.e., all Memphis teachers who were evaluated under TEM). Read More »


Teacher Surveys And Standardized Tests: Different Data, Similar Warning Labels

Posted by on February 28, 2013

As a strong believer in paying attention to what teachers think about policy, I always review the results of MetLife’s annual teacher survey. The big theme of this year’s survey, as pushed by the press release and reiterated in most of the media coverage, was that job satisfaction among teachers is at “its lowest level in 25 years.”

It turns out that changes in question wording over the years complicates straight comparisons of responses to the teacher job satisfaction over time. Even slight changes in wording can affect results, though it seems implausible that this one had a dramatic effect. In any case, it is instructive to take a look at the reactions to this finding. If I may generalize a bit here, one “camp” argued that the decline in teacher satisfaction is due to recent policy changes, such as eroding job protections, new evaluations, and the upcoming implementation of the Common Core. Another “camp” urged caution – they pointed out that not only is job satisfaction still rather high, but also that the decline among teachers can be found among many other groups of workers too, likely a result of the ongoing recession.

Although it is more than plausible that recent reforms are taking a toll on teacher morale, and this possibility merits attention, those urging caution, in my view, are correct. It’s simply not appropriate to draw strong conclusions as to what is causing this (or any other) trend in aggregate teacher attitudes, and it’s even more questionable to chalk it up to a reaction against specific policies, particularly during a time of economic hardship. Read More »


Surveying The Teacher Opinion Landscape

Posted by on October 30, 2012

I’m a big fan of surveys of teachers’ opinions of education policy, not only because of educators’ valuable policy-relevant knowledge, but also because their views are sometimes misrepresented or disregarded in our public discourse.

For instance, the diverse set of ideas that might be loosely characterized as “market-based reform” faces a bit of tension when it comes to teacher support. Without question, some teachers support the more controversial market-based policy ideas, such as pay and evaluations based substantially on test scores, but most do not. The relatively low levels of teacher endorsement don’t necessarily mean these ideas are “bad,” and much of the disagreement is less about the desirability of general policies (e.g., new teacher evaluations) than the specifics (e.g., the measures that comprise those evaluations). In any case, it’s a somewhat awkward juxtaposition: A focus on “respecting and elevating the teaching profession” by means of policies that most teachers do not like.

Sometimes (albeit too infrequently) this tension is discussed meaningfully, other times it is obscured – e.g., by attempts to portray teachers’ disagreement as “union opposition.” But, as mentioned above, teachers are not a monolith and their opinions can and do change (see here). This is, in my view, a situation always worth monitoring, so I thought I’d take a look at a recent report from the organization Teach Plus, which presents data from a survey that they collected themselves.

Read More »


Who Has Confidence In U.S. Schools?

Posted by on October 24, 2012

For many years, national survey and polling data have shown that Americans tend to like their own local schools, but are considerably less sanguine about the nation’s education system as a whole. This somewhat paradoxical finding – in which most people seem to think the problem is with “other people’s schools” – is difficult to interpret, especially since it seems to vary a bit when people are given basic information about schools, such as funding levels.

In any case, I couldn’t resist taking a very quick, superficial look at how people’s views of education vary by important characteristics, such as age and education. I used the General Social Survey (pooled 2006-2010), which queries respondents about their confidence in education, asking them to specify whether they have “hardly any,” “only some” or “a great deal” of confidence in the system.*

This question doesn’t differentiate explicitly between respondents’ local schools and the system as a whole, and respondents may consider different factors when assessing their confidence, but I think it’s a decent measure of their disposition toward the education system. Read More »


Teachers And Their Unions: A Conceptual Border Dispute

Posted by on May 2, 2012

One of the segments from “Waiting for Superman” that stuck in my head is the following statement by Newsweek reporter Jonathan Alter:

It’s very, very important to hold two contradictory ideas in your head at the same time. Teachers are great, a national treasure. Teachers’ unions are, generally speaking, a menace and an impediment to reform.

The distinction between teachers and their unions (as well as those of other workers) has been a matter of political and conceptual contention for long time. On one “side,” the common viewpoint, as characterized by Alter’s slightly hyperbolic line, is “love teachers, don’t like their unions.” On the other “side,” criticism of teachers’ unions is often called “teacher bashing.”

So, is there any distinction between teachers and teachers’ unions? Of course there is. Read More »


Be Careful What You Think The Public Thinks About Tenure

Posted by on September 19, 2011

Monmouth University polling director Patrick Murray offered this response to my criticism of how he described tenure in a recent poll of New Jersey public opinion (see my brief reply and Bruce Baker’s as well).

I’m not particularly keen on arguing about the wording of poll questions. As I stated in my original post, wordings are never perfect, and one must always take this into account when trying to interpret polling results. I took issue with Monmouth’s phrasing because it is demonstrably inaccurate, nothing more.

But I do want to quickly add that, as is often the case, responses to poll questions about tenure are very sensitive to the nature of the description offered. A 2009 PDK/Gallup poll provides an illustration. Read More »


Teachers’ Preparation Routes And Policy Views

Posted by on August 22, 2011

In a previous post, I lamented the scarcity of survey data measuring what teachers think of different education policy reforms. A couple of weeks ago, the National Center for Education Information (NCEI) released the results of their teacher survey (conducted every five years), which provides a useful snapshot of teachers’ opinions toward different policies (albeit not at the level of detail that one might wish).

There are too many interesting results to review in one post, and I encourage you to take a look at the full set yourself. There was, however, one thing about the survey tabulations that I found particularly striking, and that was the high degree to which policy opinions differed between traditionally-certified teachers and those who entered teaching through alternative certification (alt-cert).

In the figure below, I reproduce data from the NCEI report’s battery of questions about whether teachers think different policies would “improve education.” Respondents are divided by preparation route – traditional and alternative. Read More »


Matt Damon, Jon Stewart And The “Teacher In The Family Effect”

Posted by on August 1, 2011

Over the past year or so, two high-profile celebrities – Jon Stewart and Matt Damon – have expressed skepticism about the market-based education reform policies currently spreading throughout the U.S. One cannot help but notice that they share one characteristic that they both acknowledge has helped to guide their opinions: Their mothers were both PK-12 educators. I’m also the son of a teacher and I know that this has had a substantial effect on my opinions about public education. No doubt the same is true of people who are married to teachers.

It’s hardly surprising that your occupation can help to influence the views of your family members, especially those pertaining directly to that career (i.e., education policy and teachers’ families). But I found myself wondering if there was some way to get a sense of just how strong this “effect” might be. In other words, how much more likely are non-teachers from “teacher families” – those with a mother, father, or spouse who is a K-12 teacher – to hold different views toward education policy, compared with non-teachers who don’t have any teachers in their immediate families.

Let’s take a very quick look. Read More »


What Do Teachers Really Think About Education Reform?

Posted by on May 18, 2011

There has recently been a lot of talk about teachers’ views on education policy. Many teachers have been quite vocal in their opposition to certain policies (also here) and many more have expressed their views democratically – through their unions – especially in states where teachers have collective bargaining rights.

We should listen carefully to these views, but it’s also important to bear in mind that there are millions of public school teachers out there, with a wide variety of opinions on any particular education policy, and not all of their voices might be getting through.

So, the question remains: How do most teachers feel about the current wave of education policy reforms spreading throughout states and districts, including (but not at all limited to) merit pay, eliminating tenure and incorporating test-based measures into teacher evaluations?

The logical mechanism by which we might learn more about teachers’ views on these policies is, of course, a survey. Unfortunately, useful national surveys are quite rare. In order to get accurate estimates, you need an unusually large number of teachers to take the survey (a deliberate “oversample”), and they must be randomly polled (lest there be selection bias). In my last post, I suggested that states/districts conduct their own teacher surveys.  In the meantime, some national evidence is already available, and if the data make one thing clear, it’s that we need more. When it comes to supporting or opposing different policies, teachers’ opinions, like everyone’s, depend a great deal on the details. Read More »


Are Americans Really Unwilling To Pay More To Prevent Education Cuts?

Posted by on March 14, 2011

In a speech earlier today, President Obama asserted, “We will not cut education,” and implied that doing so would be “reckless” and “irresponsible.” The president’s heartening remark, however, comes as  education funding is taking a massive hit at the state and local levels in most states, including New Jersey, Pennsylvania, Florida, and, yes, Wisconsin. The damage will likely last for many years.

In all the debate about what to cut and how deeply, there seems to be an assumption that an increase in revenue for education – to avert these massive cuts – is not an option. Although there are exceptions, very few Democratic governors are supporting tax increases to make up their states’ shortfalls, while Republicans governors are, of course, adamantly opposed.

Among many members of both parties, the presumption seems to be that raising revenue is simply a non-starter, because the American people are unwilling to pay more.

I’m not so sure. There is some evidence to suggest that this assumption deserves a second look. Read More »


Treating Teachers Like Professionals

Posted by on January 18, 2011

I was very interested to see, in a post by my colleague last month, that elementary school teachers were again voted among the top five most “honest and ethical” occupations in America, by respondents to a November 2010 Gallup Poll

According to Gallup, 67 percent of respondents rated the honesty/ethics of teachers as “high/very high,” 24 percent rated it “average,” and 6 percent rated it as “low/very low” (the error margin is +/- 4 percentage points). Only nurses, military officers, and pharmacists ranked higher (with doctors, who ranked 5th, in a statistical tie with teachers).

I found this interesting because it contradicts a key underlying feature of much of our public education debate. I’ve heard many thousands of teachers speaking out against the market-based reforms that are currently in vogue among opinion leaders, and seen them effectively ignored. I’ve heard everyone from Oprah to big-city superintendents to major television networks tout “Waiting for Superman” — a movie that supposedly focuses on teacher quality as the key to improving education, yet fails to interview even a single teacher. I’ve read hundreds of articles and posts that imply (and sometimes state directly) that teachers who oppose a favored policy do so because they “fear accountability,” or that they are more interested in their compensation and job security than in the children they teach. 

Many teachers call this type of behavior ”teacher hating” or a “war on teachers.” In my view, however, the fundamental issue here is trust. And the public’s continued faith in teachers does not seem to be shared by many of today’s pundits and policymakers. These same people say frequently that they want to “treat teachers like professionals,” but there’s a lot more to that than personnel policies. Read More »


Do Americans Think We Spend Too Much On Education?

Posted by on December 13, 2010

Cost-cutting is all the rage in education policy. This makes a lot of sense during a recession (the next few years will be brutal), and even during good times we all want money to be well-spent. But much of the discussion on this topic is less about weathering the storm than about a long-term effort to stop the growth of spending on public education. The underlying assumption, hardly unique to education policy, is that people are tired of increasing school costs, and want to start cutting back.

So, I wanted to take a quick look at what Americans think of education spending, now and over time, using data from the General Social Survey (1972-2008), a nationally representative sample of U.S. opinions and other characteristics (run by the National Opinion Research Center).  The question queries whether respondents believe the U.S. is spending too little, too much, or about the right amount on improving the nation’s education system (note the question’s use of “improving,” which likely influences responses to some degree).  Also keep in mind that these are pre-recession data.

The 2008 data in the table below (non-missing sample size is 993) show that there’s actually a lot of agreement about education spending levels: Almost 3 in 4 Americans (71 percent) believe that we should spend more on improving education, while only about 1 in 20 feels that expenditures are too high. Read More »


Extra Time: More From The Magazine’s Education Poll

Posted by on September 14, 2010

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post.

A recent education poll conducted by Time Magazine has gotten a lot of attention. Many of the questions are worded so badly that the results are rather meaningless. The question on merit pay, for example, defines the practice as “paying teachers according to their effectiveness” (who would oppose that, if it could be accurately measured?). Other questions are very interesting, such as the one asking whether respondents would pay higher taxes to improve public schools (56 percent would). Or the finding that, when asked what will “improve student achievement the most,” more than twice as many people choose “more involved parents” (54 percent) over “more effective teachers” (24 percent).

But, as is sometimes the case, a few of the survey’s most interesting results were not included in the published article, which highlighted only 11 out of 40-50 or so total questions (the full set of results is available here). Here are three or four unpublished items that caught my eye (the sample size is 1,000, with a margin of error of +/- 3 percent): Read More »


Disclaimer

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the shankerblog.org may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.

Banner image adapted from 1975 photograph by Jennie Shanker, daughter of Albert Shanker.