Making Sense Of Florida’s School And Teacher Performance Ratings

Posted by on January 28, 2013

Last week, Florida State Senate President Don Gaetz (R – Niceville) expressed his skepticism about the recently-released results of the state’s new teacher evaluation system. The senator was particularly concerned about his comparison of the ratings with schools’ “A-F” grades. He noted, “If you have a C school, 90 percent of the teachers in a C school can’t be highly effective. That doesn’t make sense.”

There’s an important discussion to be had about the results of both the school and teacher evaluation systems, and the distributions of the ratings can definitely be part of that discussion (even if this issue is sometimes approached in a superficial manner). However, arguing that we can validate Florida’s teacher evaluations using its school grades, or vice-versa, suggests little understanding of either. Actually, given the design of both systems, finding a modest or even weak association between them would make pretty good sense.

In order to understand why, there are two facts to consider.

First, as shown here and here, Florida’s school grades are largely driven by how highly students score on tests (e.g., status measures such as proficiency rates), rather than how quickly they make progress (e.g., growth model estimates).

This is not only because 50 percent of the grades are based nominally on straight proficiency rates in four subjects, but also because another 25 percent comes from a growth measure that is rather redundant with proficiency, as a direct result of how the state codes students as “making gains” (see here). In addition, the absolute performance measures vary a great deal more than the “gains” scores, which means the former play a larger role in determining final school scores than their assigned weights suggest.

This doesn’t necessarily mean it’s a “bad” system – accountability is about incentives, and there’s a role for measures that capture both status and growth. It does, however, mean that Florida’s “A-F” grades tell you much more about the performance levels of students at a given school than the school’s contribution to those levels, which is why the grades are strongly associated with characteristics such as subsidized lunch eligibility (a rough proxy for poverty/income).

The second fact to keep in mind here is that Florida’s teacher evaluations, in contrast with its school grades, are intended to capture teachers’ performance in a manner that is, to the degree possible, independent of students’ absolute performance levels. Most notably, the value-added model that Florida uses controls directly for how highly students score in the previous year, which is the primary reason why the scores it yields, unlike those from the school grading system, are not correlated significantly with measurable student characteristics such as poverty (a finding to which state officials very eagerly called attention).

Similarly, the other major component of the evaluations – classroom observations – are also supposed to informally “control” for students’ performance levels. An observation score, to whatever extent it can be avoided, should not penalize teachers in classes with large numbers of lower-scoring students.

So, in summary: Florida’s schools grades are heavily driven by students’ absolute performance levels, while its teacher evaluation ratings are designed to be independent of those levels. Again, both are matters of degree, and there are other reasons to expect to find some level of concentration of “lower-performing” teachers in schools with lower absolute performance scores (e.g., recruitment/retention issues).

That said, Florida’s school and teacher rating systems are, by design, measuring different things. If anything, an extremely strong relationship between the grades and evaluation ratings might be seen as a red flag that the latter are biased. At the very least, validating one by assuming it must match up with the other is, to put it gently, inappropriate.

Perhaps we can give Senator Gaetz a pass for not being intimately familiar with the properties of these systems’ measures. But there are certainly people in Florida with a better grasp of these issues, and let’s just hope their voices are being heard.

- Matt Di Carlo


8 Comments posted so far

  • Your explanation was the closest I have seen as to what is happening but can you explain how ESE figures into any of the equations? Once again there are whole segments of students and teachers who were not figured into the equations except as an after thought yet those students scores on an entirely different test which has nothing to do with their educational needs affect the evaluations of teachers and grades of schools. Until we find a way to fairly evaluate students’ progress, teacher performance, and include all areas of education, there will be problems and confusion.

    Comment by Ann Smith
    January 28, 2013 at 8:58 PM
  • after reading the above gobbly-gook, i have only a few things to say, where ever it meets any agenda. 1) dump the Fcat testing program that has been thrust on the school children. When started, this fcat was suppose to be a test for the teachers but somehow was heaped on the heads of the student. The pressure to pass this piece of garbage was unending, thus taking away from basic and fundamental studies.2)demand that all teachers have back ground checks. 3)Demand that all teachers be certified through the state with a license.4) Insist that all teachers have the ability to teach without the constant use of computers, even as far as requiring the student to test on the device….not all students have access to these devices 5)Remind teachers that the public school classrooms are NOT a babysitting location for their elderly parents and relatives.6)Restore pride in school for the flag and pledge and return the Bible to the classrooms…7)last but not least, communication between teacher and parent is vital and should be a requirement,face to face no e-mails. This communication should be required between teacher and student.The best way to make sure students are learning is to teach the basics from American written text and not from political opinion.

    Comment by barbara durbin
    January 28, 2013 at 11:04 PM
  • [...] by Bob Sikes It’s not certain where the most recent work of the Shanker Institute’s Matt DiCarlo will take the debate on Florida’ education accountability system. DiCarlo weighed in [...]

    January 29, 2013 at 6:33 AM
  • Matthew–Florida is rated 45th in the Country in education –44th in the country with graduation rates –one of the highest in the Country with bullying problems…One of the worst in the Country with drop out rates 9-12 grades.

    Charter Schools are fudging their student grades to get higher State rankings and they are submitting false student counts to get more money from the State.

    Comment by lowell levine
    January 29, 2013 at 3:28 PM
  • Elephant in the room: The obsession with test scores occurs independantly from any proven connection to learning. This obsession flows from a marketing agenda. It has wild random error rates and no verified reliability in the evaluation of educators, administrators, or institutions. Rather, it is highly political in nature.
    http://m.washingtonpost.com/blogs/answer-sheet/wp/2013/01/30/e-mails-link-bush-foundation-corporations-and-education-officials/

    Comment by Mike Archer
    January 31, 2013 at 12:20 PM
  • The truth is parents have more impact on a child’s education than teachers. If a parent doesn’t care about the child’s education, neither will the child; if the parent blames the teacher for the child’s lack of performance, so will the child. Encouraging parental involvement, and providing mentorship when parental involvement is lacking is what is needed. Many elementary schools in Charlotte County are Title 1 and have a family center, which has various activities to encourage families to visit, and the schools have high scores. Not that these scores are due to the family center only, there are other activities and programs that help. The focus should be on figuring out what works. You wouldn’t punish a child for not knowing how to tie his shoes, you would teach him.

    Comment by Lisa
    February 4, 2013 at 11:26 PM
  • [...] Yes, but they use those scores differently, Matthew Di Carlo writes over at The Shanker Blog. [...]

    February 8, 2013 at 1:43 PM
  • [...] 2. School grades are based in part on student achievement, measured by test scores. If a class of second graders tests at a fourth grade level, their school may receive a passing grade from the state. School grades also calculate student growth — students’ improvement from year to year, with a focus on those who are struggling. But growth is not always a decisive factor in school grades, as Matthew di Carlo of the Shanker Institute spells out in detail. [...]

    February 18, 2013 at 1:09 PM

Sorry, the comment form is closed at this time.

Disclaimer

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the shankerblog.org may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.

Banner image adapted from 1975 photograph by Jennie Shanker, daughter of Albert Shanker.