Wednesday, July 17, 2013

Zero Sum Game

There are many serious flaws in the Louisiana Act 54 evaluation system, but the most serious one is that it is a zero sum game. Here is the Wikipedia definition of a zero sum game:
zero-sum game is a mathematical representation of a situation in which a participant's gain (or loss) of utility is exactly balanced by the losses (or gains) of the utility of the other participant(s). If the total gains of the participants are added up, and the total losses are subtracted, they will sum to zero. Thus cutting a cake, where taking a larger piece reduces the amount of cake available for others, is a zero-sum game if all participants value each unit of cake equally (see marginal utility).

The way the VAM portion of the teacher evaluation system was designed, there will always be 10% of teachers each year rated as ineffective. This will happen even though student performance may improve from one year to the next. The teaching force will never be seen as improving even if student scores go up statewide, because this zero sum system always dictates that teachers be ranked based on their VAM scores and the bottom 10% are always classified as "ineffective".  Likewise the top 20% are considered highly effective, no matter whether student scores go up or down. If one teacher goes up another teacher must go down. It is a zero sum game!

Also since the principals are supposed to be rated based on the performance of their teachers, the principals with the most "ineffective" teachers each year must also be rated as ineffective. So far there is no quota set by the state for ineffective principals but we can expect that the DOE will try to insure that some principals are rated ineffective and possibly fired based on the teacher VAM ratings at their school.

Superintendent White recently announced that he wanted to give more autonomy and authority to principals in determining the final evaluations of teachers. He was quoted as follows:

“Over time you really need to give the local administrators total authority over this or they won’t be able to use it as a tool to help their teachers,”

Based on that statement, one may conclude that White is finally beginning to treat principals like professionals and intends to trust their judgement on teacher evaluation. Don't believe it for a minute. You see White and his staff have a very cynical attitude toward the education professionals in Louisiana. The following is an excerpt from an email by Erin Bendily, one of the top administrators at DOE, concerning Louisiana's evaluation systems:

Dec. 31, 2011 Erin Bendily, Asst.Superintendent at LDOE, writes that they need to push “more on the CCSS [Common Core State Standards] alignment/integration throughout. "This sounds harsh, but we should show that our current/old educator evaluation system is crap and the new system is stellar."

I need to point out that even though the COMPASS was based on the rubric developed by Charlotte Danielson, current national guru on teacher evaluation, the actual author of the COMPASS system in Louisiana was a person by the name of Rayne Martin. Ms Martin was selected for this task by former Superintendent Pastorek, even though she had no experience in teaching and had never served as a school administrator who would have evaluated teachers. But Erin Bendily, who is also a non-educator, could casually claim that Louisiana's old evaluation systems were "crap" even though they were developed over the years by experienced educators based on expertise and experience actually implementing evaluation systems. 

It turns out that Charlotte Danielson whose credentials for developing evaluation systems I question also, recently disavowed any connection with the Louisiana COMPASS system claiming that too many of her components of effective teaching were dropped from COMPASS.
Here is an email from a DOE official complaining about the Danielson objections:

July 18, 2012: Kunjan Narechania, White’s chief of staff wrote:  “Charlotte Danielson, expert on teacher effectiveness, is being a pain again. Apparently some reporter interviewed her about us using a version of her rubric for our system. She said she thinks it’s a bad idea for us to use an abridged version of her rubric and that we should have piloted for a year. So lame…” 

Concerning trusting principals to make decisions of teacher evaluation, I believe the following is the real truth: White does not care much about the evaluations of teachers above the ten percentile ranking produced by VAM. That's the group he intends to throw into the volcano every year to satisfy LABI and the other critics of public education (I guess these groups are like the gods of our modern society that have to be appeased each year) That bottom ranking 10% immediately lose tenure, are first in line for layoffs based on that one year ranking, and are placed on a path to dismissal and decertification no matter how their principal rates them. This result may not be so objectionable if the VAM were a precise system for rating teacher effectiveness. But the DOE data tells us that approximately 75% of teachers who fall in the bottom 10% one year will move up to a higher ranking the succeeding year even if they change nothing in their teaching. This tells us that the VAM is extremely erratic and most probably was wrong in its original rating of a teacher as "ineffective". So when that teacher who had her/his reputation forever sullied with that one ineffective rating moves up to the effective level the following year, another teacher who was previously rated effective must take her place as "ineffective". Remember this is a zero sum game! And this will happen every year even if student performance goes up significantly state-wide.

John White recently exempted about 50 teachers of high performing students from their initial "ineffective" rating because of an admitted flaw in VAM or because of the Seabaugh solution. . .whatever. But that means that when those 50 teachers are pulled out of the "ineffective" group, another 50 teachers have to take their place as ineffective. Remember this is a zero sum game!

But what about the teachers teaching in a B rated school that moved up from a C rating the year before? Teachers at such a school would be understandably elated because their efforts finally moved their school to a highly respectable rating. It means that the students there have significantly improved their performance on state tests. But how would such a gain affect the ratings of the teachers at this school the following year? Remember that the VAM ratings of the teachers the following year will be based on growth in student performance. So kids that may have performed above normal expectations in one year must grow again in performance as dictated by their newly calculated VAM expectations. These kids may not be achieving at high enough levels to have their teachers exempted as were the 50 teachers above, so they have to do even better the next year if their teachers are to get good VAM scores. So if they just continue to perform at the same good level, their teachers could be penalized even if the school maintains its B rating the following year. You see, the teachers at other schools will not need to show as much growth in their students to move ahead of the teachers in the newly improved B school. So you could have a whole bunch of teachers in a B school who get rated as ineffective the year after their school attained a B, even if their students are still doing well. Remember this is a zero sum game!

Just one more example: One of my readers wrote in some time ago to tell me about how VAM affected him. As an 8th grade teacher, he had achieved a high score on the trial run of VAM, so his principal asked him to "teach" some of his techniques to the 7th grade teacher teaching the same subject. He did assist the other teacher and found out later that this had an adverse affect on his VAM score the following year because his students were then starting at a higher baseline score. This is not what we signed up for as educators!

I could go on and give you more examples of how this new evaluation system is not only educationally absurd but is cruel and results in professionals being treated like helpless pawns in the Jindal/White destruction of public education. That's why  I believe as professionals, we should not allow ourselves to be meekly herded down the slaughter house chute. Educators need to loudly object to zero sum evaluation systems and to all so called "reforms" that damage the teaching profession.


Anonymous said...

Excellent post, Mike. More specifically, a zero-sum game in mathematics (game theory---a focus of my undergrad degree in math), has a winner (+1) and a loser (-1) for a total of 0 for whatever the game is. In this case, 'accountability'. ~ba

Kathy Irwin said...

Dear Mike: I read one of your posts on the Diane Ravitch site. Was struck by your "amateur educator" characterization. It really does come up over and over that the individuals overseeing the train wreck, themselves have neither the professional prep nor hands-on, years in the job experience. No plumber would be selected or promoted with this lack of credentials.

Anonymous said...

I would like to just add another personal VAM experience for you. During a trial run of VAM, I was ranked in the 98th percentile- so my rating was Highly Effective. I now teach in a different school- but same subject and grade level. I was rated Ineffective this past year. Fortunately my administration is supporting me, and I am not facing termination. I am also filing a grievance to the DOE even though a system is not yet in place. It is unfair to expect me to sit back and accept this Ineffective rating that could potentially ruin my career. It is obvious to anyone who looks at my scores to realize this VAM model is seriously flawed. A drop of 90 plus percentile points for any teacher should be cause for investigation and question.

Michael Deshotels said...

I think the above example by anonymous is shocking and would not be tolerated in any other profession. If you are a member of LAE or LFT I hope you will request the assistance of their legal services to fight this obvious injustice. The developer of VAM in Louisiana made it very clear in one of the videos I have featured on this blog that one year of VAM is not enough to justify this rating.

Anonymous said...

Yes, it is shocking, and completely inaccurate. The qualitative portion of my rating was Effective. I exceeded my SLT's. My students' scores were at the state average as my school expected. I have not been able to see individual data yet to understand why I ranked in the bottom 10th percentile- and I do not understand why we weren't provided with their predicted scores in August. If Johnny was expected to score a 350, it would have been helpful to know that. I was a member of LAE for 6 years while I taught in the regular public school system. I now work at a charter school. I was under the impression that LAE and LFT do not support charter school teachers.

Michael Deshotels said...

Anonymous, LAE and LFT support all professional educators who want to be members no matter where they teach. I know that recently LAE became the collective bargaining agent for a school in New Orleans. I hope you will reach out to them in the future. Many charter school teachers read this blog. I believe teachers who teach in charters are really in the same boat as regular public school teachers except that they have fewer protections against capricious actions by their administrators. We are all in the same battle to protect professional educators.

Anonymous said...

To anonymous with the unfair VAM score: google this Advocate article--Teacher Reviews Spark New Flap. and read the comments from DrJamesFinney below the article and his knowledge of the algorithms behind the VAM scores. He has offered his services to teachers to assist in exposing the inconsistencies and to try to gain further understanding of this rating tool, if possible. It could at least provide you with the knowledge you will need to counter/appeal your score. You may want to consider contacting him. His contact info is at