Saturday, May 14, 2016

Can VAM be Fixed? Judge Throws Out VAM Eval. in New York

This Washington Post article reports that A New York judge has ruled that an "ineffective evaluation" of a  teacher because of the use of the Value Added Model (VAM) was arbitrary and capricious and invalided the evaluation. This case could have implications for Louisiana teachers because a very similar scheme is used as part of the Louisiana VAM evaluation system.

The Louisiana legislature is now in the process of considering legislation that would modify the Louisiana teacher evaluation plan by reducing the VAM portion of a teacher's evaluation from a weight of 50% to a new weight of 35%. Senate Bill 477 (click on bill # to see text)  by Senator Boudreaux has passed the Senate and is headed to the House Education Committee for consideration next. This bill is supposed to be a compromise agreed to by the hard line proponents of VAM (The Louisiana Association of Business and Industry or LABI) and the Edwards administration and teacher organizations. But even if this change in the weight of VAM is approved by the legislature, there are still other matters not yet settled that are similar to the New York case that could still derail VAM in Louisiana.

The major problem still remaining with VAM in Louisiana is because of the arbitrary ratios of teacher performance levels in the VAM calculation. The use of a Bell Curve to arbitrarily rate a certain percentage of teachers (10%) as ineffective each year and then a certain percentage (20%) as highly effective regardless of overall student performance will not hold up in court in my opinion. (See this video by Herb Bassett) The judge in New York specifically ruled that there was no justification for using a bell curve to determine a certain percentage of teachers ineffective. That was part of the reason for his "arbitrary and capricious" ruling against John King who was state commissioner of education in New York at the time of the evaluation in question. Some critical flaws in the VAM system were obvious from the start.

Within a few months of John White being appointed as State Superintendent of Education in Louisiana, I was granted a meeting with him to discuss certain issues I wanted to bring to his attention. The plan for the new Louisiana teacher evaluation system which included VAM had just been released but had not yet been approved by BESE. I wanted to try to convince White that two parts of the plan were flawed and should be changed. By the way, the draft of the teacher evaluation plan had actually been written by a non-educator (Rayne Martin) who was serving on the State Department of Education staff for a brief period. It was partially based on an observation system developed by Charlotte Danielson, who later disavowed any support for the Louisiana system because it did not effectively adhere to her program. In the course of my meeting with White, I asked what was the rationale for the 10% ineffective rating each year for all teachers who teach state tested students? He could not give me a rationale except that he felt that this was a reasonable percentage. I posed to him the following scenario: Assuming that the purpose of VAM is to encourage overall improvement in student performance by evaluating teachers using the Value Added Model, what would happen to teacher evaluation results if Louisiana actually produced better overall performance over a period of several years? The VAM system requires that the bottom 10% of the teachers get a rating of "ineffective" each year. But if student performance goes up all over the state and suppose that 98% of the teachers have more growth in their students scores than the year before, why would the evaluation system still find 10% of teachers ineffective? But that's what it does every year without regard to overall student growth. White did not make the change in the Louisiana plan when he should have. This is part of the reason the New York judge ruled that state's evaluation system arbitrary and capricious.

The reduction in the percentage of a teacher's evaluation based on VAM reduces the impact of an "ineffective" VAM component on the final evaluation after figuring other student growth factors and the observation component, but it could still put VAM rated teachers at a disadvantage compared to other teachers rated using Student Learning Targets. Also, it should still be expected that a smaller percentage of VAM rated teachers would be able to qualify for a highly effective rating and benefit from merit pay as compared to teachers of non-tested subjects.

Important notice: I reviewed my notes on the recommendations of the special subcommittee of the Accountability Commission that was required by a law passed by Rep. Hoffmann to make recommendations to BESE for changes in the evaluation system. I found that the subcommittee recommended that the "override" provision of the evaluation system be removed from the evaluation plan. The override provision mandated that when a teacher got an "ineffective" VAM score, the qualitative portion (the principal's evaluation) of the evaluation was overridden and the teacher received an "ineffective" overall evaluation. It is my understanding that this provision has been removed. Also, an official of the LDOE has informed me that the latest revision of the evaluation plan allows the principal to raise or lower the VAM by one level of performance based upon other measures of student progress.


My point is that the so called "compromise legislation" in SB 342 still leaves important issues in the teacher evaluation system unresolved. John White, in a letter to district superintendents has praised the VAM compromise. Now John White has another opportunity to try to fix the VAM system. Will he do his part in fixing the rest of VAM? But the real question is Can VAM be Fixed?