Serious Flaws in VAM

The subject of this post is extremely critical to teachers and principals in our public schools.  The new laws connecting teacher evaluation directly to layoff, salary, and tenure should require that any evaluation system be as accurate as possible.  But it seems like our education hierarchy at the state level comprised as it is of rank amateurs, is destined to make one blunder after another. I believe the evidence presented below demonstrates that the Value Added Model portion of the teacher evaluation system now being implemented by LDOE contains serious flaws that can unfairly destroy the careers of many good teachers and principals.

I mentioned in one of my recent emails that there was ample evidence coming directly from Dr George Noell (the father of VAM in Louisiana) that the value added model is erratic and unreliable in measuring good and bad teaching. Today I would like to discuss with you the clear evidence for this conclusion.

Wayne Free, Director of Instruction and Professional Development for the Louisiana Association of Educators has just written an extremely good analysis of Louisiana's Value Added Model. From this analysis we can easily conclude that the Louisiana VAM should never be used to evaluate the performance of teachers, should not be used to deternine the order of lay off of teachers, for freezing a teacher's salary, or for removing tenure and placing teachers on a path to be fired. Yet despite all the evidence I am about to present here, all of these actions that are so destructive to the teaching profession and to the education of our children will soon start happening. Please click on this link to the LAE website for Mr Free's analysis. For this post I will just summarize one of the most critical findings in his study.

The most damning evidence that this VAM is far from ready for prime time comes from several answers to direct questions posed by Mr Free to Dr Noel. Here is the most outstanding one:

Free asked: "What percentage of the teachers rated in the bottom 10% by the VAM in one year would be expected to be rated again as ineffective the following year if they did nothing to change their teaching from one year to the next?" ( The following is my comment) I believe that if this system is accurate in identifying ineffective teachers, the answer would be somewhat close to 100%, especially if the system is considered reliable enough to destroy a teacher's career. Right? ... Wrong!)

The percentage possibility of a repeat of an ineffective rating in the succeeding year according to Noell is only 26.8%. And that is assuming the teacher does not change his/her teaching. This was shocking to me. The fact that there is only a one in four chance of the same result the following year means that the system is erratic and may have incorrectly identified large numbers of teachers as being ineffective in the first place. It means that contrary to the assumption in the legislation, that the process will be deliberative and free of gross errors, such teachers will immediately be placed first in line for layoff, have their salaries frozen, and those rated ineffective next year will permanently lose their tenure and be placed on a path to dismissal. But the most damaging result of such an incorrect determination I fear, is that the local newspapers will somehow get the list of the teachers rated as ineffective and publish it in the home town paper where a teacher's reputation will be permanently destroyed and parents will be demanding that their children not be assigned to that teacher's class. This could easily happen this year to a teacher with 20+ years experience and a previously perfect evaluation record!

One of the most unfair requirements of the evaluation system, is that even if a teacher gets a great evaluation on the qualitative (Compass) from her/his principal, but gets an ineffective rating on VAM (quantitative) the final result is required to be an automatic overall ineffective rating. This rule is in direct violation of Act 54 which states that VAM is supposed to make up 50% of the overall result. (not 100%) This is sure to result in numerous lawsuits. I suggest that all teachers vulnerable to VAM join their teacher association/union now so that their lawsuits can be funded if necessary.

A perfect example of how crazy the results can be is the strange outcome reported in an editorial published recently by the Lake Charles American Press. The American Press editors were appalled by the fact that the VAM system is set up to find 10% of all teachers evaluated as ineffective no matter how well or how poorly they actually perform. This is called grading on the curve, and is not allowed in any public classroom. We measure and grade our students by how well they master the material. If 80 or 90 percent of the students master the material, that many will get an A or a B, and if no student fails, then that is something we celebrate. Not so with the teacher evaluation system. It has been predetermined that a minimum of 10% of our teachers must fail. The American Press editors say that such a system borders on being immoral!

But it gets worse. The editors found that for some strange reason, in the pilot program that was run recently by LDOE, one of the best performing school systems in the state (Jeff Davis) had only 3 percent of its teachers rated as highly effective, and a disproportionately high percentage rated as ineffective. In fact several of the highest performing districts in the Lake Charles area had some of the lowest teacher VAM scores. How can that be? This system is both illogical and destructive! See the editorial in the American Press. Not only are teacher evaluations adversely affected by this tendency, but their principal's evaluations could also be damaged because of the number of "ineffective" teachers in their school.

 The Baton Rouge Advocate carried a story recently about the teachers of a high performing school in Shreveport also getting disproportionately low VAM scores in the pilot program. Our amateur Superintendent White is quoted as characterizing this as an isolated glitch which can easily be corrected. Maybe his 12,000 per month (part time) PR person can somehow spin this fiasco as a positive.

What's worse, as is implied by White in the Shreveport situation, the LDOE managers of VAM can change the complicated formulas that produce the value added results at will without consulting anyone. So they can manipulate the results. They are probably doing just that as this post is being written and you and I have no way of knowing. I wonder what group of teachers will be penalized next time after the formula has been tweaked to take care of Shreveport? And your professional career depends on this foolishness?

For more information on flaws in VAM in general as it is used in several other states you may want to read an analysis by Gary Rubinstein. Thanks to Cathy, one of our readers for suggesting this link.

If you as as outraged by this as I am, would you please send an email to your legislator, who can easily change this in the next legislative session before it affects anyone. In my last post I gave you a link to the legislative ID system that gives you email addresses of your legislators. I know many of you meant to send emails at my last request but you may have been too busy. I know how busy you are but this is critical!

Here is a sample email: Dear (insert your legislator name), The most recent information available about the new Act 54 evaluation system clearly shows that it is an unreliable system for rating teacher performance. I hope you agree with me that the stakes are too high and that professional educators should not be rated by a system that is flawed. Please take whatever steps necessary to stop this evaluation system immediately and not implement it or any other evaluation system until it is proven reliable. Thank you for supporting your professional educators in this critical matter.
Signed: ____.

There is one more thing I am asking you to do right now. If you have not already done so, submit your contact data to me so you can become a part of my Defenders of Public Education data base. The educators, school board members and parents in this data base will receive timely notices of upcoming actions at BESE and the legislature so you can contact your representatives before they vote. Believe me all of us together can make a difference! Just send me an email at louisianaeducator@gmail.com with your home address or if you prefer send me the district numbers for your representative and senator (it will save me the trouble of looking them up). Include your preferred email address so that my emails can reach you anytime your help is needed.

Believe me we can change this is we just stick together and speak as one!

Thanks again for all you do as an educator or parent or school board member. We can never thank you too much for your valuable contribution to public education.
Michael Deshotels