Monday, October 22, 2012

Can VAM be Fixed?

Superintendent John White believes he has a solution to the problem of teachers in top rated schools receiving an “ineffective” because of the VAM portion of the new teacher evaluation system. It was discovered recently that some high performing students in exclusive magnet schools sometimes experience a lull or slowdown in their academic gains caused by factors outside the control of the teacher. Even though such students are normally high performers, in some cases a classroom of high performers may perform below its VAM predicted scores on state tests, thereby placing their teacher in the bottom 10% on the VAM ranking statewide. When simulations showed that several teachers at the top rated elementary school in the state may be classified as ineffective by VAM, the whole community and their state representative were up in arms. When this story broke in the newspapers, Superintendent White immediately adjusted his schedule to travel to Shreveport and met with teachers and administrators at South Highlands Elementary Magnet School.

Within just a few days of the discovery of the flaw in VAM for high performing schools, Superintendent White had a solution. He will be proposing to BESE that in cases where some teachers who teach high performing students get an “ineffective” from the VAM, the state will simply wipe out the VAM portion of the teacher's score and use the qualitative portion of Compass as the only factor in determining the rating of the teacher. This exemption to VAM will be applied as long as a teacher's students score at the top two levels on state assessments, even if her/his students do not show the growth the VAM formulas demand. That's a great victory for the teachers at South Highlands Elementary school and the teachers there certainly deserve to be exempted from this flaw in the evaluation system.

I believe White's recommended change for South Highlands and other similar schools will be approved by BESE, not because of the extenuating circumstances that may cause an unfair evaluation of those teachers, but because the legislator representing that district is a solid supporter of Governor Jindal and his “reform” programs. White, who has no training in teacher evaluation, no training in statistical analysis, and who has never evaluated teachers, came up with an instant solution that will fix the problem for that select group of teachers. But White's solution will still allow thousands of other teachers in the state to be vulnerable to ineffective ratings based only on the VAM even if they have extenuating circumstances in their classrooms.

For example let's look at another teaching situation. (The following is a hypothetical example because I don't have access to actual the value added growth that VAM predicts for various socioeconomic groups.) Lets consider a high poverty middle school in the inner city that has had most of its high performing students transfer to magnet schools. A 20 year math teacher (teacher B) is assigned a group of 25 students where the VAM formulas project that those students should show composite growth of .8 years in math for that year. But during that year, two of the girls get pregnant, 3 of the boys are picked up on drug charges and detained in a juvenile institution for three weeks to a month, and three other students' families were evicted from their home because their unemployed parent could not pay the rent. Those students had to live with relatives and friends where they were lucky to find a bed in which to sleep, much less find a quiet place to study. One other student's mother was murdered by her estranged husband causing traumatic shock waves throughout the community. LEAP testing produced a composite growth in math of this class of only .4 years. I guess you could say this teacher's class also experienced a lull or slowdown in academic gains probably caused by factors outside the control of the teacher. Teacher B received a good rating from her principal on the qualitative portion of Compass, but her low rating on VAM placed her in the bottom 10% statewide, so she received an overall ineffective rating. There is no appeal for an ineffective rating on VAM. That teacher immediately loses her tenure, has her salary frozen, and is placed first on the list to be laid off next year, in the event the school system is forced to reduce teaching staff. That teacher is all alone. She did not have a state legislator who was aligned with the Governor to campaign for her and get a special rule change for her extenuating circumstances. There will be no adjustment in her evaluation.

Here's another example deserving consideration. Some teachers have pointed out that there seems to be a difference in average state test scores for students from one grade level to another. That difference could be caused by several factors. It could be that the state tests do not increase smoothly in difficulty from grade to grade. LEAP and ILEAP tests are constructed by contracted testing companies that make a huge profit regardless of variations of their tests. Or it could be that students in one year where state policy requires retention of students who do not make the state cut off score work a little harder to pass the test that year than they do the previous or succeeding years. That can have both adverse and beneficial effects on ratings for teachers in different grades even if teacher performance is generally the same from grade to grade. Would White recommend a change in the teacher evaluation system in such a case? Specifically using real data if we compare the 4th grade LEAP scores of all students statewide in 2011 to the ILEAP 5th grade scores we find that 24% of students scored below basic in ELA in 2011 while 30% of those students scored below basic in 5th grade in 2012. Statistically this means that more 5th grade ELA teachers are expected to fall in the bottom 10% of VAM than 4th grade teachers because their students don't perform as well on state tests. If a disproportionate percentage of 5th grade teachers are rated as ineffective because of the factors I mentioned above, will some legislator go to bat for those teachers and get them an exemption from VAM?

One more point. It turns out that the Governor's supporters in the legislature generally represent the more affluent areas of the state. Opponents of the Governor generally represent impoverished communities. Which teachers do you think have the best chance of getting exemptions from or adjustments to VAM? The ones teaching in high poverty schools or the ones teaching in more affluent schools?

Can VAM be fixed so that it will be fair to all teachers and still produce the mandatory 10% of teachers rated as ineffective? Obviously not, because for every group of teachers granted special exemptions, the system will have to pull more or the rest of the teachers down to the the ineffective level.

I hope all teachers and administrators will stick together in opposing the continued implementation of this fundamentally flawed evaluation system. Right now I believe most teachers are appalled at the lack of accuracy and now also the lack of integrity in the decision making process relative to this evaluation system. It cannot be fixed. It must be junked. Let's not allow some good teachers to be thrown under the bus while others get an exemption.


Lee Barrios, M.Ed., NBCT said...

High stakes testing has never been a valid measure of learning for students and will never be a valid measure of effectiveness for teachers. Why do parents allow their children to be stamped with a bogus "grading " system? OPT OUT. Federal law places parent choice above all else. Why are teachers subjecting themselves to VAM? Teachers need to say NO. Yes it is that easy. Call a faculty meeting, everyone agree to disagree, don't enter the data or screw up the data entered. At some point we all have to stand up for what we know is a sham.

Again White exposes his lack of any educational expertise. #1. If teachers with high performing students should be exempt, it would indicate that all teachers of high performing students are effective and their students' high performance is a result of their stellar teaching. I am perfectly willing as a teacher of gifted to acknowledge that as a false assumption.

#2. If teachers are in inclusive classrooms with a mixture of high performing students and low performing students, the previous false assumption would indicate that the teacher is effective but the low performing students are. . . well. . .low performing. I'm not going to go there because White would accuse me of making excuses like poverty, IQ, lack of parent involvement, hormones, violence at home etc. He would have to assume I had self-serving motivations of one kind or another and was selectively giving attention to my "pets."

#3. This scenario in #1 would mean that teachers with classes of all low performing students (an anomaly unless artificially produced via strategies like selective enrollment, siphoning off students to magnet schools, vouchers, alternative schools etc. hhmmmmmm) would mean that teachers of low performing students are ineffective and should be fired. I dare say that John White would agree with that because he agrees with our new COMPASS method of evaluation.

So if student performance is in fact a direct result of teacher effectiveness it would mean that #1. Teachers should only agree to teach high performing students or #3. Students who are low performing should be grouped and assigned state certified unionized teachers w years of experience so White can with good conscience have them fired or grouped in classes assigned Teach For America instructors who will only last two years anyway and have no certification to lose and can then move up through the TFA ranks to communications director or even State Superintendent.

My suggestion would be to scrap the bogus COMPASS system and adopt a system that uses multiple measures of assessment such as quite a few offered up by organizations like the National Board for Professionaljjj Teaching Standards. NBPTS could even do the evaluations for the state as a far lesser cost than COMPASS and thereby assure quality control.

Anonymous said...

If high performing students can score a little lower from one year to the next, the same is true for all students- regardless of their achievement level the previous year. In granting the exception for South Highlands and similar schools, White has admitted that VAM is flawed. Now he needs to acknowledge that the flaws occur system-wide!

Will White do so? I doubt it. He and Gov. Jindal have too much invested in the reform movement to admit their failure.

Anonymous said...

So now they have succeeded in pitting teachers against teachers. The exemption for the teachers who have AIM and honors and AP classes because only that group of students is affected by family crisis or social ills?
The state will just sacrifice other teachers to make sure we make the states 10% quota?
I have always loved teaching the kids who struggle, challenge me daily and have unpredictable gains, but they still gain. Now I wonder if I did the right thing by refusing offers over the years to move to AP or dual enrollment classes? NO teachers should be subjected to VAM.
Now legislators are getting involved? Of course the rich areas will get support to get out of this mess and the teachers battling the effects of societies ills in schools in inner cities, rural areas far from support and without the taxes or jobs to make a politician rich will still be subjected to this unethical process? Jindal and White should be charged with malfeasance of office and unethical conduct.

Anonymous said...

There are SO many inherent flaws within the COMPASS evaluation system. This is only one of many injustices in the system. In no way should a state test be a measurement of teacher effectiveness, or 50% of a teacher's effectiveness. It is absurd. I have no problem with accountability as I am sure most teachers do not. It is the fairness of the system of which we are being held accountable that is the issue. I have a major issue as a core subject teacher in middle school with the fact that I am held to a different standard than my coworkers who decided they wanted to teach art, music, PE, or a foreign language. They do not have to worry about VAM data at all. Their school year is going to much less stressful than any teacher of a tested subject, yet they make the same money that I do. How is that fair? Why would anyone now choose to teach a core subject or tested grade over electives? It makes no sense. Even if they have to come up with SLT's, they are still not held to the same standard as tested subjects. Just SO many inequalities and injustices with COMPASS that I don't understand how it has not been thrown out. It seems common sense to me that determining student growth should come from a culmination of measures such as 6 or 9 week grades, standardized test data, and evaluations.