MET Project Dissected

Math teacher and blogger Gary Rubinstein has dissected and uncovered serious flaws in the Gates sponsored study of effective teacher evaluation systems called the MET project (Measures of Effective Teaching). It turns out that if we look carefully at the data from the MET study we find that the basic assumptions for focusing on revised teacher evaluation and dismissal procedures as a way of improving public education are really not justified.

One of the most basic assumptions driving the major revamp of teacher evaluation and dismissal procedures in Louisiana is that poor teaching must be the primary cause of low student performance in some Louisiana schools. Our reformers (Jindal, White and big business) believe that all you have to do to improve education is to create a system that fires the worst teachers and trains all the remaining teachers to teach effectively. The new laws passed by Jindal will require the firing of the lowest performing teachers each year and provide all the others feedback on how their students are doing on state tests so they can set proper goals for improvement. Finally Louisiana teachers will watch videos of supposedly great teachers doing what is considered effective teaching so they too can learn how to do it. According to our reformers these actions should rapidly correct the problem of ineffective teaching in Louisiana public schools.  Very soon we should see a dramatic improvement in student test scores.

Part of the assumption about poor teachers in our supposedly backward, lackadaisical schools in Louisiana is based on some unconfirmed reports about great teachers who have improved student achievement in some low performing schools serving low socioeconomic status students. Supposedly great teachers can advance such students by as much as 1.5 grade levels in basic skills in one year. From those non-scientific reports some education thinkers concluded that three successive years of great teachers would bring any student up to grade level performance. Just insist that all teachers utilize these miracle teaching methods while demanding and expecting excellence from all their students and the problem of low student performance would be solved.

This one basic assumption is the reason why the reformers in Louisiana found it necessary to eliminate teacher tenure, do away with automatic salary step increases for teachers in favor of merit pay, and the need to destroy or marginalize teacher unions. That's because all of the above are simply unnecessary barriers to reforming teacher evaluation, pay, and dismissal. All of the above have been put into law in Louisiana in Act 54 and Act 1 and are now being implemented by our DOE. So Louisiana should quickly become the leader in the nation in student performance. No matter that all of these "advances" had already been in full effect in the Louisiana Recovery School District for 6 years and the RSD is still dead last among Louisiana school districts in student achievement!

Now we have the five million dollar MET project that is supposed to suggest the best formula for evaluating and removing teachers. Unfortunately, Gary Rubinstein points out that the MET researchers conveniently forgot to include the present system of teacher evaluation for comparision with other methods tested in the study. Because of the approach taken in the study, we do not have a direct way of comparing the results of the study of the new evaluation systems with what we are doing now. This is one of the major points made by Gary Rubinstein in his analysis of MET. In fact he points out that the evaluation options where much of the teacher evaluation is based on administrator observation, showed more reliable results and had good correlation to tests of higher order thinking skills.  Alternately, if all we care about is rote knowledge such as has been measured by some state tests, the way to go is by putting major emphasis on VAM based systems.

Finally, Gary points out that the data produced by the MET study shows that there is very little difference in the student VAM results by the highest performing teachers compared to the lowest performing teachers. There are almost no teachers producing the huge 1.5 year VAM gains in one year (and repeat performances are rare) and there are almost no teachers that produce very low VAM results. This conclusion by Rubinstein from the 5 million dollars spent on MET is that the major assumption that poor teaching is the cause of low student performance is highly questionable to begin with! Yet many observers believe that the highly contrived conclusions of the MET researchers will serve as justification for the growing emphasis on VAM as a major component in evaluating and dismissing teachers.

Louisiana is the gunea pig for this nationwide movement. I only hope other states do not ignore the real lessons from this massive expermentation with the lives of teachers and students.