John St Julien holds a doctorate degree in curriculum from LSU. He responded to The Advocate story this week about a proposed continued suspension of VAM for the evaluation of teachers. His comments appear at the end of the story in the Comments section and are reprinted below.
The problems with judging teachers using "Value Added
Measures," VAM, are not fixable. To be blunt: it was always a stupid idea.
First, the portion of the achievement of a student on standardized achievement tests that a teacher can affect hovers somewhere in the range of 15 to 20%. That is so small that really large numbers—certainly more than most teachers teach in a year— are needed to distinguish between a real effect and pure chance at any level of confidence that could be used to make a fair policy decision about a teacher's competence or lack of competence. The INEVITABLE consequence of having such an unreliable measure is that unacceptably large numbers of teachers will be labeled incorrectly. Because of its unreliability it will never serve as a decent signal to guide instruction; any measure that shows you incompetent one year and quite competent the next with no changes on your part will get ignored when it comes to practical changes. Which is not to say that the punishments won't be meted out—just that they will be unjustified and not useful for guiding change.
The VAM is also just not valid. It doesn't measure what it claims to measure: a teacher's contribution to a students' eventual abilities and success. More than a few studies have shown that teachers can have a large effect on student outcomes that are NOT measured on a standardized test and which have a huge effect on their eventual life chances and satisfaction. Teachers instill confidence, they challenge, they open new worlds. Teachers CARE and that alone is a hugely powerful factor. Most of a teacher's efficacy lies outside what is taught for and measured on a test that is taken by every student. Teachers do their real work by touching students' lives as individuals—and research generally confirms this by tying life outcomes such as college graduation to this class of factors. That special band teacher. The English teacher you had in the 9th grade who insisted that you THINK about what you'd just written in that essay. The steely-eyed history teacher whose unflinching vision changed the way that you saw yourself in the world. The science teacher whose passion for biology was catching. We do our children no favors by distracting our teachers from their real jobs with a series of standardized achievement tests.
The whole idea is absurd.
And it is made more ridiculous by the fact that we only really apply achievement-test VAM to a minority of teachers whose subject matter is easily tested—encouraging teachers to flee the very subjects policy-makers claim to think is most important. This policy and the associated punishments also discourage teachers from taking on the challenges of teaching underachieving students. The pious claim is often made that public policy to aid underachieving students requires disciplining teachers in order to fix one or another "achievement gap." Nonsense. The VAM does not reliably and validly measure what a teacher does; instead, it encourages teachers to do what is best for the mass of their students rather than to spend any time or effort on those who are least likely to make at least the average amount of "progress" on achievement tests. Those low achievers will be reliably those students whose families have an an income gap, a health-care gap, a housing gap, a safety gap, a confidence gap, physical challenges, and unique learning issues. All those factors will continue to affect the 80-85% of the "achievement test gap" that is due to things no teacher can fix in their students' lives. If public policy succeeds in making teachers care more about tests and less about the non-testable areas of needy kids in their classes, it will be precisely these children who suffer most.
VAM is poorly conceived policy that never had a chance of working and whose major effects are to worsen the very problems that it was supposed to fix.
It is past time to end VAM.
John St Julien
John St Julien
Posts
