Standardized Test Score Setting is Looking More and More Like Sausage Making

Forget About Common Core; Forget About the New Revised Louisiana Standards Now Being Drafted; It’s the State Tests and Their Cut Scores That Are the Real Standards!

Most professional educators working in our public schools have figured out by now that the standards and the curriculum in our schools today are one thing and one thing only: the state tests. The annual LEAP and iLEAP tests are the only things that determine what the basic skills teachers will be forced to teach. In addition, the cut scores  on these tests that are set by non-teacher testing specialists will determine what percentage of students will score Basic, Mastery or Advanced. Also the cut scores will determine what grades will be ultimately assigned to each of our public schools.

The newly appointed standards review committees, the Accountability Commission and then BESE think that they are making real decisions about the standards, but their decisions are greatly overshadowed by the test designers and the cut score deciders. Superintendent White knows he can adjust the standards in any way he wants from year to year by either changing the ratio of difficult to easy questions on state tests or by changing the cut scores.

This Spring, our students in grades 3 through 8 took their usual LEAP and iLEAP standardized tests in science and social studies accompanied by Common Core aligned tests in math and English language arts based on the new PARCC test. Schools and parents were told that grading would take much longer this year and that the results would not be released until sometime in the Fall. The test results will contain no diagnostic results for individual students. They are designed mostly to measure the effectiveness on the schools and teachers in teaching the new Common Core standards. It is believed by some reformers that just requiring the new Common Core standards and testing students each year will somehow drive our schools to a higher level of excellence that will prepare our students to be much more competitive in the world job market. You know, just the way that No Child Left Behind got every student to the level of proficiency over a ten year period.

Over the previous three years, state tests had already been incorporating increasingly more Common Core aligned test questions in preparation for the 100%  PARCC test given this Spring. Upon release of the Spring 2014 test results, the LDOE announced that even though the tests in math and ELA were more challenging, our students performed just as well as the year before according to the number of students achieving a level of Basic on the tests. The press release also stated that the percentage of students achieving Mastery had increased slightly. What they neglected to tell us was revealed as a result of a public records request and a public records lawsuit. That is the fact that the cut scores for both Basic and Mastery had been lowered significantly on 3 out of 4 tests to basically insure that the number of students scoring at those levels would not drop. The students actually performed significantly lower on the new tests but the grading scale was lowered to make it look like they had done just fine.

The official score range for achievement of Basic on the 7^th grade math test for 2014  was from 292 to 375 out of a total possible 500 points. This was the same standard scale score range that was used for the 2013 7^th grade math test. But what the state didn’t tell us was that the percentage of correct answers that was translated to a scale score of 292 was only 38%. That 38% is what is called the raw cut score. The year before, it was 41%, which was translated to 292 and the year before that it was 47.5%, which also translated to a scale score of 292. So the actual performance in 2014 for a score of Basic was quite a bit lower than it was two years before Louisiana moved to testing more Common Core math. A similar lowering of the underlying performance was engineered for the level of Mastery.

Also the raw cut score for 8^th grade math was lowered from 55% to only 40% over a three-year period. The cut score for 3^rd grade ELA was lowered from 54.6% to 48% over the three-year period. 4^th grade ELA was lowered from 53.8% to 44.6% in three years. Third grade math however was increased slightly from 53% to 55.5%.  All of these cut scores were adjusted by the contracted testing company, Data Recognition Corporation, with approval of certain testing specialists at the Louisiana Department of Education. This process is called “leveling” and is supposed to make the necessary adjustments in scoring from one test form to another. As far as I can tell, last year, the testing specialists made these decisions without consulting teachers in the field.

I was informed by one of my readers that in the past, small teams of education practitioners were asked to give their blessing to the recommendations of the testing experts. One amusing story related to me was that one year, the ELA practitioners found out that the math practitioners had changed the cut score more than the experts recommended so they demanded that their ELA scores be adjusted in a similar way.  This process does not look very scientific to me. It looks more like making sausage, a process the consumers are not normally allowed to see. Maybe that’s why the experts decided to skip that part of the process in 2014. More likely I think this was done because John White had promised the local superintendents that he would not allow the conversion to the more rigorous Common Core standards to denigrate our students or educators. Hence the lowering of the cut scores to make it appear that our students were doing just as well on the harder tests.

But my question is, how can we justify a passing score of only 38% or 40% on the new tests? I believe such low cut scores basically make the tests invalid because, students can come close to a passing score by just making random choices of most of the multiple choice answers. They don’t have to know very much of the supposedly more rigorous new content to make a passing grade.

How much will the cut scores on the new PARCC-like test be lowered this year? Our State Department of Education does not intend for us to know these internal secrets of the test makers and test graders. I guess we just can’t handle the truth!  Teachers and parents are supposed to accept the decisions these non-educators are making for our children and our schools. I think it is amazing that the Accountability Commission which is being asked to make it harder for schools to achieve certain grade ratings, are not informed about how these state test scores are manipulated. The Accountability Commission has been used for years to rubberstamp the decisions of the LDOE and to make it seem like the public and the real educators approve.  It was done 15 years ago when the state decided to mandate that almost all students complete the Core 4 curriculum to the determent of all our Vo-Tech programs in our high schools. Now John White is belatedly trying to bring back Vo-Tech with the Jump Start program.

As an experienced educator, I have to tell you that I don’t have any confidence in the people who are designing the tests and making the cut score decisions on these tests. Decisions on tests that have such a huge impact on our public schools! As a state we have taken the authority for important education decisions away from our teachers and handed it to testing technicians, many who have never set foot in a classroom as a teacher. This is another staggering insult to our teaching profession and it assumes that our parents are fools whose opinions about public education can be manipulated by these non-educators who are now making education policy through the construction and grading of these state tests.