Tuesday, June 30, 2015

Standardized Test Score Setting is Looking More and More Like Sausage Making

Forget About Common Core; Forget About the New Revised Louisiana Standards Now Being Drafted; It’s the State Tests and Their Cut Scores That Are the Real Standards!

Most professional educators working in our public schools have figured out by now that the standards and the curriculum in our schools today are one thing and one thing only: the state tests. The annual LEAP and iLEAP tests are the only things that determine what the basic skills teachers will be forced to teach. In addition, the cut scores  on these tests that are set by non-teacher testing specialists will determine what percentage of students will score Basic, Mastery or Advanced. Also the cut scores will determine what grades will be ultimately assigned to each of our public schools.

The newly appointed standards review committees, the Accountability Commission and then BESE think that they are making real decisions about the standards, but their decisions are greatly overshadowed by the test designers and the cut score deciders. Superintendent White knows he can adjust the standards in any way he wants from year to year by either changing the ratio of difficult to easy questions on state tests or by changing the cut scores.

This Spring, our students in grades 3 through 8 took their usual LEAP and iLEAP standardized tests in science and social studies accompanied by Common Core aligned tests in math and English language arts based on the new PARCC test. Schools and parents were told that grading would take much longer this year and that the results would not be released until sometime in the Fall. The test results will contain no diagnostic results for individual students. They are designed mostly to measure the effectiveness on the schools and teachers in teaching the new Common Core standards. It is believed by some reformers that just requiring the new Common Core standards and testing students each year will somehow drive our schools to a higher level of excellence that will prepare our students to be much more competitive in the world job market. You know, just the way that No Child Left Behind got every student to the level of proficiency over a ten year period.

Over the previous three years, state tests had already been incorporating increasingly more Common Core aligned test questions in preparation for the 100%  PARCC test given this Spring. Upon release of the Spring 2014 test results, the LDOE announced that even though the tests in math and ELA were more challenging, our students performed just as well as the year before according to the number of students achieving a level of Basic on the tests. The press release also stated that the percentage of students achieving Mastery had increased slightly. What they neglected to tell us was revealed as a result of a public records request and a public records lawsuit. That is the fact that the cut scores for both Basic and Mastery had been lowered significantly on 3 out of 4 tests to basically insure that the number of students scoring at those levels would not drop. The students actually performed significantly lower on the new tests but the grading scale was lowered to make it look like they had done just fine.

The official score range for achievement of Basic on the 7th grade math test for 2014  was from 292 to 375 out of a total possible 500 points. This was the same standard scale score range that was used for the 2013 7th grade math test. But what the state didn’t tell us was that the percentage of correct answers that was translated to a scale score of 292 was only 38%. That 38% is what is called the raw cut score. The year before, it was 41%, which was translated to 292 and the year before that it was 47.5%, which also translated to a scale score of 292. So the actual performance in 2014 for a score of Basic was quite a bit lower than it was two years before Louisiana moved to testing more Common Core math. A similar lowering of the underlying performance was engineered for the level of Mastery.

Also the raw cut score for 8th grade math was lowered from 55% to only 40% over a three-year period. The cut score for 3rd grade ELA was lowered from 54.6% to 48% over the three-year period. 4th grade ELA was lowered from 53.8% to 44.6% in three years. Third grade math however was increased slightly from 53% to 55.5%.  All of these cut scores were adjusted by the contracted testing company, Data Recognition Corporation, with approval of certain testing specialists at the Louisiana Department of Education. This process is called “leveling” and is supposed to make the necessary adjustments in scoring from one test form to another. As far as I can tell, last year, the testing specialists made these decisions without consulting teachers in the field.

I was informed by one of my readers that in the past, small teams of education practitioners were asked to give their blessing to the recommendations of the testing experts. One amusing story related to me was that one year, the ELA practitioners found out that the math practitioners had changed the cut score more than the experts recommended so they demanded that their ELA scores be adjusted in a similar way.  This process does not look very scientific to me. It looks more like making sausage, a process the consumers are not normally allowed to see. Maybe that’s why the experts decided to skip that part of the process in 2014. More likely I think this was done because John White had promised the local superintendents that he would not allow the conversion to the more rigorous Common Core standards to denigrate our students or educators. Hence the lowering of the cut scores to make it appear that our students were doing just as well on the harder tests.

But my question is, how can we justify a passing score of only 38% or 40% on the new tests? I believe such low cut scores basically make the tests invalid because, students can come close to a passing score by just making random choices of most of the multiple choice answers. They don’t have to know very much of the supposedly more rigorous new content to make a passing grade.

How much will the cut scores on the new PARCC-like test be lowered this year? Our State Department of Education does not intend for us to know these internal secrets of the test makers and test graders. I guess we just can’t handle the truth!  Teachers and parents are supposed to accept the decisions these non-educators are making for our children and our schools. I think it is amazing that the Accountability Commission which is being asked to make it harder for schools to achieve certain grade ratings, are not informed about how these state test scores are manipulated. The Accountability Commission has been used for years to rubberstamp the decisions of the LDOE and to make it seem like the public and the real educators approve.  It was done 15 years ago when the state decided to mandate that almost all students complete the Core 4 curriculum to the determent of all our Vo-Tech programs in our high schools. Now John White is belatedly trying to bring back Vo-Tech with the Jump Start program.

As an experienced educator, I have to tell you that I don’t have any confidence in the people who are designing the tests and making the cut score decisions on these tests. Decisions on tests that have such a huge impact on our public schools! As a state we have taken the authority for important education decisions away from our teachers and handed it to testing technicians, many who have never set foot in a classroom as a teacher. This is another staggering insult to our teaching profession and it assumes that our parents are fools whose opinions about public education can be manipulated by these non-educators who are now making education policy through the construction and grading of these state tests.

Saturday, June 27, 2015

LDOE Wants to Punish Our Way to High Expectations

The inexperienced "one trick ponies" that run our Louisiana Department of Education still think that they can set learning goals for our Louisiana students and order the real educators to push students to perform at whatever level they set. They believe that if they set "high expectations" and then reward or punish schools based on the results, this system will somehow automatically cause students to achieve more. But that tactic has already been tried nation-wide and it has failed!

The latest effort by our LDOE to boost student achievement is to make it harder for a school to earn an "A" rating. The requirements for school ratings for "D" or "C" would presumedly also be raised proportionately. The idea is to link the very positive concept of "high student expectations" with punitive measures against teachers and schools that serve the most at-risk students.  The simple-minded assumption of our amateur education leaders is that when some students fail to reach the arbitrary standards set by state administrators, then it must be the fault of the real educators who apparently did not really have high enough expectations. This is a perfect example of taking a promising concept in education (high expectations) to a punitive and ultimately unproductive extreme.

John White and his staff at the LDOE have apparently forgotten that the only reason state test results have gone up in recent years in Louisiana is because the LDOE has lowered the raw cut scores to make it possible to score basic and even mastery partially by guessing or randomly picking answers to the questions. How much value can we put on school grades that depend on bogus and inflated test results?

A seventh grader in Louisiana in 2014 could score a rating of "basic" in math by just getting 38% of the answers right on the math iLEAP test. The minimum percentage of correct answers for that rating used to be 47% which was already extremely low for a passing grade. A student in the 8th grade could pass his/her math test with only 41% of the answers right. What teacher could get away with giving a passing grade to students who scored 38% or 41% on the final test?

Here is how it works: John White is now in the second phase of his tenure as Louisiana State Superintendent. This is the stage where he is supposed to demonstrate that all the recent reforms such as stripping teachers of due process rights and giving grades to schools mostly based on the poverty level of students and letting charter and voucher schools run amok with our tax dollars has actually resulted in a better education for our students.

The pressure on principals and teachers to do almost nothing but teach the state tests has gotten overwhelming. This has made a mockery of our educational system. It has killed the joy of teaching and learning and has driven many of our best teachers to early retirement. Yet White and his TFA minions still think that they can milk higher test scores out of the teachers and students. As you are reading this post, the staff of the testing company and certain LDOE staff who have never set foot in a K-12 classroom are deciding on the new raw cut scores for basic and mastery on the PARCC-like test given this Spring. These discussions are top secret and the public is never told what the real cut scores are from year to year. The only data released to the public are the scale scores which are  usually reported to be the same as the previous year even though the underlying raw scores have been drastically changed. The setting of these cut scores is what really determines the grades that will be awarded to schools.

Even with the fight over the Common Core and the drive to have high standards and high expectations, the LDOE and BESE have quietly removed all state standards for student promotions at the 4th and 8th grades. Those grades used to have minimum standards for promotion to the next grade.  But that part of high expectations and accountability has been suspended by White and BESE. That's why we have students entering high school that are 3 or 4 years below grade level in actual performance. Somehow the teachers in high school are supposed to take these students who may never have cracked a book in their lives and infuse them with "high expectations".  But high expectations only work if the learner adopts these expectations as part of his/her education goals. Many of our most at risk students are so overwhelmed by their day-to-day life challenges such as lack of proper nutrition and health care, lack of proper housing, and dysfunctional or absentee parenting that they have a tough time internalizing high academic expectations. Does it make sense to punish the dedicated educators who serve such students for factors over which they have no control?

Fifteen years ago, even before No Child Left Behind, Race to the Top, and Common Core, Louisiana announced that we would guarantee the employers of Louisiana that the high school diploma would start meaning something. We would push more and more students to take the college prep Core 4 curriculum so that almost all of our students would be prepared for college. Now after 15 years of "high expectations" our kids are doing no better in college completion than they did in the year 2000. Our college presidents are shocked, shocked I tell you, to find that our students are no better prepared than they were before we made the high school diploma stand for something.

But the reason for this is simple. When you tell the teachers who used to teach real college prep courses to the 30 percent of students who actually wanted to go to college that now they will have to teach those courses to all the students and to be sure they don't let most of them fail, we end up with watered down college prep courses. So now all students get short changed and none get real college prep in many schools. There are exceptions! The top ten school systems in the state led by the Zachery school system operate in an environment that includes a partnership with parents and community where high expectations and college prep for a high percentage of students works. But even the Zachary Community School System has a solid career prep program because the professional educators there know that many students need challenging career alternatives to college. Statewide, too many of our high schools have just watered down their college prep courses because they cannot afford to fail students who do not meet the college prep standards. Now, for students who make no effort at all to get educated, we provide easy courses, easy state tests, and we even give them bogus credit recovery courses if they still can't pass.

Yes, many more students could perform much better in school, but there will always be at least half of the students that are not 4 year college material. All students have talents and interests that schools should be identifying and developing. We mostly don't do that because the federal and state mandates tell us (even though we know it's ridiculous) that we must prepare all students for 4 year colleges. No matter what they say now, the original intent of the developers of the Common Core Standards was to somehow prepare all students for college, without regard to their abilities and interests that could have steered them to other more productive avenues for success in life. The Common Core developers believed that they could just require teachers to teach their own favorite academic priorities and even their quirky teaching methods for math and English to all our students.

If you want proof that high expectations without student and parent accountability does not work, just look at the nationwide No Child Left Behind record. That high expectations law mandated that by the end of the 2013-2014 school year every student would be proficient in English and Math or the teachers and the school would have hell to pay. Guess what? Toward the end of that mandated time period, almost no schools in America were meeting their mandated Adequate Yearly Progress goals.

The goal of college prep for all was one of the primary goals of Louisiana's highly touted Recovery School District. The new charter operators believed that the concept of "high expectations" was so powerful that even students from the most at-risk population could be indoctrinated to aspire to and prepare for an Ivy League education.  All 7,000 of the experienced teachers and other staff in the takeover schools were fired because it was assumed that they would not truly believe in and accept the high expectations challenge. In their place the new charter operators brought in young non-education graduates from Ivy League schools that they thought could be easily molded into proponents of high expectations. After 10 years we have the results of this effort. My post of May 21, compares the performance of the New Orleans RSD with the rest of the state public schools and demonstrate that the takeover charters are performing in the bottom quartile just as they were doing before the state takeover. In addition, a Cowen Institute report found that even when comparing the at-risk (high poverty) students in the RSD with similar students in the rest of the state, those students still performed slightly below the same type of students taught by traditional schools.  So was it really a good idea to fire the 7,000 experienced teachers?

Soon after the establishment of the New Orleans RSD, the LDOE also took over 7 schools in EBR, two middle schools in Caddo, a high school in Pointe Coupee, and a middle school in St. Helena. The Department tried to apply the same formula there that they thought was working in New Orleans. They failed miserably. The takeover schools in Pointe Coupee, and St Helena have now been returned to their original school boards after test scores and even attendance fell to the lowest levels in the state. All the original charters in Baton Rouge except for one have been terminated for lack of improvement, and new managers are being appointed. The charters in Caddo are still failing. In addition,  the School for the Deaf and the School for the Visually Impaired which have always been run by the "high expectations" State Department of Education are still rated "F". The Advocate reported in this story that the voucher schools across the state have produced dismal results for their choice students.

Yet after all that failure of No Child Left Behind and the Race to the Top programs, recently U. S. Secretary of Education, Arne Duncan proposed that students with disabilities of various kinds would learn more if we subjected them to the same tests and high expectations that we require of regular students. In other words, Duncan believes we can even cure learning disabilities with high expectations. Just what is the scientific basis of this theory? Certainly human beings are capable of amazing feats of learning. But there is one missing element in the high expectations theory as it is being applied in Louisiana. That is the fact that humans do not learn anything unless they are motivated to do so. You can't force your higher expectations on another person who has other priorities such as day to day survival. That's where the theory breaks down.

So now Jessica Baghian, a two year TFA teacher from a low performing school in New Orleans who has been elevated to a position of authority over experienced teachers and principals has addressed the Louisiana Accountability Commission and announced to them that "high expectations" and more punishment of our educators is the solution to the problems of Louisiana schools. 

Sunday, June 7, 2015

The Louisiana Common Core Standards Compromise

Both houses of the Legislature have now approved a combination of three bills that require BESE to review and revise the Louisiana state standards for English Language Arts and mathematics and submit them to the legislature for its review by March 4th 2016. Governor Jindal has indicated that he will sign the compromise bills into law.  If you click on each bill number, you may view the full text of each bill along with a summary interpretation of the changes to the law produced by each. The bills are: HB 373, HB 542 and SB 43.

The three bills taken together require BESE to hold open public meetings in each congressional district starting after July 1, 2015 that would theoretically allow public input into the revision of the ELA and math standards. During approximately the same time period, BESE and the LDOE must approve new contracts for the development and implementation of new state tests in ELA and math for the Spring of 2016. One of the new laws prohibits LDOE and BESE from contracting with the PARCC consortium for the development of the Spring 2016 tests. Also, these tests for grades 3 through 10 cannot include more than 49.9% of the PARCC consortium questions. There is no stipulation in the new law about where or how the rest of the questions on state tests for ELA and Math will be derived or formulated.

The revised standards for ELA and math must be submitted to the House and Senate Education Committees by March 4th 2016 and may be adopted or rejected in globo by the legislature. Also, the new governor may veto or approve the new standards but cannot make changes to them.

There is no mention in the legislation about how BESE will actually write the new standards or how members of the teaching profession or members of the public may participate in this writing process except by providing input into the public meetings that must be conducted in accordance with the open meetings law and the administrative procedures act.

It should be noted however, that BESE and the LDOE have already set into motion a process for the review and revision of state standards and testing to be completed during the 2015-16 school year. In that process there will be several committees involved in the review and writing process that include both professional educators and members of the public. It is my understanding that this process will continue even though the timelines may be shortened because of the deadlines in the new laws above. The June meeting of BESE will probably make adjustments in the revision procedure to comply with the new laws.

Now I will attempt to describe how this will all work out in practice. I believe that BESE and the LDOE will continue with their process of allowing input into the standards revisions by the committees that are scheduled to be approved by BESE at its June meeting. Certain subcommittees will probably be assigned the job of actually writing the proposed standards.  Even though the new legislation specifies that BESE, not the Department of Ed, will write the standards, based on my observations of past BESE committees, chances are these committees will be tightly controlled and guided by LDOE staff. Since there is so much work to do in such a short time, chances are the actual work will be done by just a few professionals with much guidance from the LDOE staff and then the full committees will be asked to approve or modify the result. The writing committees will probably look to some combination of the old Louisiana Grade Level Expectations (GLEs) and the new CCSS standards for actual wording of the new standards. BESE will probably end up with some hybrid comprised of those two systems.

One of the most significant complaints about the CCSS, is that much of the material is not age appropriate for our younger students. I hope the writing committees will tackle this issue and make the adjustments that our early childhood practitioners recommend. There are some serious objections by parents and educators about some of the unorthodox and confusing methods of working math problems and similar objections about the close reading methods and materials that were used in Common Core ELA standards. It remains to be seen if these problems will be corrected.

But unfortunately the elephant in the room for these education decision makers that has been ignored is the extreme over-emphasis on annual standardized testing and test prepping of our students. This test mania mandated by our legislature and BESE and the grading of schools based on test results is forcing our teachers to spend an inordinate amount of time and effort just preparing kids for tests. It is killing the joy of teaching and learning for teachers and students; it is pedagogically unsound; it violates many of the more natural and effective methods of passing on culture and knowledge to our young; and it does not prepare young people for the real world! The most educationally advanced countries in the world are not using this insane approach to education.  Not only is this test mania bad for children, but how can we attract the best and brightest young people to the teaching profession if most of their working day will be relegated to test teaching? How can we justify sacrificing the arts, social studies, geography, foreign languages, inquiry learning in science, technical education and even physical education (when we have the most obese and nutrition deprived student population in history)? What kind of a job market is there for a morbidly obese young adult whose only skill is taking standardized tests?

As an educator, I appreciate greatly the effort of the legislature to totally revise or even dispose of the Common Core Standards. These standards were poorly designed and are creating chaos all over the country.  But I am also very concerned about the lack of deliberative planning in this process and the continued mandate of standardized testing for next Spring when teachers have no idea what will be on the test!