Wednesday, July 25, 2018

How Valid are the Louisiana LEAP tests?


School reform in Louisiana was supposed to eliminate social promotion and the awarding of worthless high school diplomas. 
Superintendent John White has staked his entire career as an education reformer on improving state standardized test scores of Louisiana students. To reformers like White, test scores are everything. In their philosophy of education, you can’t trust teachers to tell us and parents whether students are learning and progressing and are going to be ready for college or careers when they graduate. Reformers believe that Louisiana needs an objective way of finding out if our students are getting diplomas that indicate that they are ready to compete with students from other countries for the best jobs in the world economy. 

John White was selected by former Governor Jindal to be our State Superintendent at the beginning of 2012 with the mission of implementing new laws that would evaluate, reward and fire teachers based on student test scores and to implement the replacement of many public schools with independent charter schools. The charter schools would live and die based on the attainment of high student test scores.

From the very beginning of our Louisiana education reforms, the reformers announced that they wanted to eliminate diploma mills that turned out graduates that had no real education and were not going to be fit for the job market or college. Corporate education reform was no longer going to allow diplomas to be awarded to functionally illiterate young people. Reformers believed that it was time to eliminate social promotion, whereby children were automatically promoted to the next grade even though they had not achieved satisfactory results on their math and ELA courses. The gate keepers would be cut scores on state tests that would indicate proficiency or failure.

The primary reform to ensure well educated graduates was to require state standardized testing to determine promotion and graduation. Key language of the Louisiana accountability laws required that students could not be promoted from grade 4 to 5 and from 8 to 9 if they failed to demonstrate proficiency on math and ELA state tests. Then students could not graduate unless they passed certain critical subject matter tests in critical courses such as Algebra, Geometry, Biology, US History, and both English I and English II or III. 

John White, a TFA corps member with experience in Chicago and New York was selected as state superintendent to implement test based accountability. 
John White was hired as State Superintendent in January of 2012. Even though White had no formal training in education, his experience as a manager in the New York City school system where he had worked for Mayor Michael Bloomberg installing school reforms was considered the best qualification to reform Louisiana schools. Part of White's job was to launch Charter schools to replace public schools. Charter schools are privately run schools that are relieved of most state regulations in exchange for test-based accountability. A charter school could be run any way the manager wanted as long as they produced good student test scores. If the student test scores on state tests were not satisfactory, the charter school would lose its charter and could be taken over by a new charter operator (not by its school board because school boards had been considered incapable of turning around a failing school)

But everything in the White administration revolves around increasing student test scores. The school rating system installed by White and his TFA cronies applies maximum pressure on school administrators and teachers to do almost nothing but attempt to raise student test scores.

Louisiana state law requires that our state tests be compatible with the National NAEP test so that our student performance can be compared to other states.
The education reform laws also required that the new Louisiana standardized state tests must be compatible to nationally recognized tests including the National Assessment of Educational Progress (NAEP). In other words, a student rating of proficient on the state tests should be the same as proficient on the NAEP test.  It was decided that a rating of Mastery on the state tests should be equivalent to a rating of Proficient on the NAEP.

So how reliable are our state LEAP and End-of-Course tests, compared to the NAEP? Does the progress of our students from year to year on LEAP match the progress measured by NAEP? Are we finally moving our students to proficiency and awarding them diplomas that future employers can trust are indications of real academic skills? Since the legislature had decided at the beginning of the reforms that we couldn’t trust the teachers to tell us whether a student was worthy of getting a diploma, did they also insist on a check-up system to see if we could trust the State Department of Education and their standardized tests to certify that a student was worthy of a diploma? 

Oops, it looks like the legislature forgot to set up an independent check on our Department of Education to see if they were faithfully holding up their end of the bargain to end social promotion and grant real diplomas. There is no one officially checking to see if the LDOE tests are really measuring proficiency  as comparable to the NAEP tests. But there is a way of checking the validity of our state tests compared to NAEP. There just is no law requiring anyone to make the comparison.  So here is my effort to provide a legitimate comparison of the two testing systems.

Based on the LEAP testing conducted in the Spring of 2011, the last year of testing before White, the following table gives the percentage of students achieving Mastery or above ratings. Those are the ratings that are supposed to be equivalent to the rating of Proficient on the NAEP.


4thgrade ELA: 28% of our students statewide achieved Mastery or above on LEAP
4thgrade math: 28% of Louisiana students achieved Mastery or above on LAEP
8thgrade ELA: 25% of Louisiana students achieved Mastery or above on LEAP
8thgrade math: 9% of Louisiana students achieved Mastery or above on LEAP

How do the above results compare to our student performance the same year on the National NAEP test? Here are the percentage of LA students achieving at least Proficient on NAEP in 2011.

4thgrade Reading: 23% of LA students achieved Proficient on NAEP 
4thgrade math: 26% of LA students achieved Proficient on NAEP
8thgrade Reading: 22% of LA students achieved Proficient on NAEP
8thgrade math: 22% of LA students achieved Proficient on NAEP

So in 2011, the Louisiana results on LEAP and NAEP are pretty close In three out of four categories. There was however a large difference in the results for 8thgrade math. The LEAP test indicated a much lower percentage of students achieving the Mastery rating than the % reaching the Proficient rating on NAEP. So probably the state LEAP math  test cut score was set too high compared to the NAEP 8thgrade math test..

In just 7 years the LEAP tests became highly inflated compared to the National tests. The average inflation amounted to 59%.
Now lets look at the 2018 results on LEAP for the percentage of students achieving proficiency compared with the 2017 results on NAEP. (2017 is the most recent NAEP test given)

4thgrade ELA/reading % earning Proficient: LEAP = 44%, NAEP = 26%
4THgrade math % earning Proficient: LEAP = 38%, NAEP =27%
8thgrade ELA/reading % earning Proficient: LEAP = 45%, NAEP = 25%
8th grade math % earning Proficient: LEAP = 28%, NAEP = 19%

Do you see the dramatic inflation of our LEAP scores from 2011 to 2018?  The LEAP results are telling us that Louisiana students have made dramatic improvement in all 4 categories since 2011. But the NAEP results tell us that our students made very little progress from 2011 to 2017. In fact for 8thgrade math, the NAEP tells us that our students are doing worse now than they were doing in 2011. Yet the LEAP tells us that our 8thgrade students have improved from only 9% achieving mastery in 2011 to 28% achieving mastery in 2018. The average proficiency rate inflation for the 4 key state tests compared to the National tests in 2018 was 59%. This is a direct violation of state law and basically invalidates all of the 4thand 8thgrade LEAP tests. So why are our students being required to take these invalid tests each year?

So not only are our LEAP tests incompatible with national NAEP test as was mandated by state law, but in at least one critical area, the LEAP results indicate improvement where there was actually a decline in performance as measured by the national test.

Students no longer need to pass their state tests to be promoted to the next grade.
But that’s not all. Superintendent White has recently convinced BESE to drop the requirement that students in 4thand 8thgrades pass their math and ELA tests in order to be promoted. Public records show that in 2016 even though 21% of our students statewide failed both their ELA and math tests, only 2.5% or one tenth of the dual failures were actually retained in their current grade. That was in violation of law in 2016 but now that BESE,  based on John White’s recommendation, has dropped all requirements for promotion, it will be OK to allow full social promotion of almost all students who fail their state tests.  According to the new BESE policy, all the schools have to do for a student failing every standardized test is to promise to give him help to catch up.

According to state testing, John White is a big success. Louisiana’s public school students are improving dramatically, and are well on their way to achieving Mastery or Proficiency by 2025. But according to national testing, achievement scores have barely improved in three areas and have dropped in 8thgrade math. Louisiana is near the bottom of the NAEP rankings. Most independent agencies now rate Louisiana as the lowest performer out of all the states in the measures of school performance.

Common core standards may be not be teachable for at least half of our students.
My opinion, which I can’t prove, is that the lack of progress in student proficiency is really a result of implementing the common core standards which are basically unteachable for at least half of our students.

Louisiana is allowing the same abuses that have resulted in charges of fraud in the reported graduation rate of the Washington D.C. school system. 
Now, not only are Louisiana students being promoted who demonstrated unsatisfactory test performance, but schools are allowed to waive the attendance requirement for promotion and graduation. Now students in high school who missed much more than the allowed absences and who failed their state tests,  often still get a diploma by just taking a few hours of credit recovery courses. For example, students can now pass their Algebra I EOC test by scoring only 23.5% correct answers.  This is exactly the same situation that caused the graduation rate in the Washington DC schools to be declared fraudulent. But here in Louisiana, no one in an official position is blowing the whistle. 

So if you think there were illiterate students getting diplomas in the old days, that’s nothing compared to the rampant awarding of diplomas to anyone with a pulse today. I certainly do not believe that the state tests are valid enough to be used as the promotion standard. I have much more faith in the judgement of teachers. Unfortunately the law that is supposed to prevent the pressuring of teachers on promotion decisions is also being ignored in the push to boost the graduation rate at all costs.

A simple comparison of LEAP to NAEP shows that our state tests are just not reliable as a measure of proficiency and eligibility for graduation.


Raw Score to Scale Score Conversion Tables

The following are links to the conversion tables for LEAP tests given to Louisiana students in the Spring of 2018. This reveals how the LDOE and their testing company produce their scale scores from raw scores. By looking at each table a teacher or parent can find the raw score equivalent to a scale score on a particular test. Even though the cut scores for Basic, Mastery and Advanced are the same for all tests,   the raw scores equivalent to these cut scores can vary greatly.

Here is the link to the raw score to scale score conversion tables for converting from raw scores to scale scores for the Spring 2018 LEAP tests for grades 3 through 8. This link is to my Google Docs. The tests included are ELA, math and social studies. There are a total of 37 pages of LEAP conversion tables.

Here is the link to Spring 2018 high school End of Course tests for English I, Algebra I, Geometry, English II, and American History. There are a total of 11 pages of EOC conversion tables.

My next post on this blog (above) will include an analysis of the validity of the various LEAP tests in comparison to the nationally administrated NAEP tests. NAEP stands for National Assessment of Educational Progress.