Friday, August 12, 2016

The Spring 2016 Scores the LDOE Didn't Want You to See

Our State Department of Education has released the scale scores of all Louisiana students for the Spring 2016 testing. The tests in English language arts (ELA) and math were based on the  same Common Core standards as they were last year. The major differences are that the tests were shortened in length with fewer total points possible, and as a result of a legislative mandate, fewer than half of the questions were allowed to come from the PARCC consortium.

Note to readers: You may click on this link to view the full 44 pages of conversion tables for converting raw to scale scores for the 2016 LEAP tests in math and English language arts for grades 3 - 8. These tables were obtained by public records requests and are now part of my Google Drive files. 

Science and social studies tests for grades 3-8 have scale scores ranging from 100 to 500. Click here to view the conversion tables for Science. Social studies conversions are not available because they are still in development.

All End-of-course tests for high school have scale scores ranging from 600 to 800. Click here to see the conversion tables for the EOC tests given in May 2016.

The ELA and math tests form a major portion of the rating system for our schools and eventually are supposed to be used in evaluating teachers. The problem is that these scores are reported as scale scores, which we found out last year, tell us almost nothing about how much of the test material our students got right. Teachers and parents are accustomed to reviewing student test performance as raw scores which tell us how many questions students got correct on a test. On most tests, teachers and parents get to see what percentage of the possible answers each student got right. BESE has a policy on grading for all public schools that sets a score of 67% as the minimum passing score on tests. But that turns out to be far from being the case with standardized testing.

Unfortunately the LDOE and their testing companies prefer to obscure vital test results by reporting only what in the industry is known as "scale scores". Scale scores often have a skewed relationship to the actual percentage of questions a student got right. For example, if a student answers absolutely zero questions right, the present system for grading the Common Core related tests assigns the student a score of 650 out of a possible total of 850 points. Also, even though BESE has adopted a scale cut score of 725 as the minimum passing score on all of the Common Core related tests, that score could represent as little as 27.3% or as much as 39% correct answers on the actual test. To most parents and even to some teachers, this is a pretty confusing way to report student test results.

That's why I submitted a public records request to Superintendent John White and to the LDOE just as I did last year, for the raw to scale score conversion tables. I plan to make these available as a Google Doc. to anyone who wants to be able to convert any scale score to a raw score on each of the tests given this Spring. Also, I believe the LDOE will soon make these conversion tables available on their web site since the cat is out of the bag. For now, with this blog post I am reporting the total possible points on each Common Core related test for grades 3 - 8, the minimum or "cut score" for a rating of basic and mastery, and the actual percentage of correct answers represented by these cut scores.

Each year, the testing companies in consultation with their clients (the LDOE) usually change the test forms to reduce the possibility of cheating. (The tests use different questions assessing the same material from one year to the next) In most cases there is some variation in the difficulty of the various test forms, so the testing company applies an equating adjustment to attempt to retain the same level of difficulty in determining test results. This is the primary legitimate reason for reporting test results as scale scores instead of as raw scores each year. (There are other reasons in my opinion that are not so legitimate. . . . such as manipulating the test results to show either false improvement or declines in performance)  This was found to be the case several years ago in the state of New York where the whole whole state average of testing results was falsely inflated.

The table below represents the conversion from scale scores of 725 for basic and 750 for mastery to raw scores for the 2016 Spring testing in ELA and math. The results of scale to raw conversion vary from test to test, but overall, the average cut scores for the ratings of basic and mastery are very close to the percentage correct used in 2015 scale scores for math and English Language Arts (ELA). The tests covered the same standards as in 2015, and the cut scores for the two years represent close to the same percentage of correct answers. The average cut percentages were set slightly higher in 2016 than in 2015.

The LDOE reports that approximately 67% of our students achieved a rating of "basic" on these tests and 37% achieved a rating of mastery in 2016. Just as my fellow analyst Herb Bassett predicted, the percentage of students achieving basic and mastery has increased by a small amount over the first year of testing. (2% and 5% respectively) This seems to be a common trend after the introduction of a new type of test. I will encourage Herb to write a post describing such trends.

For now though, I encourage my readers to put yourself in the shoes of our young students taking these tests, and try to imagine how it feels to be able to answer only about 32% of the questions correctly. I am not convinced that these tests are appropriate or valid for our students.


7 comments:

Johnny Fatheree said...

These tests are a joke. If you want to improve test scores, then get our kids reading and reading well. Poverty is holding Louisiana back and not our teachers. Take a hard look at the free lunch program, and you can easily see the correlation of test scores to poverty ratio. Every state in the nation is dealing with the same problem. Children from poverty homes are 40 million words behind by the time they start school. They are severely deficient in their vocabulary which is the basis for reading and learning. Why don't we identify these children when they start to school instead of waiting until the third grade to test them. TOO LATE! Same is true for children with learning disabilities. Tia is a travesty that must be corrected. Find the children who need help and put the resources where they are needed.

Kimberly Domangue said...

VERY HELPFUL and APPRECIATED, Mr. Deshotels!

Easy Shiksha said...

Search for best educational blog on the internet.

Pastiche said...

The raw scores are the percentage of questions that the student answered correctly? So on the Alg I EOC if you answer 36% of the questions correctly you score Mastery? Am I understanding this right?

Michael Deshotels said...

Pastiche, you are understanding this right. That's why I have made the raw scores available. Since the EOC tests are 100% multiple choice, a first grader, with a little luck, could pass the Alg I test by just guessing!

Michael Deshotels said...

Pastiche, let me clarify. The passing raw score on the Alg. 1 test is 17 out of 50. So a student can pass by only getting 34% of the questions right.

DiLina said...

Accurate how many days until Spring 2017?. Checkout exactly how many days until spring 2016 according to your country and time zone.