Thursday, March 7, 2013

Yes, the standardized tests you hear being bad mouthed really are that terrible

 Below is a released item from the PAWS, Wyoming's state standardized test. Student progress is measured by scores on this test, and teachers' evaluations are based on students' results on this test. My school's AYP scores, determining whether or not we are a "failing" school is based on results from this test. 

The state's contract with Pearson runs about eight million dollars.

And, incidentally, the test is terrible.

Consider the following released items. This is from the Reading portion of the test and is meant to assess students' ability to utilize functional texts. Three out of the ten questions connected to this item don't meet the most basic criteria for quality instruction: that they measures only what they set out to measure.

1.       Which change in the enchilada recipe should you make if you are cooking for four people?
a.       Use 1 whole yellow onion
b.      Use ½ teaspoon chili powder
c.       Use 2 pounds of ground beef
d.      Use ¼ teaspoon of garlic salt

      2.   What should the check amount be if you are ordering three copies of the cookbook?
a.       $20
b.      $40
c.       $60
d.      $80
The first two of these items mix math skills in with the "select and apply" skills meant to be measured. Now, should seventh graders be able to multiply by three and divide by two? Absolutely. CAN all seventh graders multiply by three and divide by two? Not at all. So, what happens is that lack of basic math skills interferes with measuring basic reading skills.
Not only that, but even students who do have good math skills will miss these items because they introduce weird variables into the assessment of basic skills. So, students with good test-taking skills will be more successful than those without, regardless of reading ability.
The third item here is the most nefarious.
3.       Suppose you want to add the following step to the directions. “Preheat the oven to 350 degrees.” Where would be the most appropriate place in the directions to add this step?
a.       Before Step 1
b.      After Step 5
c.       In Step 6
d.      In Step 7

This question does not measure reading ability at all. It essentially determines which kids have relevant background knowledge and which ones don't. Kids who have experience cooking at home will do well, while students who don't, won't. Put simply, this question is class warfare.
Yes, there is some basic logic that could help determine the answer, but even if students were to apply knowledge of basic prefixes to work it out, they could still choose option b, and they would still be wrong.
What actually bothers me more than assessment scores that don't measure what they claim to measure or teacher evaluations based on assessment scores that don't measure what they claim to measure is the fact that this test mostly measure students' ability to take tests, and if I want them to do well, I have to spend time teaching them how to take tests. We have to talk about what to do when you get questions that cannot be answered by reading. In other words, I have to take time away from reading instruction in order to teach them how to take a test that will not measure my ability to provide reading instruction.
When we talk about this in class, the students get angry.
I don't blame them a bit.

1 comment:

  1. Seems like Wyoming could do more with the 8 million dollars by just putting that into library books or something. But hey, I'm just a mom.