The Art of Fair Testing

  • Share
  • Read Later
One of the most difficult challenges I faced as a teacher was creating good tests and scoring them fairly. It's easy to devise thought-provoking questions; the hard part is determining exactly what and how much students need to get right in order to get an A, or an F, or anything in between. This requires making value judgments about which skills or pieces of knowledge are most important. If you get the balance wrong, you don't just penalize (or reward) the wrong students, you make all your students less willing to learn the next time around.

It's a tortuous and subjective process—and it's precisely what federal legislators now need to go through, on the grandest of all possible scales, as they meet to hammer out the differences between the education bills recently passed by the House and Senate. While the Bush administration is already congratulating itself for pushing through a "monumental achievement with bipartisan support," the real work has yet to be done. All that's guaranteed right now is that every child in America will be tested every year in grades 3 through 8 in math and reading. But such tests will mean nothing (except big profits for test makers and test review companies) unless they are good tests that are scored fairly.

Test Quality

Since federal funding is to be directly linked to how schools perform on the yearly exams, states and districts will have an incentive to dumb down their tests. The education bill passed by the Senate solves this problem by requiring all states to also administer the National Assessment of Educational Progress (NAEP). If the schools in one state show dramatic progress on the yearly state exams but not on NAEP, that would indicate to the feds that the state tests are too easy. In my government class, at the end of the year all students took the College Board's Advanced Placement exam in U.S. Government. If all my "A" students bombed the AP test, that would show that my own tests were too easy—and that my students weren't really learning anything. The House, worried about establishing a national curriculum, allows states to opt out of NAEP and use different exams for comparison. But with no national benchmark, it will be much harder for the feds to monitor the quality of state exams—and to sanction states that make their tests too easy.

Test Scoring

What's novel about the Bush education plan is that for the first time the federal government will sanction schools that fail to show progress. But what is progress, and how much is enough? Under the House bill, states would have 12 years to raise all students to proficiency in reading and math. The scores of blacks, latinos and low-income students would be separately tallied and would also have to rise—but states could determine on their own what "adequate yearly progress" is. Yet even with this flexibility many are worried that too many of the nation's schools would be identified as failures. Says Sandy Kress, Bush's top education advisor: under the House bill "virtually a hundred percent of the schools will be labeled low performing."

The Senate took the opposite approach. The test scores of each subgroup of students (blacks, latinos, low-income kids) are required to improve just 1% per year—hardly an attempt to leave no child behind. And states are allowed to average the performances of the subgroups—and therefore mask weak minority test scores beneath the high scores of white or wealthy students. This lowering of the bar was partly due to pressure from the 22 Republican governors up for reelection next year who don't want the federal government labeling their schools as failures.

What gets decided in this conference committee will have a dramatic effect on every public school classroom in America. We know there's going to be a lot more testing. But will these tests be good ones, and will they be scored in such a way that good schools are recognized and lousy schools that don't educate their poor and minority students are forced to change? That would be a true monumental achievement.