|
|---|
Issues |
|
The NAEP Comparadox by Michael T. Martin The "Daily Howler" is a website on the Internet that regularly excoriates the media for their distortions and misinformation. It is operated by Washington, D.C., political satirist Bob Somerby. His website biography reads "After graduating from Harvard in 1969, Somerby came to Baltimore as a fifth grade teacher in the Baltimore City Public Schools. His first articles in The Sun, in 1978, dealt with issues of educational testing." Since he is a former educator, it is understandable that he would look at the media's misreporting of the National Assessment of Educational Progress (NAEP) results. As part of a series skewering the misreporting of NAEP results, he impugned the New York Times' coverage of the NAEP Reading scores. Somerby primarily focused on the misreporting of the gap between high and low achievement scores. But he noted, in passing, some numbers which overlooked an interesting example of Martin's Comparadox that bears explanation. Somerby quoted the NY Times reporter Kate Zernike. "From 1992 to 2000," Zernike wrote, "the average reading scores for fourth graders on the National Assessment of Educational Progress, known as the nation's report card, remained flat." And later, "the average score [of all tested students] in 2000 was 217, the same as in 1992, …." And still later, "According to a chart which Zernike provides, scores also went up by 3 points at the 75th percentile (from 242 to 245). At the 50th percentile, scores went from 219 to 221." Bingo! Martin's Comparadox: the means are the same but the medians are different. The "50th percentile," of course, is the median. So what both the press and Somerby missed entirely was that in the ten years where the NAEP and the press claimed scores were "flat," they had actually increased considerably! Remembering the empirical rule for finding the mode of a skew normal distribution: the mode is found three times as far from the mean as the median. Thus, if the means were the same in 1992 and 2000, but the median had shifted upward by two points in 2000, then mathematically the mode had shifted upward by six points. In 1992, a mean of 217 and a median of 219 would indicate that the mode was located at 223 (219-217= 2; 2x3=6 + 217=223). In 2000, a mean of 217 and a median of 221 would indicate that the mode was located at 229 (221-217=4; 4x3=12 + 217= 229). The mode is where most students scored on the test. Thus the NAEP data actually indicated that most students saw a six point (2.7%) increase in reading scores from 1992 to 2000! Of course, Somerby had no way of knowing about Martin's Comparadox. Somerby's main point was that the scores reported by the NAEP are based on samples that should normally exhibit variation from year to year in sub-scores such as the minor changes in scores at the top and bottom of the distribution. His scorn was therefore focused on the extreme conclusions proffered by the reporters from these minor changes. In particular he noted that there was no evidence that the decline in scores at the low end had anything at all to do with the functioning of the schools. The statement that best sums up his conclusions is what he called his "Final Point:" "ANYONE WHO FOLLOWS EDUCATION REPORTING will notice one thing - our press corps simply loves to report how dumb our schoolchildren are. But the comedy almost always comes from the incompetence of the reporters themselves."
|