Part 3—By law, it’s a song sung blue: America’s alleged journalists swear by the National Assessment of Educational Progress (the NAEP), the 42-year-old federal testing program.
Routinely, they refer to the NAEP as the “gold standard” of educational testing. And testing experts seem to agree that this high praise has been earned.
But how odd! The NAEP data spill with remarkable score gains. But the American public is simply never told.
Yesterday, we cited one such score gain. This morning, let’s add to it, briefly.
Yesterday, we discussed the score gain achieved by 13-year-old black students in math. From 1978 through 2012, the average math score attained by such students rose by 39 points.
By one extremely rough rule of thumb, this is the equivalent of four academic years. Common rule of thumb: Ten points on the NAEP scale equals one academic year.
Have black kids really gained four years in math over that time period? We don’t think that makes much sense. But journalists are quick to apply that rule of thumb when it produces gloomy conclusions, gloomy conclusions about the size of the achievement gap.
They just never use that rule of thumb to tell the public about those score gains. Indeed: To all appearances, telling the public about those score gains violates Hard Pundit Law.
What kind of score gains have black kids displayed in reading? In this case, we can make direct comparisons dating to 1971. And according to last week’s new data, 13-year-old black students have gained 30 points in reading over that time span. Click here, scroll to page 17.
Those are remarkable score gains—but it seems to be against the law to tell the public about them. This brings us back to Brian Williams and the choice he made last week.
This brings us back to Brian’s song, a song he chose to sing blue, a choice which seems to be required by the diktats of Hard Pundit law.
Brian’s would be an ugly choice, except he spends so much time on his hair that he may not know that he made it. Last week, a new report about the NAEP was simply spilling with large score gains.
But Brian sang a song sung blue. This is the highly selective song the bluesman sang to the public:
WILLIAMS (6/27/13): A grim report card out tonight on our nation’s high school seniors. It’s from the government. The headline is this: They’re leaving high school no better in reading or math than students in the 1970s, decades ago. Certain subgroups, like younger students, blacks and Hispanics, have upped their grades. But apparently, more lower-performing kids are staying in school now, and that brings down the overall graduation grade average.For the most part, that short report was technically accurate, but it was very gloomy. Since the new data spilled with large score gains, how the Faith Hill did Brian Williams ever come up with that song?
Today, we want to tell you that story. It’s a classic story of selective presentation.
The story has two parts.
First, Williams chose to focus on the nation’s 17-year-old students. This let him hurry past the large score gains attained by 9-year-olds and 13-year-olds—and it brought in a bit of statistical noise involving drop-out rates.
The drop-out rate is lower now than it was in the 1970s. In theory, that’s an improvement, of course. But it tends to depress the average score attained by 17-year-old students.
Because fewer kids are dropping out, the NAEP is testing a wider array of the 17-year-old population. Presumably, that tends to depress this group’s average score. Presumably, that’s what Williams meant when he said this:
“Apparently, more lower-performing kids are staying in school now, and that brings down the overall graduation grade average.”
Very few people watching Williams fully grasped what that statement meant. Most likely, Williams didn’t really understand what he was reading either.
But because more kids are staying in school, it’s hard to evaluate the average scores attained by the 17-year-old students. It’s hard to compare today's average scores to those achieved forty years ago, when lower-achieving students were more likely to drop out of school.
So what! Williams chose to focus on the 17-years-olds’ scores—on the murkiest chunk of the data. This let him follow Hard Pundit Law and sing a song sung blue.
Because of the change in drop-out rates, the data for the 17-years-olds are the hardest to interpret. When he focused on those data, Williams told a highly selective tale.
But that is only one of the ways his report was selective. He also ignored the demographic revolution on display in the NAEP’s remarkable data.
Demographically, America’s glorious student population is quite different today. Consider what the HAEP data show about 17-year-old students:
Click here, then scroll to page 55. You will see the following:
In 1975, 84 percent of the 17-year-old student population was white. Twelve percent of those students were black. Four percent were Hispanic.
Today, our country’s student population is gloriously different. In last year’s testing, only 56 percent of the 17-year-old students were white.
Fourteen percent of the students were black. Twenty-two percent were Hispanic!
That represents a gigantic change in the nation’s student population. In our view, it’s a glorious change. A quick point of personal privilege:
In June 2012, we spent an hour in the kindergarten class of a 6-year-old relative. In that class, we got to see a glorious new America even as we were forced to read a story to the class.
Two girls in that small (public school) class weren’t yet speaking English. On their faces, we could see that it’s no fun to be in that situation when you’re only 6.
We got to see something else. As we read, we saw another girl who was bilingual scramble across chairs to cup her hands and whisper in the ear of a struggling classmate. More than a year has passed, and that picture has stuck in our mind as perhaps the best thing we ever got to see with our own eyes.
(For a similar story from a different location, see link below.)
However you want to evaluate it, we have a vastly different student population today. We think the change is a glorious change, but it does create a set of challenges for public schools and their teachers.
This is why a journalist has to “disaggregate” test scores if he has even the slightest intention of telling the truth to the public. If he intends to tell a story which isn’t highly selective.
Williams’ hair was perfect last week, but he failed that test. He focused on the “aggregate” score of the students in question, thereby comparing the new American student population to an earlier population whose demographics were quite different.
In doing so, he was telling one part of a complex story. But he was dumping other large parts of the story. To cite one example, he was refusing to tell the public the truth about black kids.
Even among 17-year-old students, black kids are scoring much higher than was the case in the past. In math, their average score has risen by 21 points since 1978, the first year which provides a direct comparison—and that is before you try to adjust for the change in drop-out rate.
The score gains among younger black students are even larger. Presumably, that's because so confusion is being introduced by changes in drop-out rates. (With very few exceptions, 9-year-olds and 13-year olds never did drop out of school.)
Those score gains by black kids are very large. The public is never told.
Because of the change in our demographics, NAEP data include a complex set of stories. If you want to tell the public the truth, you have to tell the full story—and that involves “disaggregating” the data to show how each major part of the student population has changed over the years.
Here's what you find if you do that:
All three major chunks of our student population are scoring much higher than they did in the past. But powerful interests don’t seem to want you to hear that.
For whatever reason, NBC News has always been an aggressive leader in keeping that story untold. NBC News has always been eager to sing you those gloomy songs.
Williams did a terrible thing when he offered that highly selective account. Once again, the public was given the Mandated Gloomy Story.
On the brighter side, his makeup, hair and wardrobe were brilliant. And he knew that the fiery “progressives” on the cable arm would never breathe a word.
Tomorrow: Some people you ought to despise
"We ran each into the arms of another:" Last year, we saw a child who didn’t yet have any English get help from a 6-year-old classmate who whispered through cupped hands.
Months later, we came upon this remarkable tape. It tells the same story, except in this case, the child who spoke both languages, German and Dutch, was one of the last century's most famous and most brilliant children.
Small glories have happened all over the world! On that YouTube tape from the Anne Frank House, the story is told by Hanneli Goslar, Anne Frank’s childhood friend.
Goslar spoke with her friend in Bergen-Belsen, just once, shortly before she died. That later story is also part of YouTube’s astounding collection.
0 comments:
Post a Comment