Part 4—King of the cherry-pickers: Like all public schools which ever existed, American public schools could do a better job.
On the elementary level, they could do a better job transmitting the basic skills of reading and math. They could do a better job conveying the spirit and the enjoyment of inquiry—that so-called “love of learning.”
With kids who are struggling to meet conventional norms, our schools could do a better job helping children “catch up.” They could do a better job challenging kids who may be especially “talented.”
That said, how good or how bad are American schools? In recent years, how good a job have these schools been doing?
In a nation as large and as varied as ours, that’s a tough question to answer. Last Saturday, in the New York Times, Jal Mehta pretended to try.
In an unusually lengthy op-ed column, Mehta told a very conventional story. We’d say his story was false.
Mehta is a young assistant professor at Harvard, where quite a few students got booted this year in a slightly strange case involving academic integrity.
How different the standards seem to be for Harvard’s assistant professors! In his lengthy column, Mehta copied off the papers of a wide range of our so-called “educational experts.” He cherry-picked his basic facts within an inch of their lives.
This is the portrait Mehta drew at the start of his very lengthy column. The portrait he drew is very familiar, but we would be inclined to call this portrait baldly dishonest:
MEHTA (4/13/13): In April 1983, a federal commission warned in a famous report, “A Nation at Risk,” that American education was a “rising tide of mediocrity.” The alarm it sounded about declining competitiveness touched off a tidal wave of reforms: state standards, charter schools, alternative teacher-certification programs, more money, more test-based “accountability” and, since 2001, two big federal programs, No Child Left Behind and Race to the Top.Mehta started with A Nation at Risk, as people of his class always do. From there, he drew a gloomy portrait of “stubborn mediocrity,” of an ongoing lack of progress.
But while there have been pockets of improvement, particularly among children in elementary school, America’s overall performance in K-12 education remains stubbornly mediocre.
In 2009, the Program for International Student Assessment, which compares student performance across advanced industrialized countries, ranked American 15-year-olds 14th in reading, 17th in science and 25th in math—trailing their counterparts in Belgium, Estonia and Poland. One-third of entering college students need remedial education. Huge gaps by race and class persist: the average black high school senior’s reading scores on the National Assessment of Educational Progress continue to be at the level of the average white eighth grader’s. Seventeen-year-olds score the same in reading as they did in 1971.
As the education scholar Charles M. Payne of the University of Chicago has put it: “So much reform, so little change.”
People like Mehta can, and do, tell this gloomy tale in their sleep. In order to do so, they deliberately cherry-pick their facts, even as they pray for the day when American teachers will be just a bit more like them.
In that passage, Mehta tells a very familiar tale. In the process, he deceives the American public in a very familiar way. The Times should never have published this crap. But this highly scripted, familiar old crap gets peddled somewhere every day.
Here’s what’s wrong with that portrait—and by wrong, we mean “grossly misleading.” Let’s talk about Mehta’s international data, then about his domestic test scores:
International comparisons: At present, the United States and other developed nations take part in three major international test batteries—the PISA, the TIMSS and the PIRLS. American students tend to score better on the TIMSS and the PIRLS, so people like Mehta will frequently stick to the PISA alone.
In that passage, Mehta cites U.S. performance on the 2009 PISA, fails to mention U.S. performance on the 2011 TIMSS and the 2011 PIRLS. American students scored reasonably well on those latter two batteries.
Perhaps for that reason, those more recent tests disappeared.
Some scholars think the PISA is a superior test battery to the TIMSS and the PIRLS; this may be Mehta’s opinion. (Other scholars take a different view.) But the time has come to stop the process by which readers are only told about the tests which produce the lowest scores.
For information regarding the 2011 testing, see our notes below. Let’s add one point before we move on the domestic data, where Mehta’s conduct is simply disgraceful:
In a very typical manifestation, Mehta plays the flag-waving chauvinist, seeming to sniff at the very idea that American students could “trail their counterparts in Belgium, Estonia and Poland.”
That passage ought to be embarrassing. Consider Belgium alone.
Belgium isn’t an underdeveloped Third World nation, struggling with the effects of past colonialism. Belgium is a developed, advanced western European nation. It was one of the colonizers!
There is no reason why we should assume that American students will, on average, be more accomplished in reading and math than students in Belgium are. For one thing, American schools face certain challenges based upon our brutal racial history, challenges which are not faced by Belgian schools. Whatever their other failings, the Belgians didn’t spend three hundred years trying to eliminate literacy among a large swath of their population.
Our benighted ancestors did. The tragic effects are still apparent in our public schools.
Different countries face different types of demographic challenges; it’s hard to balance the equities when we discuss such matters. At any rate, Belgian students did outscore American students in reading on the 2009 PISA, but only by a narrow margin. (“Statistically insignificant,” the PISA documents say.) In math, they outscored American students by a wider margin.
On the 2011 TIMSS, Belgian students narrowly outscored American students in math—but American students outscored their Belgian peers by a wider margin in science. And American students smoked the Belgians, by a wide margin, on the 2011 PIRLS. (The PIRLS tests reading alone.)
Having said that, forget what we’ve told you! If you read the New York Times, you will only hear a mocking complaint about the way the Belgians outscore us. The snark comes straight from the planets Jingo and Ugly American. And as Mehta snarks, the mixed outcomes in the data are simply withheld.
Should it be shocking if U.S. students are outscored on the PISA by students in tiny Estonia, a literate Baltic state? (In reading, U.S. students were outscored by one point, 501-500. Estonia did not take part in the TIMSS or the PIRLS.) Actually no, it should not be. But people like Mehta will play every card as they tell the gloomy tale that is loved in one part of our world.
The domestic data: Mehta’s international cherry-picking ought to be an embarrassment. It should be an embarrassment for Harvard and for the New York Times.
On the domestic front, his conduct is a disgrace. Let’s consider the various ways he cherry-picks from the National Assessment of Educational Progress (the NAEP), the widely-praised “gold standard” of domestic educational testing.
How does Mehta cherry-pick? Let us count the ways:
He cherry-picks data on the basis of age. He cherry-picks data by dealing in average scores alone—by failing to “disaggregate.”
To some extent, he cherry-picks data by citing the NAEP’s Long Term Trends Assessment, rather than the parallel study which is typically called the Main NEAP. He cherry-picks data by citing “gaps by race and class” without explaining that the gaps persist in spite of the large score gains shown by black and Hispanic students.
You see, white students have shown substantial score gains too. These score gains have worked to maintain the gaps, although the gaps have grown smaller. But those “gaps by race” persist despite large gains by black and Hispanic students.
Would you have any idea of such facts from reading Mehta’s account?
“So much reform, so little change?” Since Mehta himself has chosen the NAEP; since he himself has chosen to cite the NAEP’s Long-Term Trend study; since he himself chose to measure progress from 1983; since Mehta himself has made those selections, let’s see what kinds of change has occurred on the Long-Term NAEP since that ballyhooed date.
Warning: In what follows, we aren’t trying to tell you why the score gains in question occurred. Did they happen because of better instruction? Did they happen because of the removal of lead from the air?
We aren’t trying to answer those questions. We’re simply showing you some of the data our bright fellow disappeared.
The data which disappeared: Poor Mehta! He’s saddened by the mediocrity he can see all around him! He’s saddened by the lack of progress—by the “failed profession” of teaching.
He’s saddened to see that American teachers don't have a “professional code” to match those of other major professions—presumably, to match the lofty professional code observed by scholars like him.
“So much reform, so little change,” one of his experts exclaims. In this way, New York Times readers—American citizens—are once again being deceived.
What kind of change has occurred on the Long Term NAEP since 1983? That’s the testing program Mehta chose, and the baseline year.
Thanks for asking! This kind of change has occurred since that time and since 1971, when the NAEP program began:
The Long Term NAEP tests 9-year-old students, 13-year old students, and 17-year-old students. Because drop-out rates have decreased over time, it can be a bit difficult to construct meaningful comparisons among 17-year-olds.
But here’s what has happened among 9-year-olds and 13-year-olds, where almost everyone always was in school. We’ll measure from 1983 and from the early 1970s, when NAEP testing began. We’ll adjust for one minor hiccup in testing procedure the NAEP made along the way:
What kind of “progress” has been displayed by 9-year-old black students in reading? Good God! From 1971 to 2008, the average score for this group increased by 37 points, an astonishing gain on the NAEP scale. From 1984 to 2008, the gain was 21 points. (2008 is the most recent year for which Long Term NAEP data are available.)
Those are very large score gains. A rough rule of thumb is often used when journalists discuss the NAEP. According to this rule of thumb, ten points on the NAEP scale is roughly equal to one academic year.
(Warning: Journalists only apply this rule of thumb when it produces gloomy conclusions. That is, they apply it to the achievement gaps, not to the score gains of each racial group, which typically go unreported.)
Do ten points really equal one academic year? We regard that as a very rough rule of thumb. But it helps you see how crazy it is when Mehta selects the Long Term NAEP, then moans about “so little change.”
Those were the score gains in reading. What kind of progress has been displayed by 9-year-old black students in math?
Good question! From 1973 through 2008, the gain has been that same 37 points. (1973 was the first year the NAEP administered math tests.)
Among 13-year-old black students, the gains have been similar, both since the start of the NAEP and since A Nation at Risk. In reading, the average score for this group rose by 30 points from 1971 through 2008. In math, the average score rose by 39 points from 1973 through 2008.
Taken on their face, these are very large score gains. How can a scholar cite the NAEP, then moan about “so little change?”
He can do so by cherry-picking, the skill at which Mehta excels.
How much progress has occurred on the NAEP? Consider one remarkable fact from the so-called Main NAEP, the companion program to the Long Term Trends Assessment.
Surely, Mehta has heard this fact. We’ll quote educational expert Richard Rothstein, citing this remarkable fact in Slate:
ROTHSTEIN (8/29/11): The only consistent data on student achievement come from a federal sample, the National Assessment of Educational Progress. Though you would never know it from the state of public alarm about education, the numbers show that regular public school performance has skyrocketed in the last two decades to the point that, for example, black elementary school students now have better math skills than whites had only 20 years ago.Astonishing! As of 2007, black fourth graders were scoring higher in math than white fourth graders scored in 1992. That is an astonishing fact. No one could have foreseen it.
Surely, Mehta has heard that fact. It's hard to know how a scholar can consider that fact from the program which he himself cites, then moan about stubborn mediocrity and “so little change.”
Tomorrow, we’ll continue this sad rumination. Mehta will still be holding his job along the banks of the Charles.
Tomorrow: Quite reminiscent of Pravda
Scores on the TIMSS and the PIRLS: How did American students score on the 2011 TIMSS and the 2011 PIRLS? What countries did they surpass?
See our award-winning series, Fooled About Schools. For part 1, just click here.
0 comments:
Post a Comment