The Washington Post reports record test scores!

WEDNESDAY, JULY 31, 2013

It happens every spring: Everything we know about Emma Brown is good.

Brown is an education reporter for the Washington Post. She graduated from Stanford in 2000. From 2002 through 2005, she taught seventh grade in Juneau, a city believed to be in Alaska.

In 2009, she got a master’s degree in journalism from Berkeley. This morning, she writes a front-page news report, a type of report we have read ten million times before this.

Her report may be right on the money. But it’s very hard to tell—and it happens every spring!

Brown reports that DC test scores have never been higher. At the start of her report, she quotes the standard exultant claims,which may of course be right:

BROWN (7/31/13): Students in the District’s traditional public schools scored higher than ever on the city’s math and reading tests this year, also posting the largest single-year gain since 2008, according to test results released Tuesday.

The city’s public charter schools, which had higher scores than the traditional system, made their biggest gains since 2009. For the first time, more than half of charter students scored proficient or above in reading on the city tests.

Mayor Vincent C. Gray (D) hailed the results as evidence that the city’s overhaul of public education—including the advent of mayoral control of the schools and the rapid growth of charters—is working.

“I don’t think there’s any doubt we’re on the right path,” Gray said. “We just need to stay the course.”

It happens every spring! The mayor says his overhaul is working, and he may even be right.

That said, there’s a problem with Brown’s report. There’s a watchdog which isn’t barking.

Can we talk? Gains in test scores only have meaning if this year’s tests were the same as last year’s tests, or if they were “equivalent” to them. If these year’s tests were easier than last year’s tests, those score gains don’t mean squat.

Were this year’s tests the same as last year’s? If not, do we know that they were equally difficult? Brown’s report doesn’t say. We think this is somewhat odd.

In recent years, the problem of statewide tests getting easier year to year has come up again and again. Major states have thrown out entire batches of test scores because of this phenomenon.

A few years back, the state of New York, a well-known state, dumped a whole year’s worth of scores because the tests had gotten easier over the years.

The New York Times worked very hard to obscure the way this gong-show occurred, since it reflected badly on Mayor Bloomberg, a billionaire. But this has been a recurrent problem around the nation—unless you read today’s Post.

Maybe there’s some simple answer to our question, and Brown knows what it is. Maybe this year’s “statewide” tests in DC were the same tests from last year!

But the Post gave Brown 1500 words to discuss the latest score gains. Do we know that this year’s tests weren’t easier? How hard would this be to explain?

In recent years, DC was plagued by cheating scandals, a problem education reporters had been avoiding for forty years at that point. Proudly fighting the last war, Brown addresses that situation, as she should have done.

But she never says if this year’s tests are known to be as difficult as last year’s. And sure enough! When she presented the requisite words of caution about score gains, we got the standard piddle:

BROWN: Some experts cautioned that standardized testing doesn’t always tell the whole story. Bob Schaeffer of FairTest said it would be inappropriate to attach much importance to single-year gains. He said scores fluctuate and can reflect demographic changes in schools instead of changes in teaching and learning.

“The exclusive focus on test scores as the measure of educational quality should be replaced with the use of multiple performance measures including rates of graduation, college attendance, post-school employment, criminal justice system involvement, etc.,” Schaeffer said in an e-mail.

That highlighted passage is piddle.

Schaeffer is right—score gains can “reflect demographic changes in schools instead of changes in teaching and learning.” But in this case, the score gains would have to reflect a demographic change in the whole DC student population!

Schaeffer’s participation is required by law in a piece of this type. In this case, his comments are almost wholly irrelevant—and these words of caution are even worse:

BROWN (continuing directly): Sam Chaltain, a journalist who is working on a book about school choice in the city and who is the parent of D.C. charter school students, was similarly skeptical.

“We use test scores as a proxy to make it seem like we actually know whether schools are succeeding or failing,” Chaltain said, adding that the rise in scores in the District leaves much unknown. “We don’t know if kids feel more engaged and motivated. We don’t know if teachers feel more supported and prepared to do their jobs well. We don’t know if families are more or less likely to stay in the system.”

We also don’t know how the students feel about the name of Washington’s NFL team! But that is completely beside the point here, as are the three concerns Chaltain is worrying about.

Were these tests as hard as last year’s tests? If so, how do we know that? This is basic journalistic blocking and tackling, and it’s MIA here.

When we read about Trayvon Martin in the white establishment press, we think about decades of work of this type. Do white liberals actually care about black kids? Or does the establishment press just like to posture and pose?

Plainly, no—we don’t seem to care. If you actually care about a topic, you jealously guard the basic facts. You struggle to get your basic blocking and tackling right.

We have watched the establishment press produce this kind of stumblebum work for more than forty years now. In this case, Brown is fairly new to this beat—but how long has her editor been there?

Urban schools have always been reported in this half-hearted way. At some point, it gets hard to avoid the feeling that these people don’t really care, that our urban schools exist to let assorted public figures take familiar bows.

There may be an answer to our question. We’d like to know what it is.