What is the panic about pandemic learning loss actually about?
There are some legitimate reasons for concern. The testing declines are real and significant, returning national performance in math and reading to levels last seen a couple of decades ago, and imposing the largest and more worrying setbacks on the most vulnerable students. At the nationwide level, educational gains are exceedingly hard to come by, which means even modest setbacks are worrying, too. There are, as always, good reasons to ask what can be done to address learning shortfalls, and how to best support teachers and schools in addressing them — particularly as billions in federal funding to counter learning loss awaits distribution.
But when I look at the data in detail, I just don’t see the signs of catastrophe that so many others seem to. I’m inclined to see that data as, at least, a glass half full, if not quite a best-case scenario. That’s because the declines, all told, strike me as relatively small, given the context: a brutal pandemic that terrified the country and killed more than a million of its citizens, upending nearly every aspect of our lives along the way.
The panic of parents and policymakers is both unsurprising and, to a degree, productive: As we approach the three-year anniversary of the beginning of the pandemic, we should be thinking about what went right, and what went wrong, with school closures and other mitigation policies. But in doing so let’s try not to forget the scale of the impact or the context in which it happened.
As a country, prompted in part by midterm elections, we are now doing some of that reckoning — mostly in a one-sided way, with Republicans running on the Covid excesses of liberals and Democrats mostly trying to avoid the subject. But it isn’t just partisanship skewing things. We are talking a lot more about possible policy overreach — on school closures, on mask mandates — than we are about how brutal and disorienting the pandemic actually was. And we aren’t doing that in a world in which the whole thing turned out to be no big deal, or some false alarm panic that made initial precautions seem absurd and retrospective policy questions abstract. We’re doing it in a world in which a million Americans died — and we’re judging choices made before vaccines and Paxlovid and widespread natural immunity, when the risk of death was 10 times higher, by the much more laissez-faire standards we settled on much later.
ADVERTISEMENT
Continue reading the main story
Could we have managed the first year of the pandemic more strategically, doing more to protect the vulnerable and prioritize essential functions like schools? Almost certainly. (Personally, I would’ve liked to have seen schools open nationwide in fall 2020, with additional focus on rapid testing and improved ventilation.) Do we know how well each mitigation measure suppressed spread and saved lives? Not as clearly as we might like if we were trying to strategize a plan for future pandemics, and we may well be less universally restrictive if given another chance. But however open these questions may seem to you today, they were first asked not in the context of endemic Covid but of mass death and illness, uncertainty and anxiety and social disarray.
In that context, how did the kids fare? Last month, I wrote about early learning loss data from the National Assessment for Educational Progress tests, which showed, that for the country’s 9-year-olds, average test scores for math fell to 234 (out of 500) in 2022, down from 241 in 2020. For reading the scores fell to 215 in 2022 from 220 in 2020. These declines represented a setback of a couple of decades, since the average math score had been 232 in 1999 and the average reading score had been 216 in 2004. The scores varied from student to student and district to district, but nationally the effect did not resemble the cancellation of school. It was the equivalent of taking the nation’s schoolchildren, putting them in a time machine, and sending them off to be educated sometime around the year 2000.
This week, N.A.E.P. released new data covering 450,000 fourth and eighth graders, generating a new round of hand–wringing. (The Washington Post called it “a generational emergency.”) But the new data only confirms the same story.
For fourth graders, national performance in math fell from 241 in 2019 to an average of 236 in 2022 (close to what it had been in 2005). In reading, average scores by fourth graders fell to 217 in 2022 from 220 in 2019, more or less matching the average performance of 219 by fourth graders from 2005. For eighth graders, the average reading score in 2022 was 260, down from 263 in 2019. In math, the decline was a bit bigger, falling to an average of 274 in 2022 from 282 in 2019, erasing a couple of decades of gains and matching the scores achieved in 2000, when the national average was 273.
All told, at least as judged by test scores, the effect of extensive and perhaps excessive disruption to schooling was to return the country as a whole to the levels of educational achievement of the No Child Left Behind years.
State by state, it is hard to draw a line between school closures and learning loss, since some states that stayed closed longest fared best, and vice versa. Earlier research showed a clearer relationship between school closures and learning loss at the district level, but at a news conference announcing the latest N.A.E.P. report, the commissioner of the National Center for Education Statistics said, “There’s nothing in this data that tells us there is a measurable difference in the performance between states and districts based solely on how long schools were closed.”
In New York City, the nation’s largest school district, schools reopened in September 2020. There, average scores for reading fell by about a point for fourth graders and improved by about a point for eighth graders; in math, fourth-grade scores fell by nine points (statewide scores fell by 12) and eighth-grade scores fell by four points (statewide scores fell by six). In Los Angeles, the second-largest district, schools stayed closed through January 2021. There, average scores actually improved in fourth-grade reading, eighth-grade math and eighth-grade reading, where they improved by a robust nine points (to 257 from 248). Scores fell only in fourth-grade math (to 220 from 224).
In a vacuum, the pandemic declines look like bad news, if at a relatively small scale. But none of this happened in a vacuum. I’ve mentioned the million deaths not to fearmonger about how much higher those numbers might have been without school closures — the scale of that impact is, I believe, an open question — but just to point out the enormous and widespread human impact of the disease itself. And that impact was much larger than measured simply by mortality. More than 3.5 million Americans were hospitalized, according to one estimate, and probably at least as many suffered from long Covid. In the spring of 2020, the country’s unemployment rate exploded, jumping to nearly 15 percent from about 4 percent; for a brief period in April, six million new jobless claims were filed each week. In a single quarter, U.S. GDP fell by 9 percent. Murder rates grew by 30 percent; deadly car crashes spiked, too. Overdose deaths rose 30 percent in 2020 and 15 percent in 2021. According to some research, rates of depression tripled in the United States when the pandemic first hit. Some 600,000 teachers left the profession.
This is the world in which American students — most of them learning remotely for many months, many of them for close to a year, and some for longer — fell off by a handful of points, on their reading and math exams, compared with their prepandemic peers.
“The sudden onset of the pandemic has been the most catastrophic event in recent American history, making the expectation that there would not be something called ‘learning loss’ bizarre,” Keeanga-Yamahtta Taylor wrote recently in The New Yorker. “The idea that life would simply churn on in the same way it always has only underscores the extent to which there have been two distinct experiences of the pandemic,” she went on, emphasizing how much harder the pandemic was for the poor and marginalized to navigate, compared with those for whom its secondary effects were buffeted by wealth.
International comparisons offer another bit of context for test score declines. In England, schools closed in the spring of 2020, opening again in some places in early summer and across the country in the fall (with an Omicron interruption of about a month that winter of 2021). In retrospect, that would have been a plausible but relatively aggressive school reopening approach in the United States, where many schools stayed remote well into the 2020-2021 school year. It also resulted in a drop of six percentage points in proficiency scores, roughly comparable to the American experience. In other words, in England, with a close-to-optimal school reopening, they fared no better.
In the Netherlands, where schools were even less disrupted than in Britain, student performance fell by three percentage points — a bit better, but still below the standards set in prepandemic years. At the most extreme end of the spectrum, there is Sweden, which did not close schools at all, and which, some reporting has suggested, experienced no such declines. But the country also suspended its testing program, which means the data on which such claims might be based is pretty shaky.
The American data, by contrast, is quite strong, and the picture is clear: Almost everywhere, there have been declines, generally modest. And yet rhetoric about the costs of school closure appears to be only intensifying — with the secretary of education calling the test scores “appalling and unacceptable,” for instance, adding that our response to the declines will determine “our nation’s standing in the world.”
Perhaps it makes sense that in this period of reflection, schools would become such a hotly debated pandemic touchstone. Schools are hugely important, educational gains exceedingly hard to come by, and the setbacks are both real and large enough to justify plenty of genuine concern. Schools are also among the civic institutions Americans are most engaged with and involved in, which means that — unlike the running of hospitals, say, or pandemic policy in the military — the question of school closure felt much more immediate to many more people, including those whose lives were otherwise relatively untouched by pandemic disruption.
School closures also had costs that went well beyond test scores — the social and emotional costs of isolation for children and the additional impact on many parents — and those scores are perhaps one way of talking generally about that larger burden. Like crime, school performance is a perennial and widespread source of American anxiety, and earlier conflicts — between reformers and teachers’ unions, between parents and local bureaucracies — offered a kind of road map for fights over pandemic policy. Those concerned about social inequalities could see easily and clearly, beginning in the fall of 2020, just how differently the matter of reopening was being handled by private schools, public schools in wealthy places and public schools in poor places — and be rightfully outraged by divergence.
But I think, alongside those explanations, there’s something else: Americans as a whole are not exactly happy with how those two years went, and the pandemic has left almost all of us with some excess of rage and frustration. Early on, that was channeled into partisanship, with liberals blaming President Donald Trump for the pandemic itself and conservatives blaming liberals for pandemic restrictions. But the lines of Covid partisanship are much muddier now — a few years on, there’s a Democrat in the White House, and a growing recognition that, while policy matters and political leadership have surely failed the country, the virus was going to wreak some amount of havoc regardless.
But that idea remains uncomfortable for many, that the pandemic was not just a policy failure or political choice but a generational and global public-health trauma against which very few of our peer countries fared very well either. And yet even as it grows harder to pin responsibility on one party or one president, we want to pin it somewhere, on some human or humans or human institution, if only to tell ourselves that if we make the right choices we will never have to live through all that again. As a country, we burden our schools with an almost impossible set of responsibilities — undoing racial disparities, for instance, or closing yawning income gaps. It makes sense that we’ve piled additional frustration and rage on them, wanting to believe schools could have navigated the pandemic smoothly, too. But to judge by the test scores, at least, they came remarkably close.