Districts that spent the majority of that year learning remotely tended to lose more ground in fourth-grade math scores than districts that reopened sooner. Every 10 additional days of school closures was associated with a roughly 0.2-point loss on NAEP from 2019 to 2022. The pattern was statistically significant and held even when controlling for the share of students eligible for free- or reduced-price lunch, a proxy for poverty.

鈥淭he districts with more remote learning have larger test score losses,鈥� said Emily Oster, a Brown University economics professor who has tracked school closures through the pandemic. 

鈥淚t鈥檚 pretty consistent with what we have seen up until now,鈥� added the researcher, an early and ardent supporter of reopening schools during the pandemic shutdown whose positions were .

The finding adds to the that online learning during the pandemic had a negative impact on student learning outcomes, even while there is renewed debate over how strongly the 2022 NAEP scores reflect it. The highly anticipated results released Monday showed the largest drops ever recorded in 4th and 8th grade math.

Peggy Carr, head of the U.S. Department of Education center that administers NAEP exams, played down any possible relationships between school closures and test results.

鈥淭here is nothing in this data that tells us there is a measurable difference between states and districts based solely on how long schools were closed,鈥� she said during a Friday press conference.

Oster, who also of the relationship between remote learning and NAEP results, called the National Center for Education Statistics director鈥檚 statement 鈥渙dd鈥� and 鈥渘ot very consistent with what we are seeing in the data.鈥�

However, she acknowledged that there is an element of truth to Carr鈥檚 words.

鈥淢aybe what they’re saying is that [school closure] is not the only determinant, and that鈥檚 right. It is not the case that there is a straight line between remoteness and test score losses,鈥� she said.

An NCES spokesperson affirmed that stance Tuesday, denying any 鈥渟imple direct relationship between duration of remote learning and score declines based on NAEP results鈥� in a statement emailed to 社区黑料.

鈥淐ontrolling for free- or reduced-price lunch is helpful but not sufficient,鈥� the spokesperson continued. 鈥淣CES will be conducting analyses that conform to the highest statistical standards, consider multiple variables and link data collected by NCES to other high quality datasets.鈥�

On the whole, results from what鈥檚 known as the Nation鈥檚 Report Card revealed the stark drop offs in math and a slide in reading since 2019, the last time the exam was administered. Some individual school systems, however, performed better than expected, including Los Angeles, among the districts which stayed in remote learning the longest and which saw improvements in reading for fourth graders and in both reading and math for eighth graders.

Since the release of NAEP results on Monday, and have conducted several analyses correlating scores with length of school closures and found moderate, statistically significant links. However, those analyses have largely focused on state data, an approach some experts warn against because it lumps districts that reopened quickly with those that stayed shuttered much longer.

鈥淲ithin states, there’s a lot of heterogeneity in terms of closure policies,鈥� said Tom Loveless, a longtime education researcher who formerly led the Brookings Institution鈥檚 Brown Center on Education Policy.

鈥淟ooking at district data is superior to looking at state data because that’s where the [reopening] decisions were made,鈥� he said.

社区黑料 took the district-level approach, crunching data from a sample of large urban school systems included in the NAEP release. Their scores were then matched with closure data from Oster鈥檚 , which tracked the percentage of the 2020-21 school year that districts offered remote, hybrid or in-person instruction. From the full sample of 26 school systems, Fresno was removed because it had no publicized 2022 NAEP scores and New York City, the nation鈥檚 largest school district, and Shelby County, Tennessee were excluded because they had no district-level school closure data available in the Hub.

Among the 23 remaining school systems, fourth-grade math was the only subject with a statistically significant relationship between district performance and time spent in remote learning. There were weak correlations in fourth-grade reading and eighth-grade math and no association for eighth-grade reading.

鈥淚t was very hard for the little kids to focus on Zoom,鈥� said Robin Lake, director of the Center on Reinventing Public Education. 鈥淚t wouldn’t surprise me if the younger students saw more of an impact on literacy skills and early foundational computational skills.鈥�

Her research group analyzed data on the effects of school closures, finding , especially for younger students and those living in poverty. 

鈥淪chools stayed closed too long, especially in urban areas,鈥� Lake said, noting that her judgment is much easier to make now with the benefit of hindsight as opposed to during the height of COVID when the science on infections and transmissibility was still coming into focus.

The variation in the NAEP results represents 鈥渟hades of badness,鈥� she said. 鈥淪ome states are celebrating not being as bad as other states, but nobody has much to celebrate here.鈥�

NAEP results must be interpreted carefully, experts caution. They are built to show how students are doing, not to explain the reasons behind their performance, Loveless said. (He compared the exam to a thermometer: 鈥淚t can tell you if you have a temperature, but it can鈥檛 tell you why.鈥�)

However, the exam is also the only U.S. test administered to students in all 50 states, making it 鈥渢he only game in town when it comes to comparing across states,鈥� said the former Brookings Institution researcher.

社区黑料 analysis, he said, 鈥渕akes an addition鈥� to the continued dialogue on the impacts of school closures during the pandemic.

Now, with the extent of pandemic missed learning coming into greater focus across the nation, Lake said, it鈥檚 time to hone in on how to respond.

鈥淲e’ve just got a lot of work to do to give kids back what they were owed, both academically and developmentally.鈥�

Oster agreed that it may be time to put aside reopening showdowns and instead work toward recovery.

鈥淭here is a very reasonable desire to move on from the discussion of, 鈥楬ow important were school closures?鈥� into, 鈥楬ow do we fix this?鈥欌�� she said. 鈥淚’m quite sympathetic to that desire to move on.鈥�