The news that D.C. Public Schools found a sharp increase in the number of students graduating despite missing a large proportion of instructional days at school was a punch in the gut for a district that’s been hailed as the fastest improving and one that’s touted its increasing graduation rates for the past six years in a row.

In fact, only 178 graduates out of 2,307 from all DCPS high schools had satisfactory attendance during the 2016-2017 school year.

But D.C. isn’t the only school system unearthing issues with graduation rates.

Right on the heels of the D.C. disclosure came a report from the Department of Education’s Office of Inspector General took California to task for inflating its high school graduation rate.

In an audit for the class of 2014, the office found that the state had removed about 10,500 students from the incoming freshmen cohort for reasons that did not align with federal law, and also counted nearly 2,000 students as graduates despite not meeting certain requirements for a diploma.

Under federal law, schools can only subtract from the incoming freshman count if a student transfers to another school, emigrates to another country or dies. Had state education officials used those rules, the audit found, the high school graduation rate for the 2013-2014 class would have been 79 percent instead of 81 percent.

Others are struggling with graduation rate data as well: Last summer, for example, the Office of Inspector General found Alabama guilty of fudging numbers by counting students who earned alternative diplomas instead of a high school diploma, and those who hadn’t earned enough credit to graduate.

The unearthing of anomalies with graduation rates comes as the metric begins to play a heightened role in state K-12 accountability systems. That pressure, combined with recent instances of data manipulation, have prompted questions about whether schools are being transparent, whether they’re trying to game the system, and, more generally, what the value of a diploma really is.

One person who isn’t so surprised at the recent incidents: Paige Kowalski, executive vice president at Data Quality Campaign, an organization that advocates for better and more transparent education data, which published a paper in 2015 predicting this exact scenario.

“We did the paper for a good reason,” she says. “This thing nailed it.”

Indeed, the report specifically calls into question some data points that states were beginning to use to calculate graduation rates – things like counting alternative school and GED recipients as graduates or improperly coding dropouts as transfer students.

“The demonstrated increase in the number of students graduating from high school has naturally led many to ask how states were able to graduate so many students,” the paper reads.. “As stakeholders dig into the data, they are seeing enough differences across states to lead to questions about the degree of comparability and whether this metric is useful in measuring student outcomes.”

To be sure, it wouldn’t be the first that time that states and districts allowed one area of K-12 education to flail in order to drive increases in another – an unfortunate and unintended, though by no means uncommon, consequence of policies that accentuate performance measures..

When Congress passed No Child Left Behind, it tasked states with increasing the percentage of students reaching proficiency in math and reading each year, culminating with a deadline by which all students in the U.S. would be on grade level. The ambitious goal – considered unattainable my most – resulted in some states lowering their academic standards each year so that more students could meet the proficiency threshold.

But, in this case, it turns out graduation rates have become much more accurate than ever over the last decade, Kowalski notes.

“It’s better than ever before,” she says, pointing out that states once calculated graduation rates based on the number of students who entered 12th grade and finished 12th grade.

The issue now, she says, is because graduation rates play an elevated role in accountability systems, along with a slew of other data indicators, there’s pressure to increase those rates, which states and districts are doing through things like credit recovery programs and alternative graduation programs.

It’s OK for states and school districts to adopt those policies, Kowalski says, so long as they’re transparent about the fact they are using those strategies in calculating their graduation rates.

Kowalski points to Arkansas, which provided information to districts and schools about which students they’ve identified as dropping out so that the district is aware that they are being held accountable for those students.

Another standout: Maryland.

“They show their math,” she says. “They are really clear about who is included in the cohort and who isn’t. That builds trust.”

The focus on graduation rates and the metrics and data used by states and districts to calculate those rates is happening during a data boom in the education space, when states and school districts are collecting, using and publishing more and better data than ever before.

“Inevitably, the more information we have access to and the more information we look at, the more questions we will have about data quality,” Natasha Ushomirsky, director of P-12 policy development at the Education Trust, an education civil rights group, says. “By broadening the spectrum of information we look at, we’re absolutely going to uncover things that didn’t get sufficient attention before.”

What the case in D.C. makes clear, she says, is the importance of tracking multiple kinds of data. If D.C. wasn’t tracking chronic absenteeism, for example, it would have no way of knowing the discrepancy even existed.

Indeed, the issue of chronic absenteeism came to light in 2016, after the Department of Education’s Office for Civil Rights released its sprawling biennial report that included for the first time ever data about students who are considered chronically absent from school, as defined by students who miss 15 or more of the school year’s roughly 180 days.

Nearly 1 in 5 high school students is “chronically absent” from classes, the report showed, and the percentage of students who miss at least 15 days in the course of an academic year only rises among students of color. Notably, the data identified D.C. schools as having one of the highest rates of chronic absenteeism in the country, with 32 percent of students considered chronically absent.

“One thing that we need to remember is that even with these data quality issues, we’re still in a better place now than we were even a decade ago,” says Ushomirsky. “As data quality improves, the more we’re going to know and the more we’re going to attend to problems that come up.”