by Jon Sanders
Research Editor and Senior Fellow, Regulatory Studies, John Locke Foundation
We have written multiple times in dismay over the many, many problems with the reporting of data related to COVID-19 coming out of the Cooper administration. Nevertheless, the landing page for the North Carolina Department of Health and Human Services (DHHS) boasts
Data quality rated A+ by The COVID Tracking Project.
There’s grade inflation, and then there’s giving Cooper’s DHHS an A+ on data quality. Their figures are subject to frequent, even radical changes (see list below). But to put a rumor to bed: there’s no proof that the other A+ data quality rating went to Chef Boyardee for spilling a bowl of kids’ “ABC’s & 123’s Pasta.”
Still, I was curious about that rating, so I decided to check into it. Interestingly enough, the COVID Tracking Project has abandoned its grading system, saying “A single letter grade no longer feels like the right reflection of data quality.” They’ve replaced it with metrics that define aspects of states’ data reporting as to whether few, some, or serious problems exist with them.
For North Carolina’s data reporting, the COVID Tracking Project lists “Some problems exist” across the board. The project lists handfuls of such problems, such as “data for multiple key metrics is hard to discover or access” and “North Carolina switched from reporting current outbreak data to cumulative data on June 25, 2020. Interpret North Carolina’s historic data with caution.”
Man, nothing says A+ quite like Interpret their data with caution. Dante’s gate to Hell (“Abandon all hope, ye who enter”) probably earns at least an A on that scale.
I’ll conclude with the project’s history of data problems from DHHS (emphasis in original):
On February 28, 2021, North Carolina noted that its data dashboards would not update on that day in order to make changes to improve system performance.
On February 3, 2021, the North Carolina Department of Health and Human Services reported that the day’s case counts were much larger than usual because of the inclusion of a backlog of nearly 8,000 cases identified at FastMed clinics during December 2020 and January 2021.
On January 18, 2021, North Carolina announced on their COVID-19 dashboard that there will be no daily update due to the holiday, and that their data will next be updated on Tuesday, January 19, 2021.
On January 13, 2021, North Carolina reported that technical issues occurring on January 12, 2021 prevented processing of case and testing data into their COVID reporting system. As a result, case and testing data reported on January 13, 2021 are lower than they would have been had all the data been processed, and case and testing data on January 14, 2021 will be increased, since it will include the previous day’s data.
On December 30, 2020, North Carolina announced on its dashboard that due to holiday-related disruptions and technical issues, data reported on December 30, 2020 was artificially higher than usual and includes 36 hours of data, while data reported on December 31, 2020was artificially lower than usual and includes only 12 hours of data.
On December 24, 2020, North Carolina announced on their COVID-19 dashboard that their data would not be updated on December 24, 2020, or December 25, 2020 due to the holiday. On January 1, 2021, they noted that there would be no update to their data on January 1, 2021 due to the holiday.
On November 26, 2020, North Carolina announced on their COVID-19 Data Dashboard that there would be no update to their data on November 26, 2020 due to the thanksgiving holiday. However, on November 27, 2020, they will provide data retroactively for November 26, 2020 and we will backfill our historical data accordingly.
On November 13, 2020, the state of North Carolina announced that they are re-defining hospitalization data based on new guidance, whereby hospitalizations are counted by the entire length of stay and not just by length of isolation. This change will likely affect our Now hospitalized metric for the state
On September 26, 2020, North Carolina started reporting deaths among individuals who had tested positive via antigen. We record this figure in our Deaths (probable) field.
On September 25, 2020, North Carolina started reporting probable cases which caused the Total Cases to increase by 4,563.
On August 12, 2020, North Carolina announced a major correction in its Total PCR tests (specimens), resulting in a decrease of more than 220,000 reported tests.
On July 22, 2020, North Carolina started reporting the number of patients currently in the ICU.