After six months of running through the Scooby-Doo doors of North Carolina’s COVID-19 dashboard, last week’s addition of antigen tests and cases pushed me too far. That means one less meddling kid in the Mystery Machine.
Not all of the changes have been the result of Department of Health and Human Services actions, but DHHS has never been transparent with its data, its processes, or its decision criteria, which has undermined trust in what it has reported and stoked suspicions when it has made revisions. Don van der Vaart, Jon Sanders, and I have documented some of the inconsistencies and unanswered questions in a series of updates for the John Locke Foundation under the banner “The fog of COVID-19 data.” This will be my last contribution to the effort.
First, I should probably start with why the addition of data led me to stop trying to track data. The simple answer is the change made it impossible to compare recent days. When the difference between good and bad numbers is as small as it had become in the first weeks of September, minor changes in methodology can make a big difference. The change went into effect on Friday with no prior notice and coincidentally added enough new positive antigen tests to push the number of cases over 200,000. Cases was the only number the state had not revised to a new methodology since March, until they added the antigen test results en bloc, which also had the effect of pushing the number of currently active cases up by fifty percent.
Hospitalization rates
As treatment has improved and the disease has spread to less vulnerable populations, the criteria for hospitalization have eased. Despite that, except for a spike to 11% at the end of August, the seven-day moving average for hospitalizations as a share of cases two weeks earlier has been under 9% since late July and has been closer to 8% since Labor Day. The rate has fallen from 206.9% on April 1 to 25.5% on May 1, 16.7% on June 1, 10.2% on July 1, 8.5% on August 1, back to 10.5% on September 1, then back around 8% before climbing back to 9.5% September 22.
The number of hospitalizations peaked July 28 at 1,291 and has hovered largely between 850 and 950 since the end of August. New admissions in the previous 24 hours have generally made up half of the total COVID hospitalizations, with 75-80% of those being suspected cases instead of confirmed cases. COVID hospitalizations have taken 6-7% of occupied beds at reporting hospitals, 4-5% of all beds reported as staffed, and 3% of all 25,000 licensed beds since mid-August.
The number of licensed beds has also changed. Originally it was 21,222, then bumped up to 25,309 July 18 when larger hospitals began reporting more beds to the state, and had since ranged as high as 25,622 and as low as 25,466 until September 22. Twice in the past month, however, DHHS has had to go back to change 14 days’ worth of data as a group.
Positive Tests
The most visibly inconsistent information on the dashboard for weeks was the seven-day rolling average of the percentage of tests returning a positive result. Although DHHS reported a daily number of cases and a daily number of tests, the reported percent positive never matched.
A footnote provided the explanation that the percent positive was indeed based on different numbers than reported in the rest of the dashboard. It was the percent of tests reported electronically by labs that reported positive and negative results electronically. This sounded at first like a subset of the data, but was later explained to be based on completely different data. The headline numbers are aggregates reported completely separately from the electronic reporting system with no way to reconcile the two.
Worse, when asked for the raw data used to calculate the percent positive, DHHS responded, “We do not have this data in a shareable format.”
Also for weeks, the seven-day average never seemed to match the daily percent positive reported. Theories abounded online to explain away the difference, but nobody could get an answer from DHHS until one day the line changed, again with no explanation. For some reason, “A dual axis format is used to add the trend line to this chart. We encountered an issue at one point where the dual axes were not synchronized, and we remedied the issue as soon as it was discovered and reviewed.” This makes sense if “at one point” means “pretty much from the start” and “as soon as it was discovered and reviewed” means “well after it should have been obvious.” As an added bonus, the graph has never had a visible second access.
We should also recognize that DHHS takes a different approach to calculating the seven-day rolling average. Instead of taking the six previous days and the reported day as seven days as most people do, DHHS uses “includes the previous 3 days of data + today’s data + the next three days of data.” The agency did not explain why.
Implications
Jeffrey Selingo wrote his recent book Who Gets In and Why in order to, he states, “reveal the smoke and mirrors that have made applying to college a kind of mythical quest.” One of the worst parts of the college admissions process, especially at the most prestigious schools, is the concept of holistic admissions. Selingo writes, “Although it has the veneer of numerical precision, holistic admissions is pretty subjective. While colleges like the flexibility the process gives for a rough sorting of applicants, to a skeptical public, holistic admissions is confusing and secretive at best and nefarious and collusive at worst.”
Confusing and secretive, or nefarious and collusive, beneath a veneer of numerical precision — that sounds familiar. Gov. Roy Cooper and DHHS Secretary Mandy Cohen often say “science and data” guide their decisions to restrict social and economic interactions. Cooper has not provided one for either his 10,000-seat threshold on stadiums or the 7% limit on their capacity. Nobody has been able to explain why seemingly similar businesses have different rules on whether they can open and how many people they can serve.
Cohen has not provide a rationale for her 5% benchmark for positive tests as an indicator of contagion spread and opening. The World Health Organization (WHO) also set 5% as a benchmark, but it was as a benchmark for adequacy of testing, not for closing or opening an economy. By WHO’s logic, if a state has a positive rate higher than 5%, it should expand testing to asymptomatic people. That is not the approach North Carolina has taken, where testing has been stalled around 25,000, not much above the 24,000 daily in July.
It is not clear what the 5% benchmark even means for North Carolina. “Because no one metric is perfect,” a spokesperson explained, “we base decisions on a composite picture of all the various metrics as opposed to just one single metric.”
Just as students and parents will never be able to understand the difference between acceptance and rejection, we also will not be able to understand the seemingly arbitrary decisions on what activities are allowed by executive order in North Carolina. Since the standards for coronavirus data reporting change arbitrarily, they have little apparent bearing on decisions of life and death, and since the governor has shielded himself from public accountability, tracking the data has proven to be less of a Scooby-Doo pursuit and more of an Animal House-esque “really futile and stupid gesture.”
