Coronavirus

Lies, damned lies and COVID statistics

Understanding pandemics requires lots of data so reader beware of arguments based on a single-metric

Governor Cuomo's powerpoint mocking the Federal Government's COVID response.

Governor Cuomo's powerpoint mocking the Federal Government's COVID response. Mike Groll/Office of Governor Andrew M. Cuomo

Just how bad is COVID-19 surging in New York? The best answer to that question has changed over time. Total confirmed cases used to be the headline data point early on in the pandemic. Then came hospitalizations and deaths. Now, two other measures are focal points: testing positivity rate and the number of new cases per 100,000 people. 

“It’s not one metric that can tell you how a given area is doing during the epidemic,” Syra Madad, senior director at the System-wide Special Pathogens Program Office at NYC Health + Hospitals, said in an interview. “You have to look at multiple different factors.” 

These metrics all have their own statistical biases that help tell different stories about the state of the pandemic – a useful tool for anyone looking for a rhetorical edge in arguing whether things are getting better or worse in the Empire State. 

Both New York City and state regularly publish an assortment of data about the pandemic, but elected officials typically lean on a few choice metrics in their public messaging. Hospitalizations and deaths are useful, but they are lagging indicators that show the consequences of infections from days and weeks past. Real-time understanding of the pandemic largely depends on finding who has the coronavirus among the tens of thousands of people who show up every day to testing centers for whatever reason – whether out of legitimate fear of the infection or simple curiosity.

Gov. Andrew Cuomo, for example, has touted for months how the testing positivity rate – the number of reported tests that detect COVID – is much lower in New York than in other states. The 1-2% rate typically reported by the governor each day is well below the 5% threshold that health experts say is the difference between keeping the pandemic under control and beginning to see the coronavirus run amok. 

Testing the positivity rate also signals whether there is enough testing being done overall. “Early in the crisis, when we were doing positivity on our tracker, I remember looking at it and seeing it was close to like 40% or 50% at one point,” Gareth Rhodes, a member of the governor’s coronavirus task force, said in an interview. “But that was at a time when testing capacity was very limited.” Eight months later, states like South Dakota now find themselves in a similar quandary as the pandemic surges to record levels nationwide.

The metric is now a key part of the state’s new “microcluster” strategy, which is designed to avoid the need to reimpose sweeping lockdowns seen in March and April. “We have invested in the infrastructure that’s actually needed to do this protracted game of ‘Whack-a-Mole,’” Madad said. “Much more preferred than massive shelter in place.” She added that the state is generally in pretty good shape as far as deadly public health emergencies go, but there are some warning signs. 

The latest state data shows that several regions of the state are now reporting alarming increases in the number of new cases per 100,000. The Central New York and Mid-Hudson regions are reporting 10.06 and 11.76, respectively, on a seven-day rolling average. The Southern Tier is reporting 28.89 – quite a contrast to the 1.3% positivity rate. Residents of any state with a rate above 10 are subject to a two-week quarantine upon arrival in New York, though the governor has said that restriction will not apply to people from New Jersey, Massachusets – or individual regions of New York, for that matter.

So while testing positivity suggests all regions of the state are doing pretty well, the rising rate of new cases per 100,000 paints a much darker picture of what the coming weeks could bring, according to an analysis by the Empire Center for Public Policy. “The disconnect between the two measures appears to be driven by a significant increase in testing over the past month, much of it conducted by colleges and universities as part of their routine screening of students and staff,” Bill Hammond, a senior fellow for health policy at the center, wrote in an Oct. 27 blog post.

In other words, testing people who are disproportionately unlikely to have the virus is depressing the total positivity rate at a time when cases are rising to their highest rates relative to population since May. That is one reason why statewide hospitalizations have doubled in recent weeks, though they remain just a fraction of what they were in New York in the spring. “Low positivity rates are a sign that New York is doing enough testing to keep good track of the pandemic’s progress,” Hammond added. “But it’s also important to pay attention to the results of that testing – and they say that New York’s pandemic is currently getting worse.”

The Cuomo administration has pushed back at suggestions that it is paying too much attention to the positivity rate. “If you go on our (online) dashboards, we present all of it in terms of all these different things that each have their own uses,” Rhodes said. While new cases per 100,000 people is among the metrics used by the state, the mental math can get a little complicated. “If you look at TV on a daily basis, it’s the number of new cases, and it’s the positivity, right?” he added.

A third statistic illustrates how the realities of COVID-19 can impact otherwise objective-looking numbers. The R0 (pronounced “R-naught) measures how many people the average COVID-positive person infects. This is currently hovering just above 1 in New York, which suggests that the pandemic is growing slightly rather than shrinking. The big problem with this measure is the fact that just a small fraction of infected people cause somewhere around 80% of cases. “If Amazon’s CEO, Jeff Bezos, walks into a bar with 100 regular people in it, the average wealth in that bar suddenly exceeds $1 billion,” The Atlantic recently wrote about the pitfalls of R0.

There is a common problem with many of the statistics we currently use to assess the pandemic, according to Columbia University epidemiologist Wafaa El-Sadr. “What you want to do is ... a random sample of the population,” she said in an interview. “And then you go out and you test the selected individuals, and that way, you have a real snapshot as to who is infected.” To that end, an upcoming study by Columbia researchers will survey thousands of people in New York and more than a dozen other communities across the country for the coronavirus and its antibodies. 

The study will not be a truly randomized sample because participants will be the people who happen to come by testing sites located at places like parks and shopping centers. Yet, it strikes a balance between finding solid data and getting results out relatively quickly to help scientists and policymakers. “It’s a very intricate process,” El-Sadr said about randomized studies. “You have to do a lot of preparatory kind of work to make sure that people are really eager to participate if they are selected.” That takes time and money, while a venue-based approach could get results out to the public sometime in early 2021, according to El-Sadr.

Before the study sheds new light on how the coronavirus has spread based on geography, age, race and income, COVID truth-seekers are stuck with the data at hand. A comprehensive review provides a relatively nuanced snapshot of the state of the pandemic, especially compared to the limited understanding from months ago. Yet, how the numbers are sources is arguably just as important as what they ostensibly say. “A sample study is no better than the sample it is based on,” former tobacco lobbyist Darrell Huff writes in “How To Lie With Statistics.” If casual pandemic-watchers aren’t careful, they, too, can be misled by people who have their own reasons to claim that things are better – or worse – depending on the political moment. 

NEXT STORY: Candidates’ ad hoc mask approach

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.