Potential Biases in Estimating Absolute and Relative Case-Fatality Risks during Outbreaks

Estimating the case-fatality risk (CFR)—the probability that a person dies from an infection given that they are a case—is a high priority in epidemiologic investigation of newly emerging infectious diseases and sometimes in new outbreaks of known infectious diseases. The data available to estimate the overall CFR are often gathered for other purposes (e.g., surveillance) in challenging circumstances. We describe two forms of bias that may affect the estimation of the overall CFR—preferential ascertainment of severe cases and bias from reporting delays—and review solutions that have been proposed and implemented in past epidemics. Also of interest is the estimation of the causal impact of specific interventions (e.g., hospitalization, or hospitalization at a particular hospital) on survival, which can be estimated as a relative CFR for two or more groups. When observational data are used for this purpose, three more sources of bias may arise: confounding, survivorship bias, and selection due to preferential inclusion in surveillance datasets of those who are hospitalized and/or die. We illustrate these biases and caution against causal interpretation of differential CFR among those receiving different interventions in observational datasets. Again, we discuss ways to reduce these biases, particularly by estimating outcomes in smaller but more systematically defined cohorts ascertained before the onset of symptoms, such as those identified by forward contact tracing. Finally, we discuss the circumstances in which these biases may affect non-causal interpretation of risk factors for death among cases.The case-fatality risk (CFR) is a key quantity in characterizing new infectious agents and new outbreaks of known agents. The CFR can be defined as the probability that a case dies from the infection. Several variations of the definition of “case” are used for different infections, as discussed in Box 1. Under all these definitions, the CFR characterizes the severity of an infection and is useful for planning and determining the intensity of a response to an outbreak [1,2]. Moreover, the CFR may be compared between cases who do and do not receive particular treatments as a way of trying to estimate the causal impact of these treatments on survival. Such causal inference might ideally be done in a randomized trial in which individuals are randomly assigned to treatments, but this is often not possible during an outbreak for logistical, ethical, and other reasons [3]. Therefore, observational estimates of CFR under different treatment conditions may be the only available means to assess the impact of various treatments.

Box 1. Definition of the CFR.

The CFR itself is an ambiguous term, as its definition and value depend on what qualifies an individual to be a “case.” Several different precise definitions of CFR have been used in practice, as have several imprecise ones. The infection-fatality risk (sometimes written IFR) defines a case as a person who has shown evidence of infection, either by clinical detection of the pathogen or by seroconversion or other immune response. Such individuals may or may not be symptomatic, though asymptomatic ones may go undetected. The symptomatic case-fatality risk (sCFR) defines a case as someone who is infected and shows certain symptoms. Infection in many outbreaks is given several gradations, including confirmed (definitive laboratory confirmation), probable (high degree of suspicion, by various clinical and epidemiologic criteria, without laboratory confirmation), and possible or suspected (lower degree of suspicion). This paper describes issues in estimating any of these risks or comparing them across groups, but does not go into the details of each possible definition.Furthermore, unlike risks commonly used in epidemiologic research (e.g., the 5-year mortality risk), the length of the period during which deaths are counted for the CFR is rarely explicit, probably because it is considered to be short enough to avoid ambiguity in the definition of CFR. However, a precise definition of the CFR would need to include the risk period, e.g., the 1-month CFR of Ebola. Clearly, the definition of CFR for a particular investigation should be specified as precisely as possible.However, observational studies conducted in the early phases of an outbreak, when public health authorities are appropriately concentrating on crisis response and not on rigorous study design, are challenging. A common problem is that disease severity of the cases recorded in a surveillance database will differ, perhaps substantially, from that of all cases in the population. This issue has arisen in the present epidemic of Ebola virus disease in West Africa and in many previous outbreaks and epidemics [4–9] and will continue to arise in future ones.Here we outline two biases that may occur when estimating the CFR in a population from a surveillance database, and three more biases that may occur when comparing the CFR between subgroups to estimate the causal effect of medical interventions. We also briefly consider the applicability of these biases to a different application: comparing the CFR across different groups of people, for example, by geography, sex, age, comorbidities, and other “unchangeable” risk factors. Such factors are “unchangeable” in the sense that they are not candidates for intervention in the setting of the outbreak, though some could, of course, change over longer timescales. The goal of estimating the CFR in groups defined by such unchangeable factors is not to understand the causal role of these factors in mortality, but to develop a predictive model for mortality that might be used to improve prognostic accuracy or identify disparities. Such predictions may be affected by survivorship bias and selection bias, but not by confounding, as we discuss.