Vaccine Efficacy and the Number Needed to Treat

HART Editor COVID Data

What does 95% efficacy actually mean? 

When you hear that a vaccine has 95% efficacy, what do you think that means?

Most people think that it means that, if 100 people are vaccinated against a virus, 95 won’t get that virus. That’s not what it means and the truth might surprise you.

In November 2020, Pfizer became the first pharmaceutical company to report some early results from its vaccine. Pfizer announced that its vaccine “was found to be more than 90% effective in preventing COVID-19.”[1] COVID-19 was defined as a positive PCR test and at least one symptom. 

The vast majority of participants didn’t get Covid-19

Pfizer first released data in a press release and then followed up with an academic paper with slightly different figures. The press release figures reported that 43,444 out of 43,538 participants – that’s 99.8% of people – did not get Covid-19.

The academic paper figures reported that 36,353 out of 36,523 participants – that’s 99.5% of people – did not get Covid-19.

The tiny number of people who tested positive is striking. In May 2020, there were discussions about the ethics of whether or not vaccine volunteers should be deliberately infected with the virus because so few people were catching it naturally, and vaccine trials were at risk of not having enough cases to study.[3]

What did Pfizer say?

The press release reported that 43,538 participants had been enrolled and that 94 cases of Covid-19 had been confirmed among the participants (9 cases in the vaccine arm and 85 in the placebo arm).

On December 10th, 2020, an academic paper updated the press release numbers to report that there were 8 cases among 18,198 people who received the vaccine and 162 cases among 18,325 people who received the placebo, which the paper reported as 95% efficacy.[2] (Nine cases in the drug arm had strangely become eight and 43,538 participants reduced to 36,523, for various reasons, but let’s go with it).

Pfizer sold the drug on the basis of its efficacy or the percentage reduction in cases in the vaccine group compared with the placebo group. Efficacy is calculated by the reduction between cases only. The size of the trial does not matter. The size of the trial only matters when we get to numbers needed to treat (NNT). The figure of 90% efficacy (from the press release) is calculated as the percentage reduction between the cases i.e., 85 minus 9 divided by 85 = 90%.

The figure of 95% efficacy (from the subsequent paper) is calculated as the difference between the cases i.e., 162 minus 8 divided by 162 = 95%.

This is referred to as a “relative risk” measure. It is a measure that is relevant to the whole population.

Number needed to treat (NNT)

What matters most to a patient is knowing how much protection the vaccine provides them as an individual; this must incorporate the notion of what the background risk to them is of actually getting the disease should they choose not to be vaccinated.. The number needed to treat (NNT) is the number of patients, on average, who would need to receive experimental treatment (as opposed to control treatment) for one additional patient to not have the study outcome.[4] The NNT is calculated from the absolute risk difference. This is where the size of the trial gets taken into account.

When 90% efficacy was reported, there were 21,769 people in both the intervention and control groups. The 9 cases in 21,769 people gives a case rate of 0.04%. The 85 cases in 21,769 people gives a case rate of 0.39%. The absolute difference between these two numbers is 0.39% minus 0.04% = 0.35%. The NNT is calculated as 100 divided by the absolute difference, i.e., 100 divided by 0.35% = 286. At the time of the first Pfizer press release, it was known that 286 people would need to be treated for 1 person to avoid being a Covid-19 case (of any severity at all, and most cases were in fact mild, not requiring hospital treatment). This was known, but it was not widely known.

When 95% efficacy was reported, in the later paper, the additional cases in the placebo group and the reduced participant numbers used overall (between the first announcement and the first paper) meant that the absolute difference increased to 0.84%. This meant that the NNT reduced to 100 divided by 0.84 = 119.

Perception vs reality

Virtually every person will think that 90% vaccine efficacy means that 90% of people vaccinated won’t get the disease they were vaccinated against. The fact is that – in the trials of most interest and relevance today – 99% of people who were given the placebo didn’t get Covid at all. The average person won’t realise that the data we started with meant that more than 100 people needed to be vaccinated to avoid 1 case. The average person also won’t realise that we ‘went into bat’, with the goal of vaccinating billions of people on the back of a couple of hundred PCR tests: the vast majority of trial participants did not have their symptoms confirmed or ruled on by any sort of testing or doctor’s diagnosis.

References

Please follow and like us:
Facebook
Facebook
fb-share-icon
Twitter
Visit Us
Tweet