The Dual Views of Risk Data
Most people have difficulty understanding what different levels of risks mean to them and translating them into decisions they make in everyday life.
Those that stand to gain from this public misinterpretation of the significance of various risks know this full well, and manipulate the way in which statistics are presented to further their ends.
Pharmaceutical products are often sold and marketed using relative risk reduction (“RRR”) rather than absolute risk reduction (“ARR”). This is in contravention of various guidelines which suggest they ought not to do this; see for example the Association of the British Pharmaceutical industry’s code of practice. The medicines regulator, the MHRA, is also clear in its guidance that companies must not mislead in matters of risk. As with many other declarations, rights and regulations written by sober experienced people during “normal” times, it is frightening to see how easily these were ignored in the heat of the “emergency.”
What is the difference between ARR and RRR?
Imagine that without a certain treatment you have a 5 in 1000 chance of dying from a disease. That’s 1 in 200. But with a new treatment, it is reduced to 1 in 1000.
Expressing this as “the treatment reduced my risk by 80% (5 reduced to 1 in every 1000) would be the RRR.
That sounds impressive, doesn’t it. Who wouldn’t want that?
But this metric does not convey a sense of what the background risk of dying was in the first place, and how low it was. To incorporate that, we look at the ARR.
In this case, the ARR was a reduction from 0.5% (5 in 1000) to 0.1% (1 in 1000), or 0.4%.
Perhaps more simply, the calculation can also be done thus: the reduction was from 5 in 1000 to 1 in 1000, so it’s a reduction of 4 in 1000, or 0.4%
So we can see that the same data can create the claim that the drug reduced mortality by 80% or by 0.4%. The former does not consider background risk, and the latter does. Of course, considering background risk would be an important thing to include in the decision-making process, especially if the drug had known side-effects, or a novel mode of action which meant that it could well have unknown longer-term effects.
Applying the above to a real example, the Pfizer/BioNTech covid vaccine
Following the original Phase 3 trial, a 95% efficacy was touted. Ignoring issues around the veracity of the data, relevance of the measures and so on, we can see that this originated from a reduction in the number of “covid infections” from 162 to 8, which is indeed a 95% reduction
As an aside, when the test results were ignored in favour of symptoms alone the figure fell to only 19%, but for the purposes of this piece, let’s run with their headline figure of 95%.
95% is the RRR. What about the ARR – which takes account of the fact that there were approximately 22,000 subjects in each of the 2 treatment groups (active and placebo).?
As pointed out in the pinned tweet of one of Hart’s co-chairs – the ARR is less than 1%.
Observant readers will note that this is in any case referring to a reduction in the number of PCR positives with symptoms. Nearly all these people had a mild cold-like or flu-like illness. Even if the numbers themselves are trustworthy, they’re of questionable relevance, since the only metrics which really matter are surely serious illness or death. The ARR numbers for serious illness and death are of course even smaller still (again with the caveat that they shouldn’t necessarily be taken at face value anyway).
Authors happily use ARR when it suits them
In 1997, a randomised trial was conducted in 20 centres across the UK comparing angioplasty for “stable” angina with drug treatment alone.
You can read more about the trial here.
The point of mentioning this is to show that when it suits them, authors will refer to ARR rather than RRR.
In this trial, the risk of death or a heart attack was ~3% in the group treated with drugs, but ~6% in those treated with angioplasty.
Did they describe this as a doubling (which would be the additional relative risk)?
Nope, or course not. It was referred to as “a small increase”.
The “small increase” is referring to the 3% – the absolute risk increase; it’s actually debatable if 3% can credibly be described as “small”, though that isn’t the point we are making here.
But imagine if, instead of increasing the incidence of death or heart attack from 3% to 6%, the angioplasty had reduced it from 3% to 1.5%.
What do you think this would have been described as – a small reduction?
We don’t think so. The study would have been trumpeted with the headline of:
“Angioplasty halves death / heart attack risk compared to drug treatment alone”
We are not, by the way, saying that it is always right to refer to ARR over RRR or vice-versa. It depends on the circumstances, notably the thing being averted, its consequences, and the additional risks of the intervention.
In practice, a treatment decision should involve a discussion of both aspects – the risk reduction from the intervention AND a semblance of background risk. It is only by considering both aspects that the significance of known and potential downsides to the intervention can be meaningfully considered by the intervention. Discussion of background risk is especially important if this varies widely between different group eg for Covid the background risk is around 500 x higher for an 80 year old than an 8 year old, so the absolute risk reduction for a child given a covid jab is effectively zero.
In our view, therefore, that the failure to discuss background covid risk before the administration of the covid injections (as well as the failure to disclose many other matters) nullified any consent given, as it is established law that for consent to be valid it must be “fully informed”.
