Top tips on ensuring vaccine success (and for other treatment failure)
There are ways to ensure any trial will give you the results you want. Here are the top six methods used to manipulate covid research results.
1. Carry out the study when cases are falling
A favourite trick that has been used repeatedly throughout the pandemic.
Vaccination is a process that takes time. At the beginning of the study all participants are enrolled as unvaccinated. People move into the vaccinated cohort over time. The result is that the unvaccinated are exposed at periods of higher prevalence and for a longer time. The vaccinated are only exposed from a period later on and further down the curve. The vaccinated will therefore be less likely to catch covid and the vaccine can be made to look like it worked.
The ONS recently deployed this trick to claim a 32 fold lower mortality among the vaccinated. It was extreme data manipulation whereby they included the majority of deaths in winter prior to vaccination instead of starting from spring when a fairer comparison could have been made. The Office for Statistics Regulation has upheld a complaint about the ONS’s manipulation of data in this case.
2. Don’t include meaningful outcome measures like death
The vaccine trials have not demonstrated prevention of death.
Death from covid was too uncommon for the trials to be able to show an effect. Among 44,000 participants in the Pfizer trial, for example, there were a total of
29 deaths before the placebo group were themselves vaccinated. 15 of these were in the vaccinated including one of the three covid deaths. There were only 20 severe covid cases reported in the Pfizer submission to the FDA for Emergency Use Authorisation, including those that did not need hospital care.
Instead of using outcomes which are hard to distort, like death, the outcomes used were based on test results.
3. Only measure outcomes for part of the time after vaccination
Vaccinated people are at increased risk of covid infection in the first two weeks after vaccination. After the first two weeks the susceptible will have acquired natural immunity. By ignoring the first two weeks, a distorted picture of the overall impact of vaccinations can be presented. If all the vaccines are doing is causing the susceptible to have their infections earlier then the whole period from the date of vaccination needs to be included in assessing any benefit.
4. Use modelled data
The most reliable scientific papers ensure that the population being studied is representative of the population as a whole. Sometimes that is hard to achieve and adjustments need to be made afterwards to account for any differences e.g. the sample may be younger than the population as a whole. These adjustments are a reasonable thing to do to correct for small problems in the sample.
However, if the sample is so different to the general population that massive adjustments need to be made then effectively the study is no longer based on real world evidence. Instead it has become yet another prediction based on modelled data.
For example, this CDC study claimed, contrary to all other evidence, that people with a prior infection were 5 times more likely to become infected than those who had been double vaccinated. The actual results from the study showed a 70% higher rate but their adjustments made this into 500%. There were other serious flaws in the study which makes even the 70% claim highly dubious.
5. Ignore a crucial subgroup
If the findings are equivocal in most of the population then modelling might be enough to get the result you want. However, if there is a subgroup which has clearly contrary results then they can be left out entirely. When trying to calculate the impact of vaccination on transmission in winter the most obvious group to study would have been the heavily vaccinated care home residents. PHE omitted this group from their paper with no reason given. The consequence was that the overall household transmission rate was much higher in the real world (see figure 1) than in the study. Using published data on mean national transmission rates each week the expected rates in the study would have been 12.4% in the unvaccinated (who were overrepresented when transmission rates were highest) and 11.7% in the vaccinated cohorts. However, the study reported a rate of only 10.1% in the unvaccinated and only 6% in the vaccinated. The population excluded from the PHE study clearly contributed to the much higher real world transmission rates for both the unvaccinated and the vaccinated.
6. Use the wrong dose of the drug
A drug can be discredited in a trial where the protocol is deliberately designed to give a drug at an inappropriate time or dose. For example, the RECOVERY trial was said to show that hydroxychloroquine treatment was not helpful in covid. The trial design used a dose of hydroxychloroquine that was in the toxic range and may well have been responsible for the deaths of participants in the treatment arm of the trial. Hydroxychloroquine doses over 1500mg are associated with cardiovascular, neurological, and can be potentially fatal. The trial used a dose of 2000mg in the first 18 hours and 400mg every 12 hours thereafter.
Scientists are human and are under pressure to produce results in their papers as that increases the chances of publication which is needed for career progression and securing funding for research. There are numerous ways in which data can be altered to tell a story and many of them have been put to use during the pandemic. Dramatic results have become headline grabbing news but often the truth was far more mundane. If a result sounds too good to be true, don’t believe it.