Can we rely on NIMS (National Immunisation Management Service) figures?
To understand case, hospitalisations and death rates in the vaccinated and unvaccinated populations it is essential to know how many people there are in each. Measuring a population is hard, as demonstrated by the recent dramatic shifts in the numbers within the NIMS database. The National Immunisation Management Service is a database maintained by UKHSA which counts those people resident in the UK and registered with a GP and matches them to the vaccination record. UKHSA are under pressure to keep a huge complex database as accurate as possible, while compiling different data sources and with weekly publications of their results. Before drawing conclusions using NIMS data it is important to understand its flaws and how that impacts those conclusions. This article explains what NIMS are, and outlines its weaknesses, details the changes in the numbers over time and then examines the implications of those changes.
Over 850,000 under 60 year olds were added up to 1st October. Over the same period, 15% of the unvaccinated over 60 year old population were removed without being vaccinated. A proportion of these may have been vaccinated and added to that cohort but in the meantime other vaccinated people had died and been removed. 1 in 20 people who were newly vaccinated were added to the vaccinated cohort without reducing the numbers in the unvaccinated cohort. This implies they were not previously registered in the system. The proportion who have been added in this way has risen significantly. The implication is that significant numbers of the previously unregistered population are counted towards the vaccinated total but they are omitted from the unvaccinated total. Because NIMS is a measurement not an estimate, it is not possible to acknowledge that there is a problem with undercounting of the unvaccinated community to rectify this bias.
What is NIMS and what are its weaknesses?
The main weakness of NIMS is its reliance on measuring those with an NHS number. The unvaccinated numbers are calculated by subtracting the vaccinated numbers from the total population, which for NIMS is measured as the total number of people registered with a GP and resident. The population who are not registered with a GP cannot be included. Entries for people who have left the country and duplicate records may inflate numbers, however, regular cleaning of the database should minimise such errors.
As vaccination rollout progresses, people who were not registered with a GP have had to become registered to receive their vaccinations. The population of the young in particular, has therefore increased significantly as vaccination continues. Even while more and more previously unregistered people are counted in the vaccinated cohort, the unregistered, unvaccinated population will never be counted and the total for the unvaccinated will always be smaller than reality.
How have the numbers in NIMS changed?
Since April 2021, there have been large additions to the total population number, predominantly from previously unregistered people becoming vaccinated. The total number of people who have added to the vaccinated total without causing a reduction in the unvaccinated total is 850,000. In October alone, there were 450,000 additional people, over the age of 30, with a recent first dose whose addition to the database caused no reduction in the unvaccinated numbers.
In September, for every additional person with a first dose added under the age of 60, only 70% of them resulted in a reduction in the number of unvaccinated people. The remainder were people not previously registered on NIMS. Between 16th April and 1st October 850,000 people under 60 were added to the database in total. During October, 260,00 under 30 year olds were removed.
|From 16th April to 1st October||During October|
|INCREASED||850,000 under 60 yr old newly vaccinated people that were not removed from unvaccinated cohort||270,000 vaccinated over 60 year olds|
28,000 unvaccinated over 60 year olds
|DECREASED||160,000 total over 65 yr oldsIn the unvaccinated cohort who were not vaccinated|
100,000 vaccinated over 75 year olds
|260,000 unvaccinated people under 30 including 250,000 under 12 year olds|
62,000 16 and 17 year olds who had been recorded as having had a second dose
Since 1st October the NIMS database has changed in ways that are harder to understand. From 8th October to 22nd October, 134,000 people who had been registered as having had a second dose were removed from the database, including 93,000 16 and 17 year olds. 712,000 under 30 year olds were removed from the unvaccinated population during October but only 450,000 were newly vaccinated. It is important to note that this shift changes the size of the unvaccinated population by less than 2%.
The changes are more stark for the over 60s. In the period between 16th April and 1st October, 160,000 over 60 year olds were removed from the unvaccinated population without being vaccinated, accounting for 15% of the unvaccinated population. 23,000 of these unvaccinated people were then added back in over the course of October.
Between April and the end of September, only 28,000 over 60 year olds were vaccinated for the first time. However, in October an astounding 270,000 over 60 year olds were apparently vaccinated for the first time. Perhaps the latest campaigns were more motivating than the five months of campaigns that preceded them. Perhaps someone adjusted the settings so numerous people were simultaneously moved up to the over 60s cohort having had a birthday. Or perhaps people attending pop up vaccination centres without ID or NHS registration would still have received a vaccine and may only lately have been identified within the system as people have returned for boosters.
These are cohorts of people, so even with people dying the cohort should remain steady although there will be movement when people die or have their birthday and the totals may change. As people over 60 die, they should be replaced by people having their 60th birthday. In fact, as the baby boomers age, this cohort should be increasing, not shrinking. Overall the over 60 population had shrunk by 160,000 in the NIMS database from April to end of September, but the additional vaccinated people registered in October meant that by the end of October the overall change since April was a rise of 140,000. Has there been a clear out of duplicate records? If so, why were there so many more in the older age groups?
What are the implications of these changes?
Finding 270,000 over 60 year olds to vaccinate who weren’t previously registered is a significant finding. The whole population of unvaccinated over 60s is estimated at under 1 million. The implication is, therefore, that prior to registering these people the unvaccinated population was significantly underestimated.
Changes in the NIMS measurement over time have an inherent bias. While the people being vaccinated were largely those who already had an NHS number then the comparison between vaccinated and unvaccinated would have been stable. However, as more people come forward for vaccination who were never registered, the vaccinated denominator will increase. At the same time, the unvaccinated denominator fails to record those unregistered people who are not coming forward for vaccination. Therefore they are no longer measuring the same thing. When reaching easy to reach groups for vaccination, both measures were of the number of people vaccinated and unvaccinated in the registered community. Now the vaccinated measure extends into the previously unregistered community but the unvaccinated measure does not.
The importance of the numbers is that they are used to calculate the rates of cases, hospitalisations and deaths in the vaccinated and unvaccinated populations. It is very hard to measure the population of a country. ONS attempts it through censuses and surveys. PHE attempts it through counting records of individual people. There are problems with both methods. The PHE measurement is bigger than the ONS estimate, which perhaps is not surprising. The incentive to become registered with the authorities in order to receive free healthcare must surely be greater than the one to register with no direct benefit.
Since September, the period since which vaccine surveillance reports have been available and reporting case rates, the number of unvaccinated people over 50 years of age has increased by 19,000. The number under 40 has fallen by 50,000 excluding those who moved into the vaccinated cohort by getting vaccinated.
Figure 3 shows the estimated case rates based on the UKHSA calculation of the unvaccinated population and based on estimates derived variation in that measurement by 3% in either direction. An over measurement of the total population size of over 60 year olds, e.g. from duplicate records of 3%, would give a case rate identical to the double dosed. There will be uncertainty within the NIMS figures and it is not unreasonable to think that may account for a few percent of the totals. Changing the total population estimate results in large swings in the estimate of ‘case’ rates in the unvaccinated. Given the inability to be accurate within a few percentage points, and the uncertainty about whether the inaccuracies are overestimating or underestimating the total, the differences between populations are not as informative as the trend in rates within each of the populations.
There are numerous reasons to doubt the accuracy of any estimate of the size of the unvaccinated population. However, attempts are clearly being made to increase accuracy. When using the NIMS data it is important to consider how it has changed over time and how its bias towards underestimating the unvaccinated compared to the vaccinated may also have changed. NIMS is not perfect but it is the best measurement of the size of the population (as opposed to the ONS which is a modelled estimate not a measurement). It is not reasonable to draw dramatic conclusions of increased case rates in the double dosed vaccinated population while a mere 3% error margin would lead to the rates being identical. However, by any measurement a clearly greater increase has been seen in the vaccinated compared to the unvaccinated and this trend is what causes concern.