How Confident is the Government in its ‘Evidence’ on Masks

It would seem that Nadhim Zahawi’s most recent promised ‘material’ evidence for his recommendations for face coverings in secondary classrooms is as flimsy as some of the cloth masks our teenagers will need to resort to using, as they do their best to cope with the challenges of learning in 2022.

Dr Val Fraser, retired Lecturer in Teacher Education, Subject Expert for Ofqual and former OFSTED School Inspector, puts the last UK Government’s mask missive under the linguistic microscope:

What is the “material evidence” Nadhim Zahawi, Education Secretary speaks of (TalkRadio Monday 3^rd January 2022) for recommending face coverings to be worn in secondary school classrooms and, more importantly, how convincing is it? The government document entitled Evidence Summary: Coronavirus (COVID-19) and the use of face coverings in education settings needs an understanding of ‘modality’ to help evaluate how robust this evidence is.

Modality is a term used in the study of grammar and linguistics to signal certainty.  Verbs qualified with modal verbs suggest whether an event or a claim is possible, probable, likely or certain.  The principal auxiliary modal verbs when placed on a continuum from possible to certain show this range: can, could, may, might, should, would, shall, must and will.

“Manchester United can win the league” is a hedging statement suggesting some caveats to be considered.  However, “Manchester United will win the league” is a definite statement of certainty and expectation. Advertisements make heavy use of modal verbs to sell their products without making claims that leave them open to legal difficulties. ‘Wrinkles can be reduced by up to 50%’ is a possibility of smoother skin that sells the product without over-promising.

Modality may also be conveyed by the use of adverbs. The famous example of “Probably the best lager in the world” steers Carlsberg away from litigation, whilst selling its product as a high quality one – “the best” is what resonates. Other adverbs making clear possibility, obligation and emphasis are: generally, maybe, perhaps, possibly, probably, promisingly, obviously, certainly, clearly and definitely. Again the range from least to most certain shows a continuum of expectation.

A document that is succinctly entitled Evidence Summary is a bold statement: the reader would expect to see certainty of claims, anchored in a secure evidence base and/or data providing concluding proof. However, an examination of the use of language in this particular document reveals a distinct hedging when it comes to the claims being made, in this case an attempt to underpin the government’s policy decision to recommend face coverings for secondary school classrooms.

Below are examples of how the document is using modality to avoid claiming any certainty for its evidence base:

• ‘Face coverings can contribute to reducing transmission’. This is a general statement about the possibility (but not certainty) of masks helping to reduce viral spread.  There are two qualifiers in that clause: one is ‘can’: the author does not want to make a definite claim; the other is ‘contribute’: there are no claims that in and of itself masking is going to achieve a positive outcome.  This is an introductory comment and sets the tone for hedging, cautious claims and caveats. The same statement opens the main body of the text.
• The reader is informed that the mode of transmission of the virus can be via droplets, aerosol particles and by contact. It is curious that, two years into the science studying the virus, that ‘can’ needed to be added.  A more definite statement such as ‘transmission occurs through’ would convey a more authoritative stance. Note again that possibility is being claimed not certainty. There are 17 uses of the modal verb ‘can’ revealing that this evidence submitted is peppered with a significant level of uncertainty and hedging of claims.
• Could is used nine times. An example of this is, ‘Using a different maximum weighting threshold could result in slightly different results’. This is an alarming disclaimer for the validity of the claims provided as evidence.  ‘Could’ like ‘can’ distances the author from taking responsibility for a definite view or position.
• We are further informed that masks ‘may further reduce risks of longer-range airborne transmission’. The term ‘may’ also indicates a possible but not a certain effect. There are 15 uses of the term ‘may’.
• There is even less certainty in the document concerning how the Omicron variant is transmitted.  We are told it might show more airborne transmission (the reason for recommending masks now). When ‘might’ is used it is indicating guesswork.  The author is saying we simply don’t know and we have to signal that.

Modality and uncertainty are also conveyed through the use of adverbs as indicated above. An example is contained in this sentence: (researchers) ‘could explore expanding the time-period under study to potentially yield more precise estimates’.  Potentially is another term which pulls back from providing a more assertive claim for an outcome.  Moreover, this is only one of the three examples of the limitations of the evidence in that sentence: ‘could’ is used as prevaricator avoiding being drawn into a commitment to obtaining more concrete data (for the precise estimates – which in themselves, as estimates, are predictive not determined).

There are 42 uses of modal verbs and 18 uses of adverbs on the low certainty spectrum (as explained above). Why is the government presenting its findings in a tenuous and circumspect manner? Modality of language can be tracked in the methodology and findings of its ‘research’ but, more importantly, we can see the limitations of the research itself, which obliges the authors to also limit the claims they can present as evidence.

We learn from the research design that:

• To evaluate the efficacy of face masks in schools they examined attendance rates, with no compelling rationale for this perceived correlation being offered.
• The data collection period was from two separated out weeks in October 2021 which included some missing data.
• They candidly state that it is a ‘preliminary, experimental analysis, which would benefit from robust external peer review to a longer timescale’.
• They further cast doubt on their findings when they acknowledge that the results may not have any statistical significance as the differential is within a chance outcome.
• They did not isolate the variables to be sure that face coverings were the determining factor in lowering absence rates. Further they state the study did not draw data for long enough time periods and different methodologies would have yielded different results.
• The schools categorised as mask wearing ones were not a homogenous group in terms of their defined use.  Some used them only for communal areas and some for classroom use too but they were not differentiated for that within the categorisation.
• Other variables such as Local Authority guidance and implementation and local rates of cases and infection were not considered.
• The raw results showed that non-masking schools had a significantly lower absence rate and it was only after modelling that a positive outcome was found. The authors concede that using different assumptions for this modelling, different “weighting thresholds”, could result in different results.
• They advise that a more robust study would go onto consider community COVID-19 case rates, regional data (LA, information on LA wider response to COVID-19, etc), other characteristics of pupils (proportion of pupils with SEND, etc) and any information on differential use of face coverings and would offer more reassurance about the validity of this evidence than they can currently provide.
• They found that absence rates in the control group (unmasked) remain lower overall than those in the treatment group (masked). This is a surprising admission towards the end of the report.
• The researchers consulted other studies.  This research method would normally give more validity to the findings, in terms of the triangulation of data with their own.  However, they had to acknowledge that the results from those were inconclusive, ‘mixed’ and the majority were observational studies, with only 2 RCTs, neither involving schools.
• No data was available on Omicron: the variant of the virus for which the recommendations were being brought in to address.

The qualifications and caveats above reveal the report is at best a tentative proposal, which has not been subject to the usual quality assurance procedures before publication. The research design points to an insecure hypothesis between mask wearing and attendance rates which was neither explained, tested beforehand nor validated after. The methodologies did not keep the variables stable and therefore did not isolate the variable (masks) they were expecting to be able to analyse and base the claims upon. The results did not provide a secure evidence base to form a compelling case for recommending face coverings.

With these limitations in the research study, a reader would expect to see, as indeed is clear, a report sewn together with tenuous arguments, circumspect claims and qualified results and recommendations. The only way to compose such a report is prolific use of modal verbs and adverbs as indicated above.

Yet the harms of wearing face coverings in educational settings are openly stated in the report and couched in more definite measurable claims and certainty of language:

• 80% of pupils reported that wearing a face covering made it difficult to communicate, and 55% felt wearing one made learning more difficult.
• Wearing face coverings may have physical side effects and impair face identification, verbal and non-verbal communication between teacher and learner.
• Almost all secondary leaders and teachers (94%) thought that wearing face coverings has made communication between teachers and students more difficult, with 59% saying it has made it a lot more difficult.
• Research into the effect of mask wearing on communication has found that concealing a speaker’s lips led to lower performance, lower confidence scores, and increased perceived effort on the part of the listener.
• Meta-cognitive monitoring was worse when listening in these conditions compared with listening to an unmasked talker.
• A survey of impacts on communication with mask wearing ….reported that face coverings negatively impact hearing, understanding, engagement, and feelings of connection with the speaker.
• People with hearing loss were impacted more than those without hearing loss. The inability to see facial expressions and to read lips have a major impact on speech understanding for those with hearing impairments.
• The WHO reports that “the wearing of masks by children with hearing loss or auditory problems may present learning barriers and further challenges”.

Note the more certain arguments (some with precise percentages attached) in the above for the harms of mask wearing and especially for children. There are far fewer modal verbs used and the claims are, in the main, unambiguous: ‘were impacted’, ‘negatively impact’, ‘was worse’, ‘led to’. ‘made worse’, ‘more difficult’. The evidence for the harms of face coverings is measurable, precise, unambiguous and certain and the language used for presenting the evidence base, is equally unequivocal.

It would seem that Nadhim Zahawi’s promised ‘material’ evidence for his recommendations for face coverings in secondary classrooms is as flimsy as some of the cloth masks our teenagers will need to resort to using, as they do their best to cope with the challenges of learning in 2022.In conclusion, perhaps we should ponder on the one piece of data expressed as a precise statistic, which might be driving this new guidance, namely: ‘71% of UNISON support staff thought face coverings in schools were an important safety measure’. If our Education Secretary has sacrificed children’s learning and social communication opportunities in schools, to appease Trade Unions, he will have to provide much more compelling evidence that schools are in any way unsafe for children or staff than he currently has. He has stiff opposition in the form of 150 comparative studies, peer reviewed with robust research, which come to the very definite and certain conclusion that, “to date, the evidence has been stable and clear that masks do not work to control the virus”. There is not a whisper of modality in that concluding statement either.