Masks and evidence

Zeynep Tufekci, one of today’s better science journalists, provides a good explanation of the Cochrane paper on masks. The latter has generated far more press than it deserves, with many writers who don’t understand science commonly reaching some wrong conclusions.

The first wrong conclusion is that it significantly changed the evidence on masking. It didn’t. It was a metastudy. The authors did no primary research, and generated no new primary data. Metastudies are nice. The best ones provide insights into a large pile of studies, whose overall signal is made more clear. Still, the primary data after a metastudy is the same as it was before. If that metastudy caused a sea change in someone’s views on masks, it’s likely because their views weren’t much connected to the evidence.

Or, because they aren’t adept at reading studies. The second and related wrong conclusion is that the Cochrane paper says more about whether masking works than it does about how hazy the current evidence is. Some people develop an acute sense for the qualities of evidence as a dimension separate from what answers the current evidence suggests. It seems most never do.

There are practical reasons it’s hard to get good evidence on masking. Adherence usually is an issue. Experimental blinding is nigh impossible. Individual exposure is happenstance. The course of a disease outbreak varies. Collecting data involves tracking hundreds or thousands of subjects, or relying on tangentially collected data. The small numbers who acquire the disease in an experimental timeframe leads to large confidence intervals. (The metastudy excluded several otherwise good studies on that ground.) Masking changes other social behavior, which may affect disease transmission through other mechanisms. So it’s not surprising that the studies on masking use a wide variety of methodologies, most of which are lacking in one or more important regards, often measuring somewhat different things. Or that, as consequence, a review fails to find firm evidence.

For what it is worth, I am not as positive about masking as Tufekci. I still do it in crowded indoor spaces, such as grocery stores. But have never thought the evidence for it was more than lukewarm.

It is easy to conceive the experiment that would firmly answer the salient questions. A teaching hospital identifies incoming patients who both are symptomatic and test positive for Covid, and cajoles them to give a one hour talk in a small lecture room, where medical students are assigned to listen. The patients are randomly assigned to wear masks or not, as are the students attending. The students are tracked for some weeks after, to see which acquire Covid. This is repeated with enough speakers and enough students infected, to settle the core questions about how effective masks are.

There is one glaring problem with that experiment. The ethics committee would shoot it down twenty seconds after reading it.

Well, Don Diego Vega would know how to answer that. The statistics professor who defends such experiment before the ethics committee needs to wear a mask. And flash a rapier.

Advertisement