A Journal of the Plague Year

Someone's got it in for me, they're planting stories in the press

Long-term subscribers might remember that back in July I suggested it was time for me to “draw a line under some of the pandemic stuff, and to move on”. I’ve not done a bad job of sticking to that and broadening out to other topics, I think, but I’m going to break that vow today.

I can only apologise: if you don’t want to read any further, I don’t blame you. Have a Happy Christmas, go and read something more festive instead, and I’ll see you in the New Year!

For the hardcore among you, I want to talk about the paper by Independent SAGE (a UK-based pandemic group) which was recently published in the journal Nature Protocols. While it feels odd for a group like that to be marking its own homework, I don’t think the paper itself is too bad: it’s good for people to reflect on what happened and why.

However, my reaction overall includes a sigh, a raised eyebrow, a factual correction and an elephant in the room, so I’d like to share those with you.

A sigh

At the end of the paper, there are some reflections on how Independent SAGE could have done better. The first two are good, I think, but there’s a sigh from me in terms of “what took you so long, guys?”. That is, for the first time (I think?) they acknowledge that the name Independent SAGE itself was problematic because it “led to confusion among the lay public … and caused some offense in official circles”.

Similarly, they admit that their slogan “Following the Science” wasn’t well chosen because

the reality was of multiple scientific and science-related questions that were often value laden and could be framed in various ways, multiple streams of potentially relevant evidence on which there was little or no consensus among experts, and multiple implications of that body of evidence for different policy decisions

But again, it feels a bit late when some of us have been pointing it out for years! Back in May I wrote on Substack about the dangers of blurring the line between science and activism for example. And as long ago as September 2022, when the British Medical Journal was commissioning articles from many of the same authors for its COVID Inquiry, I suggested that it would be better to get balance:

only commissioning articles from one side of the argument is a blatantly wrong way for scientific and medical journals to behave

The reaction that

Given evidence on school mitigations & massive UK kid infection since 2021, what's the "other side"?

didn’t exactly feel like alternative viewpoints were welcome. So I’m glad that things might have changed nuance-wise, but you’ll forgive me for not celebrating too wildly.

A raised eyebrow

In the same section on reflections, the Nature Protocols paper chides the group for not being firmer in responding to critics and for allowing themselves to be portrayed as extremists:

We were, perhaps, insufficiently proactive in responding to misleading information and deliberate untruths about Independent SAGE and its members. […] Some people incorrectly depicted us as politically motivated and wedded to particular extreme policies (such as pursuing an uncompromising ‘zero COVID’ agenda

As someone who did more than his fair share of pushing back on some of Independent SAGE’s positions, and getting periodically monstered as a result, it doesn’t really feel like a lack of response to critics from them was a problem. And I really don’t understand the idea that presenting them as pursuing Zero COVID was a caricature.

It’s something the group themselves tweeted to advocate dozens of times between July 2020 and February 2021. And while there was always some doubt as to what it meant in practice - did it mean “very low numbers” or “elimination” of the virus? - it’s hard to see how on the basis of this report with its header

Independent SAGE believes that the UK government must fundamentally change its approach and we propose a new overarching strategic objective of achieving a Zero COVID UK, i.e. the elimination of the virus from the UK. [bold text mine for emphasis]

it was wrong to describe them as “pursuing an uncompromising ‘zero COVID’ agenda”. Sure, we can argue whether they were right or wrong to take that viewpoint, but I don’t see the point in them claiming they never did.

A factual correction

In a section on funding, the Nature Protocols paper mentions that

Independent SAGE used two crowdfunder exercises to raise relatively modest sums (£15,000–20,000 each time) to continue supporting briefings and communication activity in 2022 and 2023 [links added by me for clarity]. In each case, the crowdfunders were closed after a few days due to the rapid and generous responses.

This is true, but omits important information. That is, in 2020 Independent SAGE held another crowdfunder, which remained open for months. In the end it raised £60,673 from over 2,000 donations. Not quite so short-term or modest!

For a paper which talks about the need for “providing consistent and accurate messages”, it’s not great that the true amount of crowdfunding was around 2.5 times larger than it stated.

It’s not the end of the world, but I hope the paper can be corrected.

An elephant in the room

While the paper reflects on various places where Independent SAGE might have got it wrong, it doesn’t really discuss at all what I regard as the group’s biggest error: putting too much faith in their own in-house modelling in 2020.

This might seem an odd thing to say. If you go to their own website, there’s a clear and unambiguous statement that

Independent SAGE does not commit to – or engage in – any particular modelling initiative.

However, let’s just say that recollections may vary. Thanks to the Wayback Machine, we can see that from November 2020 to February 2021 at least, the group’s own website referred to one of its members as being “in charge of developing a generative SEIR COVID19 model”. In their very first report, their fourth overall recommendation was to advocate for “a technique called Dynamic Causal Modelling (DCM)”, developed by the same member, who referred to themselves in print as having “special responsibility for modelling”.

My problem with this, beyond the retcon, is that this was a speculative and unproven model. It was clear as early as May 2020 that it required ad hoc corrections (“immunological dark matter”) to explain the difference in pandemic trajectories in the UK and Germany, for example.

This was particularly frustrating for me, because I’d produced a paper at the time which suggested that 45% of the difference in European trajectories could be explained by a non-standard (“lived”) measure of population density. I still think it was an interesting idea, but we never got a journal to take it seriously enough to publish it. That’s how things go sometimes, but it’s still a shame.

Meanwhile, because they included the likelihood that a sizeable fraction of the population may have pre-existing immunity, the DCM projections tended to be extremely optimistic on second waves. In the Independent SAGE report on schools this led to a graph of risk which simply didn’t run past August 2020, perhaps implying that the correct strategy was simply to optimise over this period with continuing closures.

In a paper published in BMJ Global Health, three members of Independent SAGE used the same approach to make “a worst-case prediction, under which we would expect a second peak of about 100 deaths per day”. The model made it into the 18th September ISAGE briefing:

giving a far too optimistic view of the possible unfolding catastrophe, suggesting any second wave would be a fraction of the first one. (Meanwhile a day later I was using far simpler methods to project that hospital beds occupied in the North West might reach the levels of the first peak by Halloween - and guess what day the second lockdown was announced?)

Like I keep saying, I don’t think any of this stuff is disqualifying:

I’ve said it many times before, but it’s exceedingly unlikely that anyone even got all the main calls right, let alone being correct on every detail. If we got rid of everyone from the discussion who’d made mistakes there would be nobody left. That’s fine, we learn from our mistakes and move on.

But it’s just frustrating that in a paper which aims to reflect on mistakes, the reflection only seems to go so far.

However, it’s not the time of year for frustration! And there’s ever-decreasing value in going over and over ground from four years ago, so I’ll try my best not to do it too much again. But in the meantime, have a very happy Christmas, thanks for subscribing, and I’ll see you in 2025!