US election numberwang

You got the Silver

I’m writing with an important message: you’ve got to stop obsessing about Nate Silver’s latest US election model run. Yes, you.

It might seem strange to say this: very few people have done such a good job of putting the idea of “numerate commentary on the news” on the map, and as I might have mentioned once or twice I’ve written a book myself about how to use numbers to make sense of the world.

But an important part is knowing the limitations of that approach. There’s no silver bullet.

I think it’s natural that we all want certainty in our lives, and the right numbers can help provide that. When COVID was a thing, we all got obsessed with the minutiae of the latest data, as a way of tracking whether the limitations being put on our lives were having an effect in stopping a deadly virus. But those numbers were imperfect: “cases” were always positive tests and not infections. Daily hospital and death data were in some sense random samples from some true underlying distribution, and obsessing about any day’s particular fluctuations (rather than looking at the longer term picture) meant that you were likely over-analysing random noise.

I think a similar thing is going on now with people getting far too excited with the outputs of US election polling models like Nate Silver’s.

The key thing is to understand what is being plotted here. This isn’t a direct polling average: if Trump really had 53% of the vote to Harris’s 47% then we could basically start planning his inauguration right now. It’s the output of Silver’s model: he gives Trump a 53% chance of winning the election, whereas a few weeks ago it was more like 45%.

But as far as I am concerned, this is noise. Firstly, there haven’t really been major changes in the fundamentals. Silver’s model does more than average the polls: it factors in a bunch of vibes-based things (how well is the economy doing) and accounts for timing effects (that the extra media interest at the time means that parties tend to get a polling bounce after their convention, for example). The main driver of the recent change on that graph was the fact that Harris-Walz didn’t get much of a polling boost, presumably because they’d had a lot of focus in the news recently anyway. That’s all fine.

But my problem is the following: there is essentially no difference between a race with 55-45 probabilities and one with 45-55 probabilities. Even if Silver has developed a fantastic tool that can exactly calibrate the probability of the outcome, we can’t tell if he’s right or wrong. And if we can’t tell a 10% difference apart, what’s the point in stressing about a 1% difference, let alone quoting the probabilities to one decimal place?

To show what I mean about there being no difference between these two races, think about the following. Suppose we have two coins: Coin A is heads 55% of the time, Coin B is heads 45% of the time. I give you a coin and ask you to tell them apart.

A natural idea is that you toss the coin: if you get heads, it’s more likely it was A. If you get tails, it’s more likely it was B. But this rule is wrong 45% of the time! It’s slightly better than a random guess, but only just.

So maybe we do lots of tosses. Take the best of 3, or the best of 33, or the best of 103. If you see more heads, it’s more likely you have Coin A, and vice versa. This seems like a pretty good plan (and in fact, it’s the best you can do). But even this rule is disappointingly high in error, even with lots of tosses:

Even with 101 coin tosses, this rule is wrong about a sixth of the time. You’d have to do a serious number of tosses to be anything like certain (say to within 5% or 1% error probability) that you have the right coin.

But in the election scenario, we don’t have that luxury. There is a single coin toss, a single election. Whatever the outcome, there’s no way we’ll be able to tell if a 55% Trump win probability or a 55% Harris win probability was the correct model output. So in that sense clinging to the minutiae of Silver’s model (or 538’s, or anyone else’s) as a means of guiding your sanity for the next couple of months doesn’t feel like a particularly sensible use of your time.

If you want to know what’s going on the race: Trump was likely to win, then Biden dropped out, and ever since it’s been a coin toss. Anything more than that is spurious precision in my view.

Sure, if things change, and Silver’s model moves out to 70-30 again in either direction, then I’d like to know about it. But my suspicion is that it’s extremely likely that won’t happen, and that we’re locked into a death spiral of “too close to call” for the next two months.

This isn’t really a surprise. The key thing is that you can ignore most of the polls. Or at least you can discount national voting intention polls to the extent that Harris being 3-4% ahead probably represents a dead heat (because of the electoral college, piling up more votes in places like California doesn’t help).

As for example

Paul Mainwood

has argued, it all likely comes down to Pennsylvania and its 20 electoral college votes. It’s extremely likely that whoever wins Pennsylvania wins the election, and it’s pretty much on a knife edge. In 2016, Trump won by 44,292 votes out of six million (less than a 1% margin). In 2020, Biden won by 80,555.

What happens this time is anyone’s guess. All the recent polls are still within the margin of error:

But remember that these models aren’t just trying to capture what would happen if the election was today, but explicitly modelling what will happen in two months’ time. When it comes down to whether 60,000 or so people in one particular state will change their mind over that period of time, we’re into the realms of soothsaying. One terrorist attack, one scandal, one big factory closing somewhere could tip the whole thing in either direction.

It’s a toss up. We just don’t know.

Like I say, I get the idea that we’d all like to find some certainty and some closure about this. If there really was a way that some modern-day haruspex could sift through the polling and economic data and give us our answer now, I’d be all for it. But I think that obsessing over anything more than leading-digit shifts in Silver’s model is a waste of everyone’s time.

In fact, as

Ben Recht

has argued, the only real reason that we might care about small fluctuations in model outputs is if we were going to trade them like financial products. And in fact, that’s exactly what is going on:

For Silver to take a job at Polymarket while claiming to be the sage for our highly probabilized society is the definition of a conflict of interest. What On the Edge finally shows us is that Silver is no longer just a handicapper: he’s a bookie

Good luck to the guy, I hope he sells lots of copies of his book, and lands as many $95 annual subscriptions to his Substack as he can off the back of all this. But as someone once wrote, as news consumers, we’ve got to be able to tell the signal from the noise.