A common issue I notice is people making statements they have no grounds to make. It’s not that they lack the authority or credentials, but rather that they don't have enough information to make the statement often because this information can’t exist in the way the statement implies.

If this sounds too abstract, let's take a look at an example I came across recently. The slide in question comes from Chamath Palihapitiya’s “Deep Dive” on the global energy transition. This report was developed with the assistance of his fund, Social Capital_, and was informed by teach-ins from McKinsey partners who educated Chamath on the subject:

I connected with a friend of mine who was a Senior Partner at McKinsey who told me a little secret: when folks like Bill Gates want to learn important new topics, McKinsey helps to educate them by bringing in their Partners to do teach-ins.

Over the next many months, I decided to do the same. They helped teach me a lot, but it also cost me more than $2mm! 

What is good enough for Bill Gates is surely good enough for us, so let's take a quick look at what one gets for all that money. (Access to these “Deep Dives” costs $1,000 annually.)

The slide projects global per capita GDP in 2100 under 1.5°C warming versus 2°C warming. To state the obvious: this is absurd pseudo-science on a staggering level, and everyone involved in creating and publishing this slide should be barred from ever voicing an opinion on anything again. This includes everyone at CarbonBrief, where they sourced these numbers from.

For these numbers to have any meaning, someone must have built an accurate and precise model of the following from now until 2100: global population growth, global GDP (i.e., the entire global economy), and climate change. Their model of the global economy is supposedly so detailed that it accounts for temperature changes and so precise that we can distinguish single percentage points nearly 100 years into the future. It should be obvious to any sane person that this is impossible.

If an honest scientist were to model anything approaching the complexity of the entire global economy, the error bars on an estimate 80 years into the future would dwarf the presumed effect of anything. Naturally, an honest scientist would never even attempt to create such a model in the first place.

Additionally, the numbers on the slide, as well as those on the CarbonBrief website, lack any error bars, ranges, or other indications that the authors are cognisant of the limitations of their models.

Speaking of error bars and confidence intervals, here’s an interesting example I came across while browsing the CarbonBrief website:

Here they have modeled the difference of heat-related deaths in China between 1.5°C and 2°C warming. Below, in barely readable grey, they provide a range: for 1.5°C, they give -0.6% to +1.8%, and for 2°C, they give -0.6% to +2.5%. Their model suggests that heat deaths might increase or decrease as the globe warms. I agree that this is about all we can reliably say on the matter.

All this is to make a simple but crucial point: you can't usefully model everything. As Niels Bohr famously said, "It is difficult to make predictions, especially about the future." Always consider what features the underlying model must capture and to what degree of accuracy for a prediction to be meaningful. Often you will discover that these models can't possibly be accurate enough for the prediction to be of any value.

Unfortunately, no one pays $1k a year for a report that admits we currently have no idea or that certain questions about the future are impossible to answer—even though this would be more valuable than relying on obviously flawed models.

A flawed model is worse than no model because decisions will inevitably be based on it. Moreover, knowing that in many cases no one could possibly have an accurate model is extremely valuable: it allows you to (1) immediately discredit anyone who thinks they do and (2) profit by betting against people making decisions based on these models (à la Taleb).

This knowledge can also save you money in the short term: instead of wasting $1,000 a year on a series of clueless deep dives (a stupid decision), you can subscribe to this blog for free and send $500 to my Bitcoin address bc1qe3lg4vlfllhazavz2vu0ran9vr6t8zvuaqxs8n – saving yourself $500 (a smart decision)!