In late 2021, Reserve Bank of Australia (RBA) Governor Phil Lowe predicted that interest rates would not rise until 2024. But the RBA has now lifted the cash rate from 0.1% in early 2022 to 3.35% in February 2023. Clearly, economic forecasting is challenging, even for those as well placed as a central bank governor. It turns out that subject matter expertise is not necessarily highly correlated with forecasting ability. Other traits and habits are necessary. Arguably, economic forecasters in central banks and Treasury departments could take lessons from so-called superforecasters, people who have demonstrably better forecasting performance than others.
Warren Hatch, CEO of Good Judgment, a highly successful forecasting business based in New York City, gave an excellent talk in 2019 on superforecasting (see What is Superforecasting? – Warren Hatch, Good Judgement). Good Judgment forecasters are renowned for their superior forecasting performance. For instance, in early 2020, Good Judgment superforecasters estimated the United States would have over 200,000 deaths from COVID-19 with 99 percent certainty, an estimate that was considered by many as excessive at the time. According to Warren (at around 11:00 in the video linked to above), traits of superforecasters include:
Incidentally, pattern recognition can be tested for using Raven’s matrices, a non-verbal abstract reasoning test which was first widely used by the British Army in World War II. Some people are born with good pattern recognition skills but others need to develop them, including by more closely observing the world and looking for patterns. Numeracy is something which can be improved through education and practice, and your knowledge base can be expanded by reading and watching or listening to educational videos and podcasts. The other two traits of superforecasters, being a creative information seeker and cognitive reflection, are habits which can be cultivated.
In the video linked to above (at 12:00), Warren Hatch offers some good advice regarding the tactics of superforecasters:
It is important that forecasts are specified as precisely as possible, including percentage likelihoods and timeframes – e.g. there is a 60% chance Australia will have a recession in the next 12 months. That way, you can keep score and assess how different forecasters perform over time.
The compare step is very important, as it allows a forecaster to test their forecasts with others and to possibly learn new things or consider alternative perspectives. In a recent conversation with Adept Economics Director Gene Tunny on the Economics Explored podcast, Warren Hatch noted the benefit of assembling a forecasting team with 6-12 people of diverse backgrounds and with different perspectives. A combination of subject matter experts and non-experts can end up yielding superior forecasts than any one expert by taking advantage of the wisdom of crowds. For links to the conversation on various different podcasting apps and for a transcript, see:
Superforecasting w/ Warren Hatch, CEO of Good Judgment – EP176 – Economics Explored
Or you can listen to the conversation via this podcast player:
A prominent Australian proponent of a superforecasting approach is Nicholas Gruen, CEO of Lateral Economics. Gruen has recommended forecasting competitions as a way of getting better economic forecasts. In a 2018 Mandarin article Making better economic forecasts he wrote:
If we held forecasting competitions of the kind I’ve discussed, we’d start to sort the forecasting sheep from the goats quite quickly. On the back of that, we could get competitors to specify their own estimates of the chance of recession over some given timeframe. It would take longer to get the data given the rarity of recessions, but even while we were waiting for the data to come in, we’d generate transparency of informed expectations of the likelihood of recession – a potentially valuable thing to know in its own right.
More generally, here and elsewhere, we could compare different forecasters forecasts to the eventual outcomes and so develop a deeper knowledge of what works, why and who’s our best guide.
There is a lot of merit in this concept. It would require expert economists to adopt a humble attitude but it would probably lead to much better forecasts and hence superior macroeconomic policy settings.
Published on 2 March 2023. Please get in touch with any comments or questions via contact@adepteconomics.com.au or by calling us on 1300 169 870.