The problem with prediction, in particular presidential prediction

Rod Jacka - Thursday, November, 10, 2016

I really like the word phantasmagorical but I rarely have a chance to use it.

It’s by no means a common word – so a simple definition might be useful:

‘a confusing or strange scene that is like a dream because it is always changing in an odd way’
Mirram-Webster Online Dictionary

Now with that in context, it’s fair to say that Donald Trump’s election has surprised more than a few, and that the use of the word phantasmagorical is highly justified. From many a perspective, it was inconceivable that someone with his rhetoric or political inexperience would be elected leader of the USA.

From the start, Nate Silver’s FiveThirtyEight website has provided great insight. It’s had a solid history of good predictions and is seen as a reliable source of information, condensing large numbers of variables into a predictive state. During the election I also chose to revisit Nassim Taleb’s book Antifragile as a companion to FiveThirtyEight. Ever since reading his first two books, Fooled By Randomness and The Black Swan several years ago I have been fascinated with statistics, volatility and randomness in general.

A key message I took from Taleb’s books was that prediction of rare events is not possible. The world is a chaotic and complex system with many variables where interactions lead to unexpected and unpredictable outcomes. Whilst we can measure many things using solid historical data, it is the unseen variables and their even greater unknown interaction (both seen and unseen) that can quickly send predictions through the floor…..as we saw.

As a long term practitioner of analysis using statistical tools and approaches I have long come to respect that models are just models – they are not the world itself. In the oft quoted words of George P. Box and Norman R. Draper (1987) “Essentially, all models are wrong, but some are useful”. Equally I have come to understand that to correctly use statistical tools you absolutely must deeply understand how they work and what their limitations are. (Like whether the media is involved?)… side thought.

There are many cases where forecasts and predictions based on data and analytics has worked, but we must always be mindful when using these tools that we are making predictions based on past observations. There have been 58 presidential elections in the entire history of the USA. As such, any model trained on predicting an event which only occurs every 4 years with 58 outcomes across 227 years is likely to get it wildly wrong from time to time. To give an analogy, all swans are white until you see a black one.

What I think we have seen with the US election results, and Brexit too, are black swan events.  They’re emerging from many factors which are leading towards some large changes in the future. It seems very likely to me that the forecasts which were being made during the election campaign were inaccurate simply because nothing like this had been observed before.


http://projects.fivethirtyeight.com/2016-election-forecast/

http://www.dailywire.com/news/10660/just-how-wrong-were-pollsters-final-polls-vs-final-james-barrett#Statistics and predictive analytics are great tools, but I also strongly encourage you to examine and understand the work of Nassim Taleb, Daniel Kahneman, Amos Tversky, Philip Tetlock and many others who work in the areas of of understanding human behaviour, the limitations of our minds and complexity science.They may not help you to predict your outcomes, but adding their approaches to your toolkit of techniques will help you to better understand the world, appreciate the limitations of data and analytics, and make you a much better marketer and analyst.
Comment

Your email address will not be published. Required fields are marked *

Search