May 15, 2017

Quick Thoughts on The Signal and the Noise

Learning to think probabilistically is one of the most critical skills one can master. Nate Silver’s The Signal and the Noise: Why So Many Predictions Fail–but Some Don’t is a valuable book on thinking probabilistically and forecasting in an uncertain environment. It compares and contrasts examples across multiple disciplines, including weather forecasting, seismology, finance, and more.

This book pairs well with Against the Gods, Fortune’s Formula and Superforecasting. Against the Gods is in my opinion, the most important book on the development of probabilistic thinking. Early civilizations were good with geometry and logic, but helpless with uncertainty. Ironically it was gamblers and heretics who moved mankind forward by developing the science of probability, statistics, and ultimately risk management. Fortune’s Formula shows the connection between information theory, gambling, and correct position sizing for investors. It helps the answer the question: when you have a slight edge, how much should you bet? Nate Silver draws heavily on Superforecasting. Particularly important is the idea of “foxes and hedgehogs”. Foxes are multidisciplinary, adaptable, self critical , tolerant of complexity, cautious and empirical. In contrast, Hedgehogs are specialized, stalwart, stubborn, order-seeking, confident, and ideological. As you might expect, foxes make far better forecasters than hedgehogs, even though hedgehogs make for better television.

Anyways, here are a few key insights from my notes on The Signal and the Noise

1) Data is useless without context.

There are always patterns to find in data, but its critical to understand the theory behind the system you are studying to avoid being fooled by noise. This is true in forecasting the weather, investing, betting on sports, or any other probabilistic endeavor. The ability to understand context is also a critical advantage humans have over computer programs.

“Statistical inferences are much stronger when backed up by theory or at least some deeper thinking about their root causes. “

The importance of understanding context comes to the forefront when you compare human’s success with weather forecasting, vs relative failure with earthquake forecasting.

“Chaos theory is a demon that can be tamed- weather forecasts did so, at least in part. But weather forecasters have a much better theoretical understanding of th earth’s atmosphere than seismologists do of the earth’s crust. They know more or less, how weather works, right down to the molecular level. Seismologists don’t have that advantage. “

The ability to understand context is what separates success from failure in all pursuits dealing with uncertainty. The profile of professional sports gambler Bob Voulgaris, is highly instructive. Voulgaris focuses on NBA basketball. A key insight is that Voulgaris has powerful tools for analyzing data, and he makes good use of the data, but he also has deep understanding of the qualitative subletities of how NBA basketball works. Obvious statistical patterns are quickly incorporated into betting lines, whether they are signal or noise. Voulgaris looks deeper, and finds places where the line misprices true probabilities.

“Finding patterns is easy in any data rich environment; thats what mediocre gamblers do. The key is in determining whether the patterns represent noise or signal. “

2) Beware of overconfidence

“… the amount of confidence someone expresses in a prediction is not good indication of its accuracy, to the contrary, these qualities are often inversely correlated. “

3) Think big, and think small. Mix the macro and the micro.

“Good innovators typically think very big, and they think very small. New ideas are sometimes found in the most granular of details where few others bother to look. And they are sometimes found when you are doing your most abstract and philosophical thinking, considering why the world is the way that it is and whether there might be an alternative to the dominant paradigm.”

This is reminiscent of the “global micro” approach used by several manager’s profiled in Inside the House of Money: Top Hedge Fund Traders on Profiting in the Global Markets

4) Recognize the Value of Bayesian Thinking

The work of Thomas Bayes forms the framework underlying how good gamblers think.

Bayes was an English minister who argued in his theological work that admitting our own imperfections is a necessary step on the way to redemption. His most famous work, however, was “An Essay toward Solving a Problem in the Doctrine of Chances,” which was not published until after his death. One interpretation of the essay concerns a person who emerges into the world( ie Adam, or someone from Plato’s cave), and rises to see the sun for the first time:

“At first the does not know whether this is typical of some sort of freak occurrence. However each day that he survives and the sun rises again, his confidence increases that it is a permanent feature of nature. Gradually, through this purely statistical form of inference, the probability that he assigns to his prediction that the sun will rise again tomorrow approaches(although never exactly reaches) 100 percent.”

In essence, beliefs on probability are updated as new information comes in.

Ironically Bayes philosophical work was extended by the mathematician and astronomer Pierre Simon-Laplace, who was likely an atheist. Although Laplace believed in scientific determinism, he was frustrated with the disconnect between (what he believed to be the perfection of nature, and human imperfections in understanding it, in particular with regards to astronomical observations. Consequently, he developed some measuring techniques that relied on probabilistic inferences, rather than exact measurements. “Laplace came to view probability as a waypoint between ignorance and knowledge.” The combined work of Laplace and Bayes led to simple expression that is concerned with conditional probability. In essence Bayesian math can be used to tell us the probability that a theory or hypothesis if some event has happened.

5) The road to wisdom is to be less and less wrong.

forecasting, or at least operating in an uncertain environment, is an iterative process.

Nate Silver titles one of the chapters “Less and Less Wrong, as a homage to the Danish mathematician, scientist, inventor, and poet Piet Hein, author of Grooks: