Test counter-intuitive things because no one else ever does
Alchemy one of a small set of books that helped fill out important gaps in my understanding of how the world works. Its essential reading for people who think they are logical, and valuable for everyone else. I’ve organized my (copious) notes and highlights around key themes below. But seriously you should just get the book.
- Not everything that makes sense works, and not everything that works makes sense.
- Test counterintuitive things and ask dumb questions .
- Never denigrate something as irrational until you have considered what purpose it really serves.
- Try to understand the real reason for things(not the surface reason)
- Cooperation has major evolutionary value, but most deductive logical thinking ignores it.
- If you optimise incentive systems(or anything else) in one direction, you may be creating a weakness somewhere else.
- People are way too confident in traditional technocratic approaches and big data solutions relative to how well they actually work
- Hacking personal improvement: Many important features of the human brain are not under our direct control but are instead the product of instinctive and automatic emotions.
- Embrace uncertainty.
Not everything that makes sense works, and not everything that works makes sense.
We used to have a shortage of conventional deductive logic. Now we have too much. Conventional deductive logic is often a good thing. However we have venerated this manner of thinking so much that we have become blind to the situations in which it doesn’t work. Complex and evolved systems are not consciously designed, and often second order consequences that aren’t readily apparent are more important than what can be analyzed on the surface. Evolution doesn’t care if things make sense- it only cares if things work. Therefore, trial and error usually works better than reasoning everything out before acting.
Often when a phenomenon or group behavior doesn’t seem logical/rational, it is because the observer has an incomplete model of reality.
Not everything that makes sense works, and not everything that works makes sense. The top-right section of this graph is populated with the very real and significant advances made in pure science, where achievements can be made by improving on human perception and psychology. In the other quadrants, ‘wonky’ human perception and emotionality are integral to any workable solution. The bicycle may seem a strange inclusion here: however, although humans can learn how to ride bicycles quite easily, physicists still cannot fully understand how bicycles work. Seriously. The bicycle evolved by trial and error more than by intentional design.
There are two separate forms of scientific enquiry – the discovery of what works and the explanation and understanding of why it works. These are two entirely different things, and can happen in either order. Scientific progress is not a one-way street. Aspirin, for instance, was known to work as an analgesic for decades before anyone knew how it worked. It was a discovery made by experience and only much later was it explained. If science did not allow for such lucky accidents,* its record would be much poorer – imagine if we forbade the use of penicillin, because its discovery was not predicted in advance? Yet policy and business decisions are overwhelmingly based on a ‘reason first, discovery later’ methodology, which seems wasteful in the extreme. Remember the bicycle. Evolution, too, is a haphazard process that discovers what can survive in a world where some things are predictable but others aren’t. It works because each gene reaps the rewards and costs from its lucky or unlucky mistakes, but it doesn’t care a damn about reasons. It isn’t necessary for anything to make sense: if it works it survives and proliferates; if it doesn’t, it diminishes and dies. It doesn’t need to know why it works – it just needs to work.
Perhaps a plausible ‘why’ should not be a pre-requisite in deciding a ‘what’, and the things we try should not be confined to those things whose future success we can most easily explain in retrospect. The record of science in some ways casts doubt on a scientific approach to problem solving.
Like thinking fast and slow, Alchemy makes an intellectual reader see the value of humility:
Once you accept that there may be a value or purpose to things that are hard to justify, you will naturally come to another conclusion: that it is perfectly possible to be both rational and wrong. Logical ideas often fail because logic demands universally applicable laws but humans, unlike atoms, are not consistent enough in their behaviour for such laws to hold very broadly.
It’s true that logic is usually the best way to succeed in an argument, but if you want to succeed in life it is not necessarily all that useful; entrepreneurs are disproportionately valuable precisely because they are not confined to doing only those things that make sense to a committee
… if we allow the world to be run by logical people, we will only discover logical things. But in real life, most things aren’t logical – they are psycho-logical. There are often two reasons behind people’s behaviour: the ostensibly logical reason, and the real reason
Test counterintuitive things, ask dumb questions
Alchemy provides the blueprint for a type of inversion one can do in approaching problems.
Test counterintuitive things, because no one else ever does.
Here’s a simple (if expensive) lifestyle hack. If you would like everything in your kitchen to be dishwasher-proof, simply treat everything in your kitchen as though it was; after a year or so, anything that isn’t dishwasher-proof will have been either destroyed or rendered unusable. Bingo – everything you have left will now be dishwasher-proof! Think of it as a kind of kitchen-utensil Darwinism. Similarly, if you expose every one of the world’s problems to ostensibly logical solutions, those that can easily be solved by logic will rapidly disappear, and all that will be left are the ones that are logic-proof – those where, for whatever reason, the logical answer does not work.
Most political, business, foreign policy and, I strongly suspect, marital problems seem to be of this type.
Similarly, if you expose every one of the world’s problems to ostensibly logical solutions, those that can easily be solved by logic will rapidly disappear, and all that will be left are the ones that are logic-proof – those where, for whatever reason, the logical answer does not work. Most political, business, foreign policy and, I strongly suspect, marital problems seem to be of this type.
The mental model at work here is like a broader application of the case in World War II , where Navy analysts had to figure out how to build stronger planes not by thinking about the missing bulletholes.
Closely related is how great insight comes from asking dumb questions. Leaders must allow and encourage their team to ask dumb questions:
This freedom is much more valuable than we realise, because to reach intelligent answers, you often need to ask really dumb questions.
Never denigrate something as irrational until you have considered what purpose it really serves.
There are a lot of social psychology lessons in Alchemy Many of them are more fun(and easier to process) examples of the phenomonen revealed through evolutionary case studies in Secrets of Our Success.
There is an important lesson in evaluating human behaviour: never denigrate a behaviour as irrational until you have considered what purpose it really serves
Try to understand the real reason for things(not the surface reason)
Sometimes behavior that seems illogical has a hidden evolutionary value. People make mistakes when they judge others by surface actions, without considering things working below the surface.
there is an ostensible, rational, self-declared reason why we do things, and there is also a cryptic or hidden purpose. Learning how to disentangle the literal from the lateral meaning is essential to solving cryptic crosswords, and it is also essential to understanding human behaviour.
….Sometimes human behaviour that seems nonsensical is really non-sensical – it only appears nonsensical because we are judging people’s motivations, aims and intentions the wrong way. And sometimes behaviour is non-sensical because evolution is just smarter than we are. Evolution is like a brilliant uneducated craftsman: what it lacks in intellect it makes up for in experience.
….One problem (of many) with Soviet-style command economies is that they can only work if people know what they want and need, and can define and express their wants adequately. But this is impossible, because not only do people not know what they want, they don’t even know why they like the things they buy. The only way you can discover what people really want (their ‘revealed preferences’, in economic parlance) is through seeing what they actually pay for under a variety of different conditions, in a variety of contexts. This requires trial and error – which requires competitive markets and marketing.
There are no universal laws of human behaviour:
Our very perception of the world is affected by context, which is why the rational attempt to contrive universal, context-free laws for human behaviour may be largely doomed.
Cooperation has major evolutionary value, but most deductive logical thinking ignores it
Humans are a social species. This is not exactly a big insight, yet much of standard applied psychology and economics actually ignores this in practice. Indeed many ostensibly logical people ignore this in attempting to understand the world. This is made worse by the contrived nature of a lot of academic research.
Robert Zion, the social psychologist, once described cognitive psychology as ‘social psychology with all the interesting variables set to zero’. The point he was making is that humans are a deeply social species (which may mean that research into human behaviour or choices in artificial experiments where there is no social context isn’t really all that useful).
Humans are willing to forgoe short term expediency in order to cooperate and sacrifice for other people.
This is not irrationality – it is second-order social intelligence applied to an uncertain world.
There is an important biological reason:
Unlike short-term expediency, long-term self-interest, as the evolutionary biologist Robert Trivers has shown, often leads to behaviours that are indistinguishable from mutually beneficial cooperation. The reason the large fish does not eat its cleaner….
If fish (and even some symbiotic plants) have evolved to spot this sort of distinction, it seems perfectly plausible that humans instinctively can do the same, and prefer to do business with brands with whom they have longer-term relationships. This theory,
As a result, like any social species, we need to engage in ostensibly ‘nonsensical’ behaviour if we wish to reliably convey meaning to other members of our species.
One of the most important ideas in this book is that it is only by deviating from a narrow, short-term self-interest that we can generate anything more than cheap talk. It is therefore impossible to generate trust, affection, respect, reputation, status, loyalty, generosity or sexual opportunity by simply pursuing the dictates
There is a parallel in the behaviour of bees, which do not make the most of the system they have evolved to collect nectar and pollen. Although they have an efficient way of communicating about the direction of reliable food sources, the waggle dance, a significant proportion of the hive seems to ignore it altogether and journeys off at random. In the short term, the hive would be better off if all bees slavishly followed the waggle dance, and for a time this random behaviour baffled scientists, who wondered why 20 million years of bee evolution had not enforced a greater level of behavioural compliance. However, what they discovered was fascinating: without these rogue bees, the hive would get stuck in what complexity theorists call ‘a local maximum’; they would be so efficient at collecting food from known sources that, once these existing sources of food dried up, they wouldn’t know where to go next and the hive would starve to death. So the rogue bees are, in a sense, the hive’s research and development function, and their inefficiency pays off handsomely when they discover a fresh source of food. It is precisely because they do not concentrate exclusively on short-term efficiency that bees have survived so many million years.
If you optimise incentive systems(or anything else) in one direction, you may be creating a weakness somewhere else.
Incentives, incentives incentives.
In institutional settings, we need to be alert to the wide divergence between what is good for the company and what is good for the individual. Ironically, the kind of incentives we put in place to encourage people to perform may lead to them to be unwilling to take any risks that have a potential personal downside – even when this would be the best approach for the company overall. For example, preferring a definite 5 per cent gain in sales to a 50 per cent chance of a 20 per cent gain.
If you optimise something in one direction, you may be creating a weakness somewhere else.
In any complex system, an overemphasis on the importance of some metrics will lead to weaknesses developing in other overlooked ones. I prefer Simon’s second type of satisficing; it’s surely better to find satisfactory solutions for a realistic world, than perfect solutions for an unrealistic one. It is all too easy,
See Also: Goodhart’s Law and the Fall of Nick Schorsch , Department of Unintended Consequences, The Value of Improvisation and Informal Processes
People are way too confident in traditional technocratic approaches and big data solutions relative to how well they actually work
We don’t need to throw out economic models- but we need to spend time considering what htey ignore.
By using a simple economic model with a narrow view of human motivation, the neo-liberal project has become a threat to the human imagination.
…The alchemy of this book’s title is the science of knowing what economists are wrong about. The trick to being an alchemist lies not in understanding universal laws, but in spotting the many instances where those laws do not apply. It lies not in narrow logic, but in the equally important skill of knowing when and how to abandon it. This is why alchemy is more valuable today than ever.
…The technocratic mind models the economy as though it were a machine: if the machine is left idle for a greater amount of time, then it must be less valuable. But the economy is not a machine – it is a highly complex system. Machines don’t allow for magic, but complex systems do. …
…We should absolutely consider what economic models might reveal. However, it’s clear to me that we need to acknowledge that such models can be hopelessly creatively limiting. To put it another way, the problem with logic is that it kills off magic. Or, as Niels Bohr* apparently once told Einstein, ‘You are not thinking; you are merely being logical.’ A strictly logical approach to problem-solving gives the reassuring impression that you are solving a problem, even when no such process is possible; consequently the only potential solutions considered are those which have been reached through ‘approved’ conventional reasoning – often at the expense of better (and cheaper) solutions…
The advent of big data doesn’t change this:
We should also remember that all big data comes from the same place: the past. Yet a single change in context can change human behaviour significantly. For instance, all the behavioural data in 1993 would have predicted a great future for the fax machine.
Tangentially related, Alchemy connects economics with psychology and evolutionary biology in a better way than just about anything else I’ve read(some other examples see: Ecological Consequences of Hedge Fund Extinction)
Because they offer competing choices, consumer markets provide a guide to our unconscious in a way that theories don’t. For this reason, I have called consumer capitalism ‘the Galapagos Islands for understanding human motivation’; like the beaks of finches, the anomalies are small-but-revealing.
Hacking personal improvement: Many important features of the human brain are not under our direct control but are instead the product of instinctive and automatic emotions.
Alchemy has many insights for how one can approach personal improvement, and “hacking oneself.” Many important features of the human brain are not under our direct control but are instead the product of instinctive and automatic emotions:
There is a good evolutionary reason why we are imbued with these strong, involuntary feelings: feelings can be inherited, whereas reasons have to be taught, which means that evolution can select for emotions much more reliably than for reasons. To ensure your survival, it is much more reliable for evolution to give you an instinctive fear of snakes at birth than relying on each generation to teach its offspring to avoid them. Things like this aren’t in our software – they are in our hardware.
Useful to have a machine metaphor. I think of it like a computer. He uses a car:
…The truth is that you can control the gearbox of an automatic car, but you just have to do it obliquely. The same applies to human free will: we can control our actions and emotions to some extent, but we cannot do so directly, so we have to learn to do it indirectly – by foot rather than by hand.
Closely related, he had some insights into the placebo effect:
The placebo effect, like many other forms of alchemy, is an attempt to influence the mind or body’s automatic processes. Our unconscious, specifically our ‘adaptive unconscious’ as psychologist Timothy Wilson calls it in Strangers to Ourselves (2002), does not notice or process information in the same way we do consciously, and does not speak the same language that our consciousness does, but it holds the reins when it comes to much of our behaviour. This means that we often cannot alter subconscious processes through a direct logical act of will – we instead have to tinker with those things we can control to influence those things we can’t or manipulate our environment to create conditions conducive to an emotional state which we cannot will into being.
The actions required to create such conditions may involve a certain degree of what appears to be bullshit – but it is only bullshit when you don’t know what its reason is. It is this oblique hacking of unconscious emotional and physiological mechanisms that often causes suspicion of the placebo effect, and of related forms of alchemy. Essentially we like to imagine we have more free will than we really do, which means we favour direct interventions that preserve our inner delusion of personal autonomy, over oblique interventions that seem less logical.
Many problems come from seeking out false certainty. Avoiding uncertainty can have social benefits within certain institutions, and it can abodi discomfort. But ultimately certainty is an illusion, and pretending it is reality is a disaster. Embracing uncertainty is a prerequisite to major breakthroughs. See also: Thinking in Bets
The modern education system spends most of its time teaching us how to make decisions under conditions of perfect certainty. However, as soon as we leave school or university, the vast majority of decisions we all have to take are not of that kind at all. Most of the decisions we face have something missing – a vital fact or statistic that is unavailable, or else unknowable at the time we make the decision. The types of intelligence prized by education and by evolution seem to be very different. Moreover, the kind of skill that we tend to prize in many academic settings is precisely the kind that is easiest to automate.
If you want to look like a scientist, it pays to cultivate an air of certainty, but the problem with attachment to certainty is that it causes people completely to misrepresent the nature of the problem being examined, as if it were a simple physics problem rather than a psychological one.
Related Ockham’s Notebook Posts
- Why its Wise to Think in Bets
- The Ecological Consequences of Hedge Fund Extinction
- Empty Spaces on Maps
- Goodharts Law and the Fall of Nick Schorsch
- Department of Unintended Consequences
- The Value of Improvisation and Informal Processes
Empty Spaces on Maps
Prior to the 15th century, maps generally contained no empty spaces. Mapmakers simply left out unfamiliar areas, or filled them with imaginary monsters and wonders. This practice changed in Europe as the great age of exploration began. In Sapiens, Yuval Harari argues that leaving empty spaces on maps reflected a more scientific mindset, and was a key reason that Europeans were able to conquer and colonize other continents, in spite of starting with a technological and military disadvantage. Conquerors were curious, but the conquered were uninterested in the unknown. Amerigo Vespucci, after whom our home continent was named, was a strong advocate of leaving unknown spaces on maps blank. Explorers used these maps to move beyond the known, sailing into those empty spaces so they did not stay unmapped for long.
The same phenomenon occurs in business. In the Innovator’s Dilemma, Clayton Christensen shows why large established ostensibly well-run companies so frequently miss out on major waves of innovation. A key principle in the book is the difference between sustaining technologies, which merely improve the status quo, and disruptive technologies, which offer a new and unique value proposition. Large companies will frequently focus on sustaining technologies, and ignore disruptive technologies that serve fringe markets initially. Ultimately its disruptive technologies that define business history. Yet complacent companies don’t figure that out until its too late.
Companies whose investment processes demand quantification of market sizes and financial returns before they can enter a market get paralyzed or make serious mistakes when faced with disruptive technologies.
There are two parts to overcoming the innovator’s dilemma:
- Acknowledging that the market sizes and potential financial returns of a nascent market are unknowable and cannot be quantified (drawing the blank spaces on the maps) and;
- Entering the nascent market in the absence of quantifiable data- (travelling into the empty space)
Analogous ideas also apply to investing. In Investing in the Unknown and Unknowable, Richard Zeckhauser distinguishes between situations where the probability of future states is known, and when it is not. The former is the realm of academic finance and decision theory. The latter is the real world.
The real world of investing often ratchets the level of non-knowledge into still another dimension, where even the identity and nature of possible future states are not known. This is the world of ignorance. In it, there is no way that one can sensibly assign probabilities to the unknown states of the world. Just as traditional finance theory hits the wall when it encounters uncertainty, modern decision theory hits the wall when addressing the world of ignorance.
Human bias leads us into classic decision traps when confronted with the unknown and unknowable. Overconfidence and recollection bias are especially pernicious. Yet just because we are ignorant doesn’t mean we need to be nihilists. The essay has some key optimistic conclusions:
The first positive conclusion is that unknowable situations have been and will be associated with remarkably powerful investment returns. The second positive conclusion is that there are systematic ways to think about unknowable situations. If these ways are followed, they can provide a path to extraordinary expected investment returns. To be sure, some substantial losses are inevitable, and some will be blameworthy after the fact. But the net expected results, even after allowing for risk aversion, will be strongly positive.
Examples in the essay include David Ricardo buying British Sovereign bonds on the eve Battle of Waterloo, venture capital, frontier markets with high political risk, and some of Warren Buffet’s more non-standard insurance deals. Yet since even the industries that seem simple and steady can be disrupted, its critical to keep these ideas in mind at all times in order to avoid value traps.
The best returns are available to those willing to acknowledge ignorance, then systematically venture into blank spaces on maps and in markets.
Investing in the Unknown and Unknowable
When is the crowd right?
Humans have flawed brains that cause them to act crazy sometimes. And in groups, people get even more crazy. Many smart people believe dumb things. Sometimes a group of otherwise completely sane people come together and do something insane. History is full of examples of the “ Extraordinary Popular Delusions and the Madness of Crowds, or Manias, Panics and Financial crisis. Humans sometimes join suicide cults. Human literally burned witches not that long ago. Groupthink is a helluva drug.
Financial markets provide an arena in which the biggest gains can be made betting against consensus. Yet statistically speaking, the consensus is usually right. The times when the crowd goes crazy are notable because they are exceptions. One must carefully decide when to be a contrarian.
Under what conditions is the consensus likely to be wrong? Lets invert the question: under what conditions is the crowd likely to be right?
Most of the time, a large group of people actually comes to a more accurate conclusion than any one individual. The success of Estimize, which crowdsources earnings estimates is one useful empirical example. “Wisdom of Crowds” is a well documented phenomenon, and well summarized in the book by the same title.
Why is the crowd so often right? What must happen for the crowd to be right ? Researchers have identified four interrelated conditions that encourage the wisdom of the crowds:Diversity, Information Availability, Decentralization, and existence of an Aggregation Mechanism.
Understanding these conditions can help one know when to follow the zeitgeist, and when to make a contrarian bet against it. A firm grasp of the facts on both sides of a controversy is necessary, but possibly not sufficient. One can never be sure of all the facts. Its also useful to understand the broader social forces, and how they influence the likelihood of the consensus being right or wrong. Searching for these conditions(or their absence), can be a useful method of avoiding cults and identifying opportunity, beyond just the facts of the individual situations.
The biggest gains are available by being a contrarian who understands the crowd.
The key is understanding when the wisdom of crowds flips to the madness of crowds. And the essential insight is that it has to do with a violation of one or more of the core conditions for a wise crowd.Michael Mauboussin: Who is on the other side?
Diversity implies that each person has their own point of view and some private information, even if only their unique interpretation of the available public information. Diversity is important because it adds different perspectives and increases the amount of available information.The Value of Crowdsourcing: Evidence from Earnings Forecasts
Crowded trades have a tendency to crash- leading to no bid markets all the way down. Academics have noted that in a run up to a market crash diversity of population declines. The market becomes fragile, and eventually there is no one to buy from (or sell to).
One reason Estimize’ earnings estimates have tended to be better than Wall Street Sell side is that Estimize analysts are a diverse group of independent thinkers from around the world, holding a variety of different jobs. A lot of Wall Street analysts go to the same conferences, went to the same schools, etc.
When people come to the same conclusion from different backgrounds, logical methods, etc , their collective wisdom refines understanding of reality. The opposite occurs when people feel pressure to conform. Its important to note this is a genuine deep diversity of thought and perspective, not a superficial check the box diversity.
If multiple people with different viewpoints all come to similar conclusions, the odds of the opposite being true decrease substantially. On the other hand, if there is an obivous archetype of the thinker on the other side, then maybe a contrarian opportunity is available.
As Michael Maubossin has noted- conformity is a nonlinear process:
Scientists even have a sense of the neurobiological basis for conformity.Informational cascades occur when individuals follow the decisions of those who precede them without regard to their personal information.
Epidemiological models are useful here.
The wisdom of crowds does not emerge in groups of idiots. It only applies when there is widespread access to information necessary to reach a conclusion. The extreme opposite occurs in totalitarian societies (or large corporations), where information is tightly restricted. Prior to the internet, information sometimes diffused slowly through a market, leading to massive price discrepancies obvious at even a quick quantitative glance.
Quality of input is critical to the success of crowdsource analysis:
Alternatively, it is possible that the inclusion of forecasts from certain individuals, such as Non-Professionals, may provide no value, or worse, cause the Estimize consensus to deviate further from actuals. Surowiecki (2004) states that although diversity matters, assembling a group of diverse but thoroughly uninformed people is not likely to lead to wise outcomes.
Independence is related to diversity. People need to be able to freely analyze reality and discuss opinions. Conventional wisdom is more likely to be accurate when it is freely subjected to challenge. When there are institutional or social factors that make people extremely afraid to speak truth, what everybody says to be true, may be wrong.
Children learn of this phenomenon early through the Emperor Wears No Clothes. For grown ups, see Death of Stalin for an example of lack of independence leading to morbidly hilarious results.
Independence requires relative freedom from opinions and actions of others, not complete isolation. Independence enables people to actually express their diverse information and reduces potential bias in the group decision.
Decentralization allows people to specialize and draw on local knowledge, without any individual or small group dictating the process.
Diversity and independence all fit in nicely with decentralization. Through specialization, decentralization encourages independence and increases the scope and diversity of information. Decentralization reduces the risk that independence and diversity will go away. Similarly having capital flows from all around the world, not just from a small group of schools or similarly thinking firms, increases the likelihood that markets become more efficient.
Existence of an Aggregation Mechanism
Finally, an aggregation mechanism is necessary to collect the individual opinions and harness the ‘wisdom-of-crowds’ effect.
This is basically why capitalism has succeeded. The price mechanism aggregates facts about supply and demand better than any bureaucracy could. At the same time, this why often the best opportunities to earn an investment profit are in illiquid asset classes where the market does not function as an aggregation mechanism to make the price close to right.
Of course, just because the market consensus is wrong, doesn’t mean that is necessarily wise to bet against it today. Must also consider reflexivity, narratives, and capital flows etc, and maintain a balance sheet that allows one to survive long periods of mass delusion.
Postscript: This is all indirectly related earlier post on finding underfollowed opportunities: The hard thing about finding easy things . This linked the ideas of both Sun Tzu and Warren Buffett. Some of the specific opportunity sets mentioned in this post have since been too widely known and we’ve moved further into more esoteric off the beaten path ideas. Nonetheless the basic principal still holds: there are more likely to be opportunities where the crowd isn’t looking.
The Paradox of Alternative Data
“My God, they’re purple and green. Do fish really take these lures?” And he said, “Mister, I don’t sell to fish.
Informational edge can drive fantastic alpha while it lasts. This explains the increasing investment industry focus on non-traditional data sources, aka alternative data. If you are the first to acquire a new alternative data set, you might be able to develop insights no one has.
Yet once a lot of people are using it, it is less likely to drive alpha. It might be table stakes to not get screwed, or it might be already be instantaneously reflected in the price of assets.
Furthermore, most opportunities really hinge on a couple factors- more info isn’t always useful. That won’t stop the alternative data industry from doubling to $400 million by 2021, as a widely cited Tabb Group report predicts This is worth considering while one is caught up in an alternative data arms race.
Yet some data sets genuinely will provide an edge.
Perhaps a data set that no one else is looking at provides the edge you need. If a data set isn’t established as useful, the provider of that data will probably offer it cheaper in the early days of their business. There are so many alternative data providers out there, that marketing strategy is important for startups.
Ironically, the provider can charge higher price once word about its value gets out among investors.. So later adopters might pay more for an edge that is already gone.
Of course eventually someone will put all the data online for free and meta data of how investors use that alternative data can also be useful.
Now can I interest you in an alternative data feed that will make all your dreams come true?
See also: https://ockhamsnotebook.wordpress.com/2016/09/18/the-hard-thing-about-finding-easy-things/
The ecological consequences of hedge fund extinction
Investing goes through fads. Investing strategies and fund structures(1) go in and out of style. Nowadays long/short hedge funds are out and infrastructure funds are in. Within the public equity markets, value is out, growth/momentum is in. Each time this happens, people forget how the cycle repeats.
In fact, one CIO contended that if he brought a hedge fund that paid him to invest to his board, the board would dismiss it without consideration — simply because it’s called a hedge fund, and hedge funds are bad.Institutional Investor
Hedge funds may have to do a name change if they want to raise capital.
Remember last time?
And yet people forget:
Allocators woke up craving the next rising hedge fund star and couldn’t invest enough at high and increasing management fees after the widespread success of long-short funds in the weak equity markets of 2000-2002. Board rooms back then castigated CIOs for not having long-short equity hedge funds in their portfolios.
This isn’t the first time:
People forget that 40 years ago, officials such as Paul Volcker of the Federal reserve wanted an active hedge fund industry to absorb the risk that was not well managed by state-insured banks.Financial Times
Each investment strategy picks up a certain type of risk(and potentially earns a profit in doing so)- if a strategy disappears that particular risk can become a systemic issue. Fortunately, around this time it also becomes more lucrative to bear the risk others are unwilling to bear. Eventually the risk reward tradeoff starts to make sense again.
Different, different, yet same
In the 1960’s Warren Buffett put up ridiculous returns, and Alfred Winslow Jones proteges profitably exploited anomalies in markets. By the mid 1970’s of there were many articles about hedge funds shutting down though. Industry AUM declined ~70% peak to trough. Nifty fifty boom and bust followed by the long nasty bear market. But as the institutional architecture of international trade and currency shifted we entered glory years of global macro/commodities traders. Then the 80’s were great for Graham deep value and Icahn style activist investing after the 70’s bear market left a huge portion of the market selling below liquidation value.
Likewise late 90’s again saw the death of hedge funds as day traders in pajamas earned easy returns from the latest dot-com- until the crash. Yet out of the rubble of the tech bubble rose a new generation of great hedge fund managers. There was rich pickings for surviving value hunters- and those with the guts and skills to execute became household names a few years later. Many value managers that nearly went out of business during the tech bubble put up ridiculous numbers 2000-2002 and through the next financial crisis. (See: The arb remains the same)
The greatly exaggerated death of a style gives rise to an environment where there is a plethora of opportunities for something similar to that style to work. Each time the narrative in the greater investment community favors some type of uniform strategy, and LPs give less capital to other strategies- causing them to nearly die off. But then the lack of people pursuing the out of fashion strategy makes its return potential more lucrative. Eventually someone finds a new method to pick up those dollar bills on the ground that shouldn’t exist.
Economics emphasizes rational actors and equilibrium. Yet the messy reality is far more complicated. Ecology is a far more useful mental model.
A giant self over-correcting ecosystem
There is in ecological function to speculative capital and over time there should be some excess returns to those willing to take mark-to-market lossesFinancial Times
Like biological species, financial strategies can have competitive, symbiotic, or predator-prey relationships. The tendency of a market to become more efficient can be understood in terms of an evolutionary progression toward a richer and more complex set of financial strategies.Market force, ecology and evolution
Ecology emphasizes interrrelationships between different individuals and groups within a changing environment, and indentifies second order impacts.
Thinking like a biologist
One can develop a useful framework by replacing species with strategy, population with capital, etc
Flows and valuation interact, self correct, and overshoot.
….capital varies as profits are reinvested, strategies change in popularity,and new strategies are discovered. Adjustments in capital alter the financial ecology and change its dynamics, causing the market to evolve. At any point in time there is a finite set of strategies that have positive capital; innovation occurs when new strategies acquire positive capital and enter this set. Market evolution is driven by capital allocation.
Market evolution occurs on a longer timescale than day-to-day price changes. There is feedback between the two timescales: The day-to-day dynamics determine profits, which affect capital allocations, which in turn alter the day-to-day dynamics. As the market evolves under static conditions it becomes more efficient. Strategies exploit profit-making opportunities and accumulate capital, which increases market impact and diminishes returns. The market learns to be more efficient.
When an ecoystem is overpopulated with a certain species, it eventually overshoots and results in mass starvation. Populations fluctuate wildly across decades, and sometimes species go extinct or evolve into something that seems new.
New conditions give rise to new dominant species.
George Soros on disequilibrium analysis
The arb remains the same
Investing: The Last Liberal Art
More Money than God
(1) Although I am frequently pedantic about the differences between structure, strategy, and sector, many in the media seem to use these interchangeably when discussing reversion to mean situations. Fortunately they all exhibit the same boom/bust phenomenon, so I am using them interchangeably here.
The arb remains the same
Imagine if Warren Buffett of 1960 puts down the deadtree 10-K he got in the mail and time travels forward to 2019. Then he looks over the shoulder of an analyst at present day O’Shaughnessy Asset Management. He would find the scene unrecognizable.
Or, if the original Jesse Livermore time traveled from the 1920s stock exchange to the present day trading floor of DE Shaw or Renaissance. Again, completely unrecognizable.
Back in the day people went to the SEC office in the Washington DC to access annual reports faster. That was how one got a fundamental edge. Now people scrape filings the minute they come out. Or use satellites and credit card data to get an edge on information before it hits regulatory filings. People used to gauge momentum by looking at the facial expressions of other traders, now they use complex computer models. People mine market and fundamental data around the globe looking for a bit of an edge. New techniques, same thing.
Over time there is the change in the physical activities, and words we use to describe the process of identifying and exploiting market inefficiencies. Nonetheless the ecological function is the same. Investors are just looking for mispriced risk, and exploiting it till its no longer mispriced.
Around the world there are unfair coins waiting for someone to flip them. Arbitrageurs will need to use weirder and weirder methods to find and exploit them. Methods change, but the arb remains the same.
The hard thing about finding easy things
Riches among the ruins
Books I kind of liked in 2018
- Technological Revolutions and Financial Capital
- Capital Returns
- Modern Monopolies
- How the Music Got Free
- Laws of Human Nature
- Debt’s Dominion: A History of Bankruptcy Law in America
- Thinking in Bets: Making Smarter Decisions When You Don’t Have All the Facts
- The Coffee Trader
- Three Body Trilogy :
- Pattern Recognition
Technological Revolutions and Financial Capital did more to advance my understanding of the modern economy than any other book I read in 2018. Traditional economics treats sudden changes of technology as a separate exogenous factor. This mode of thinking is pretty much useless for anyone forced to allocate capital under uncertain conditions. In reality there is strong empirical evidence that financial markets and technology interact in repeated cycles . Carlota Perez shows the pattern in these cycles by breaking them down into Installation and Deployment Phases. She identifies four main parts of each technological cycle: Irruption, Frenzy, Synergy and Maturity. The book illustrates these patterns in the most important disruptions in economic history, including: the industrial revolution , steam engines/railways, steel/electricity/heavy engineering, automobiles/mass production, and information/telecom. Marc Andreesen hailed Technological Revolutions and Financial Capital as the single best book for understanding the software industry. Since software is eating the world, its probably also the best book for understanding any industry.
Capital Returns is about how competitive advantages change over time, and how changes in capital availability can alter industry dynamics. It covers a fund manager’s thoughts and activities from 2002- 2015, including periods of disruption and volatility in a variety of industries. Ultimately the book is about how mean reversion occurs and and impacts investors.
Modern Monopolies deserves most of the hype that it has received. Many business decision makers were schooled in “linear” business models, but the most valuable and disruptive businesses are platforms, business model that facilitate the exchange of value between two or more user groups, a consumer and a producer. This book covers how technology has enabled more platform business models, and what the implications are for investors and entrepreneurs.
How the Music Got Free tells the story of how streaming music and piracy upended the music industry. It also covers the story of how the MP3 format struggled to gain acceptance for many years before becoming ubiquitous. The contrasting attitudes and actions of the winners and losers in the story are an entertaining juxtaposition , and a useful case study. As a teenager I had been one of the early pirates, but was less aware of what was going on inside the old line music companies, and had not zoomed out to consider the broader industry implications.
Laws of Human Nature is an essential guide for understanding how people act and how society functions. It teases out key lessons from psychology along with successes and failures throughout history. Reading one Robert Greene book provides a reader with the benefits of reading dozens of business biographies and history books. It provides ideas for avoiding certain problematic tendencies in oneself and exploiting traits in others. Laws of Human Nature looks at the dark reality of patterns in thought and behavior that don’t conform to an idealistic rational expectation:
As long as there are humans, the irrational will find its voices and means of spreading. Rationality is something to be acquired by individuals, not by mass movements or technological progress. Feeling superior and beyond it is a sure sign that the irrational is at work.
Debt’s Dominion: A History of Bankruptcy Law in America, is a bit wonkish, but well worth the slog for the insight into the American bankruptcy system. Its easier to understand the subtle nuances of the bankruptcy process and navigate opportunities in the distressed debt market when I think about how the system evolved. Indirectly the book is also about the slow process of institutional change.
Thinking in Bets is basically a long essay on behavioral economics, with one valuable message. Under a plethora of entertaining anecdotes about professional poker it provides an immediately applicable framework for functionning in an uncertain world. Basically its a slightly less nerdy and less nuanced companion to Fortune’s Formula. It also fits in well with some of the more important behavioral finance books, such as…. Misbehaving, and Hour Between Dog and Wolf, Kluge, etc. (more notes here)
The Coffee Trader is historical fiction taking place in 17th century Amsterdam, when coffee was first brought to Europe and the modern concept of stock and derivative markets were just beginning to form. It tells the story from the vantage point of Miguel Lienzo, a small time speculator who must navigate complex social structures, and deal with shady counter parties and aggressive creditors and competitors while trying to get rich in a nascent market. He is also a Jewish refugee who fled the Portuguese inquisition but then struggles with overly conservative Jewish leaders in Amsterdam, who constantly threaten to shun him. Complicating matters further, he has a conflict with his brother, who is at the beginning a far more successful and respected member of the community.
The first person narration includes gems like this one:
There was no shortage of my kind in Amsterdam. We were as specialized as taverns, each of us serving one particular group or another: this lender serves artisans; that one, merchants; yet another, shopkeepers. I resolved never to lend to fellow Jews, for I did not want to travel down that path. I would not want to have to enforce my will on my countrymen and then have them speak of me as one who had turned against them. Instead, I lent to Dutchmen, and not just any Dutchmen. I found myself again and again lending to Dutchmen of the most unsavory variety: thieves and bandits, outlaws and renegades. I would not have chosen so vile a bunch, but a man has to earn his bread, and I had been thrust into this situation against my will.
Three Body Trilogy is a deep character study of humanity itself in the form of an epic science fiction story spanning multiple centuries. It ultimately focuses on the way people act in situations of duress and conflict, both within and between different countries and groups. The Three Body Problem starts slowly, opening during the Chinese Cultural revolution, and tells of the initial contact with alien civilizations. The Dark Forest explores the paradoxical alignment of incentives in high stakes inter galactic diplomacy. The ultimate conclusion for the future of civilization in Death’s End is pessimistic, but getting there is thrilling.
Pattern Recognition is a thriller set in the early 2000s that feels like a science fiction take on modern media. The protagonist is a branding consultant/corporate spook who gets sent on a mission to uncover the anonymous creator behind a series of video clips that have spawned an obsessive subculture of fans. She tracks people and clues down in London, Tokyo and Moscow meeting many bizarre and unseemly characters along the way.
As with all Gibson novels, the dialogue and descriptions are fantastic.
“Of course,” he says, “we have no idea, now, of who or what the inhabitants of our future might be. In that sense, we have no future. Not in the sense that our grandparents had a future, or thought they did. Fully imagined cultural futures were the luxury of another day, one in which ‘now’ was of some greater duration. For us, of course, things can change so abruptly, so violently, so profoundly, that futures like our grandparents’ have insufficient ‘now’ to stand on. We have no future because our present is too volatile.” He smiles, a version of Tom Cruise with too many teeth, and longer, but still very white. “We have only risk management. The spinning of the given moment’s scenarios. Pattern recognition.
Ideas, people and detachment
“To Destroy the Ego,
One Must First Find it”
-Wu Hsin, Aphorisms for Thirsty Fish
One ego suspension trick I find useful is to separate myself from my ideas. Once I think of and share an idea I mentally treat it as this thing that is different and separate from me.
So if someone says the idea sucks- I can view their feedback objectively. They’re not attacking me. The way forward is not a choice between “my” idea and someone elses idea. Its just a choice between ideas.
The same goes for actions already completed. If someone questions a move I made in backgammon, or someone I hired at my company, or a major life choice, I can view it objectively and potentially improve in the future.
Its kind indirect way to achieving the benefits of Kipling’s call to:
“… trust yourself when all men doubt you
But make allowance for their doubting too. “
I just make it easier by stepping outside and not being attached to the “you”. The criticism and criticized are in the distance.
Sometimes this leads to better insight. There is no need to preserve an identity that may be attached to a particular idea. Just find the idea that is better according to some criterion and go there.
This isn’t a perfect system. There is only human software operating here. But it helps. Its a cruder version of what many great thinkers in the past have known.
Borrowing the east wind
Around 200 BC Chinese strategist Zhuge Liang first used a sales trick still re purposed by consultants and lawyers today.
Zhuge Liang was an amateur meteorologist, and he used this fact to convince people that he could control the weather. His knowledge of meteorology was very basic, something any farmer who paid attention would have known. Nonetheless his enemies didn’t have this knowledge. So it was easy to bamboozle them.
During one battle , he realized that the wind was likely to switch direction in a manner that was highly favorable for his army.
He made sure the enemy saw him do an elaborate ceremony that looked like black magic. He kept at it until the wind changed direction. As a result his reputation as a fearsome indispensable strategist grew massively.
This was featured in the historical fiction Romance of the Three Kingdoms . The phrase “Borrow the East Wind (借东风) refers to this story. Its sometimes used to described taking advantage of a situation.
A bit of dancing, drumming and smoke. Zhuge Liang took basic observation skills and sold them as black magic.
Modern knowledge work
I think of this anytime I see a knowledge worker selling their work makes it look more complex than it really is.
Jargon, chartporn and powerpoint replaces dancing drumming and smoke. Or alternatively with legal and compliance work, fear of regulatory risk leads to a company paying high fees to avoid problems. Even if all that is needed is filing a simple form at the right time.
There is a risk of a similar phenomenon in any business where there is a huge knowledge gap between seller and customer. Will the seller take advantage of that gap in a way that harms the buyer?
It may seem like there is one fundamental problem with this comparison: Zhuge Liang was a diplomat/military strategist. A sales call isn’t a war. Its not supposed to be adversarial!
That might actually be the problem. An honest sales process is about helping the client see the value. The battle is against any misperception not against the client. A dishonest sales process is about taking as much from the client as possible.
Zhuge Liang’s life was on the line. And warfare (against sentient opponents) is all about deception. Deceiving competitors is justifiable. But deceiving customers is not. Some businesses may feel their life is on the line, but I bet they could make a good living by reducing complexity rather than playing it up. I know I’m willing to pay up for reduced complexity!
Dealing with this issue has proved to be a major challenge in dealing with lawyers, compliance consultants and technology contractors. I’ll ask around and get quoted absurdly large price ranges for the same set of work.
I’m getting better at asking the right questions in order to see what services are really worth.
I place great value on lawyers, consultants and developers who can cut through the bullshit.
Antifragile Morning Routine
The idea of having “morning routine” is a favorite topic on the interwebs these days. Its now at the point where I can barely tell apart parody from serious articles. I think Tim Ferriss started it all by asking every single one of his guests their morning routine. Most people couldn’t resist listening. I know I couldn’t.
When a “high performer” provides a detail on their morning routine, they are merely providing an example of a thing that works for a very specific set of circumstances. Nothing more nothing less. Just because it worked for them, doesn’t mean it will work for you.
Nonetheless it is still a useful data point. Weird low cost ideas are usually worth trying.
Eliminating decision fatigue
I view morning routines as constrained optimization problem that will have different solutions for different people. It might even have different answers on different days for the same person.
I’m generally skeptical of anything overly complicated. The key benefit of a morning routine is it reduces decision fatigue. The morning routine is a “default setting” . But if one is obsessed with following it, it becomes a burden and prevents serendipity.
Towards an antifragile morning routine
The ability to adapt to changing circumstances is more important than having a set routine. Some days you have to work late. Other days you may have an early meeting. Or maybe your child gets sick. What if your spouse or mistress needs your support? What do you do when you travel or are awoken by a phone call? What then?
It doesn’t make sense to rigidly expect the world to conform to your plan. A fragile morning routine is worse than no morning routine. A robust morning routine is better. An antifragile morning routine is best.
Personally I have a “default setting” to do when I’m home and nothing special is happening. I have a handful of different algorithms I can enact depending on other circumstances that come up. This gives me the comfort of a routine without rigidity. If I’m travelling I’ll usually explore the local area just a bit.
Sometimes an emergency leads to a new insight.