Why Most Things Fail

1. Failure is pervasive and inevitable in complex systems

Failure is all around us. Failure is pervasive. Failure is everywhere, across time, across place and across different aspects of life.

Ubiquity of failure: Failure is an inherent characteristic of complex systems, whether in nature, business, or society. 99.9% of all biological species that have ever existed are now extinct. In the business world, more than 10% of all companies in America disappear each year. Even large corporations are not immune to failure, with only 28 of the world's top 100 companies in 1912 surviving and growing by 1995.

Complexity breeds failure: The sheer complexity of interactions within systems makes failure inevitable. In biological evolution, random mutations drive change, but most mutations fail. In business, companies constantly adapt strategies, but most fail to achieve desired outcomes. This pervasive failure is not due to lack of effort or intent, but rather the inherent uncertainty and unpredictability in complex, interconnected systems.

2. Intent does not guarantee outcome in decision-making

Intent is not the same as outcome.

Uncertainty in decision-making: Despite the ability of humans and organizations to act with purpose and intent, the outcomes of their decisions are often unpredictable. This is due to the complex, interconnected nature of the systems in which decisions are made.

Examples of failed intent:

• Microsoft initially focused on OS/2 as the future of computing, nearly abandoning Windows
• RJR Nabisco's CEO misjudged the consequences of his attempted management buyout
• The collapse of Long-Term Capital Management, despite its Nobel Prize-winning founders

The gap between intent and outcome highlights the limitations of rational planning and strategy in complex environments. Even with the best information and analysis, the future remains inherently uncertain.

3. Power laws govern extinction patterns in biology and business

The frequency with which we see extinctions of any given scale falls away with the square of the size.

Power law distribution: Extinctions in both biological species and businesses follow a power law distribution. This means that while small extinction events are common, large-scale extinctions, though rare, occur more frequently than would be expected in a normal distribution.

Key characteristics of power law distributions:

• Scale-invariance: The relationship between frequency and size holds across different scales
• Fat tails: Extreme events are more likely than in a normal distribution
• No typical scale: There is no "average" extinction size that characterizes the system

This pattern suggests that both biological and economic systems operate far from equilibrium, in a state of self-organized criticality. Small perturbations can occasionally lead to large-scale effects, making the systems inherently unpredictable and prone to periodic large-scale failures.

4. Conventional economic theories fail to explain real-world phenomena

Vernon Smith noted as much in his acceptance speech, when he stated perfectly frankly, 'We do not understand why markets work as they do.'

Limitations of traditional models: Conventional economic theories, based on concepts like rational actors, perfect information, and equilibrium, fail to explain many real-world phenomena. These models often assume away the very complexity and uncertainty that characterize actual economic systems.

Examples of theoretical failures:

• The Phillips curve: The supposed trade-off between inflation and unemployment has broken down
• Efficient market hypothesis: Stock market behavior exhibits patterns inconsistent with perfect rationality
• Perfect competition: Real markets are rarely, if ever, in a state of perfect competition

New approaches, drawing inspiration from complex systems theory, evolutionary biology, and network science, are needed to better understand and model economic behavior. These approaches recognize the inherent uncertainty, non-linearity, and emergent properties of economic systems.

5. External shocks and internal dynamics both contribute to failure

There is a positive relationship between the size of the shock and the size of the extinction which occurs.

Dual causes of failure: While external shocks can trigger failures, the internal dynamics of complex systems also play a crucial role. In many cases, it is the interaction between external events and internal vulnerabilities that leads to significant failures.

Endogenous vs. exogenous factors:

• Exogenous: External shocks like natural disasters, technological disruptions, or regulatory changes
• Endogenous: Internal dynamics such as competitive interactions, feedback loops, and network effects

Models that incorporate both endogenous and exogenous factors better explain observed patterns of failure and extinction. These models suggest that while large external shocks can cause significant failures, even small perturbations can sometimes lead to large-scale effects due to the internal dynamics of the system.

6. Learning and knowledge acquisition have limited effectiveness

The clear implication of this abstract theoretical model is that agents, firms, individuals, governments have very limited capacities to acquire knowledge about the true impact either of their strategies on others or of others on them.

Limits to learning: While acquiring knowledge can improve survival chances, there are inherent limits to how much agents can learn about their complex environments. Even small amounts of knowledge can significantly improve survival rates, but perfect knowledge is unattainable.

Implications of limited learning:

• Strategies based on past success may fail in changing environments
• Overconfidence in predictive abilities can lead to increased risk-taking
• Adaptability and robustness may be more valuable than optimization

The limited effectiveness of learning underscores the importance of maintaining diversity, flexibility, and resilience in complex systems. Rather than seeking perfect knowledge, agents should focus on developing adaptive strategies that can cope with uncertainty and change.

7. Failure at the individual level can benefit the system as a whole

Failure at the level of the individual component part can, paradoxically, enhance the fitness of the system as a whole.

Creative destruction: Joseph Schumpeter's concept of "creative destruction" suggests that the failure of individual firms can drive innovation and progress in the broader economy. This process allows for the reallocation of resources to more productive uses and creates space for new entrants and ideas.

Benefits of failure for the system:

• Promotes innovation by clearing space for new ideas and approaches
• Enhances overall efficiency by weeding out less productive entities
• Increases system resilience by maintaining diversity and adaptability

Recognizing the potential benefits of failure at the individual level can inform policy approaches. Rather than trying to prevent all failures, policymakers should focus on creating an environment that allows for productive failures while mitigating systemic risks. This approach can lead to more dynamic, innovative, and resilient economic and social systems.

Last updated: October 21, 2024