Taking one deliberate step back from a stable position often yields much larger gains.
I’ve long had a vague sense that certain ultimate meta-principles exist in the world — not confined to any single discipline, but appearing like a hidden structure that repeats itself in different forms across different domains. Investing, entrepreneurship, technology, relationships, personal growth, even the rise and fall of civilizations — at their core, they’re all playing the same game.
Recently, I’ve come to identify one of them:
Taking one deliberate step back from a stable position often yields much larger gains.
In other words, short-term retreat is sometimes the necessary path a system must travel to reach a higher equilibrium. This is an idea about local optima and global optima.
I want to trace it through a single thread: start with how a person makes a single decision in a single moment, then zoom out to how a life unfolds over years, then push further to how a civilization rises and falls. The same structure appears at all three scales.
I. Local Optima: The Elegant Trap
From a systems perspective, we always want to reach the highest mountain. But for any individual, constrained by limited knowledge and information, we almost always stop at a local peak.
What makes local optima dangerous isn’t that they’re bad — quite the opposite. They’re too comfortable, too efficient, too correct. So we mistake this hill in front of us for the entire world. Viewed from above, it might be a small hillock.
The real problem has never been failure. It’s: reaching a place that’s already successful enough, and losing the capacity to keep exploring.
II. A Single Decision: The Courage to Descend, and What It Costs
Zoom all the way in: a person standing at a peak, facing a choice that looks worse than staying put.
Simulated Annealing: The Misunderstood Art of Deliberate Choice
In ordinary optimization algorithms, a computer is like a blind mountain climber: keep moving as long as the next step is higher; refuse to move the moment it requires going downhill. So the algorithm gets permanently stuck on the first peak it finds. This is the fundamental limitation of greedy algorithms — they can only improve, never worsen.
Simulated Annealing offers a more sophisticated wisdom: it allows the system to accept a worse outcome with some probability.
But there’s a detail here that’s often overlooked — and it’s the most interesting part. The algorithm doesn’t accept worse solutions randomly or recklessly. According to the Metropolis criterion, the worse a choice is, the lower the probability of accepting it; and the higher the current “temperature,” the more tolerance there is for inferior solutions. In other words, the system isn’t jumping around blindly — it’s making probabilistic trade-offs about how much worse it’s willing to go, governed by a temperature schedule. The worse the choice, the harder it is to accept — but it’s never categorically off the table.
Early in the algorithm, temperature is high and the system has enormous tolerance for error. This maps onto: youth, time remaining, chips still in hand, low cost of experimentation. So it can boldly accept attempts that look absurd or even foolish — switching industries, starting a company, crossing disciplines, going somewhere completely unfamiliar. Only large-enough perturbations can knock the system out of its existing cognitive basin.
As the temperature cools, the system matures. Resources begin to converge. The proportion dedicated to exploration should decrease. You can’t spend a lifetime going downhill. Eventually, the system locks onto the real peak and begins climbing in earnest.
Exploration vs. Exploitation: The Other Side of the Same Coin
Annealing gives us the mechanism. Decision science gives us the trade-off. There’s a famous tension in reinforcement learning: Exploration vs. Exploitation.
Exploitation means extracting what’s already known — certain, efficient, immediate reward. Exploration means seeking what’s unknown — inefficient, messy, full of apparent losses. But both carry their own curse.
Perfect exploitation kills the future: the better a system gets at exploitation, the harder it becomes to absorb new information, until it gets locked in place by its own success. Pure exploration collapses value to zero: if you’re always drifting, always experimenting, nothing new ever settles into understanding, and your life becomes a page that’s perpetually refreshing.
So the optimal solution was never to pick one. It’s to dynamically switch between exploration and exploitation. The best decisions measure not just current reward but current reward plus the expected value of future optionality — maximizing total utility across time. This is exactly what the temperature schedule in annealing accomplishes: explore more when young, exploit more when mature.
III. A Life: Choosing to Descend at the Summit
Zoom out further. If the previous section was about a trade-off at a single point in time, this section is about stretching that trade-off into a curve spanning years — when going downhill is no longer a momentary decision but a valley you have to pass through.
Charles Handy proposed the famous theory of the Second Curve. The first curve represents what you’ve already built: growth, cash flow, reputation all rising continuously. But the danger arrives precisely here — you’re approaching local optima.
The genuinely hard part: the second curve must begin while the first curve is still at its peak, not after it collapses. Which means you have to deliberately redirect resources toward something that looks like a step backward — at the very moment when your income is highest, your recognition is greatest, and you’re most comfortable.
The result is a J Curve. From the outside, all anyone sees is declining income, declining efficiency — maybe they wonder if you’ve made a mistake. But what’s actually happening is that you’ve temporarily left one mountain and are crossing the valley between two peaks. Once the second curve passes its inflection point, its growth rate and eventual height will far exceed the first. That’s the leap.
IV. A Civilization: Why Economies Must Go Through Recession
Push the lens to maximum zoom. The same structure, at civilizational scale, manifests as the boom-and-bust of technology cycles — what was individual fear becomes industrial bankruptcy and unemployment.
Schumpeter called it Creative Destruction. When each technology cycle matures, capital, labor, and institutions all gradually get locked into the old system. The entire economy reaches a highly efficient local optimum.
Then a new technology arrives. AI, the internet, electricity, the steam engine — what they bring first is usually not prosperity but disruption: old industries declining, companies failing, unemployment rising, economic data deteriorating. In the short run, everything seems to be getting worse.
But this disruption is precisely what releases resources — capital flows toward higher-efficiency uses, and eventually a new, higher level of prosperity forms. So in many cases, recession isn’t a sign that the system is broken. On the contrary: recession itself is part of the system’s upgrade.
Note the distinction between this level and the previous one: the Second Curve is a descent that an individual or organization actively initiates. Creative destruction is a disruption that civilization passively endures. Same mechanism — one held in your own hands, one pushed by history.
V. Life Is Not a Straight Line, but a Series of Annealing Processes
Bring the lens back to yourself. If you stretch the time scale long enough, maybe life isn’t a continuously upward line at all — but a series of cycles:
Rise → Stabilize → Deliberate disruption → Descent → Leap
Many phases that look like failure are in fact necessary turbulence within the annealing process. Those anxiety-inducing moments — taking a pay cut to change careers, leaving a familiar environment, giving up an identity you’ve built, starting from zero in something new — they look like losses. But from a systems perspective, they may simply be the cost of gaining greater degrees of freedom.
The core logic of simulated annealing is extremely simple. You can write it in a few lines of code. But lived in a human body, the hardest part has never been the calculation — it’s the emotion.
The human brain is wired to hate uncertainty. So when we’re about to deliberately walk downhill, the amygdala fires its alarms — anxiety, reluctance, the ache of what we might lose. So we’d rather slowly age out at our local optimum than endure the temporary chaos of something unfamiliar.
This is why knowing is easy and doing is hard. Because what blocks us has never been logic — it’s fear.
So perhaps the real art of growth isn’t always moving forward — it’s knowing when to keep climbing, when to stop optimizing, and when to deliberately descend.
Because sometimes stepping back isn’t a compromise. It’s trading time for space. A local retreat may be exactly the path to something greater.
Like simulated annealing — on a long enough time scale, truly great systems are never afraid of temporarily getting worse. Because they know:
That’s not the end. It’s the entrance to the next mountain.