Every investor I know has a version of the same experience. A piece of news breaks, the stock drops fifteen percent, and within about ninety seconds you have a view.
Not a hypothesis. A view.
You “know” what the news means, you “know” why the market has overreacted or underreacted, and you “know” what happens next.
The speed of it, the speed of arriving at these conclusions, should worry you more than it does.
The Question to Ask Before Believing a Story
This week, I have been trying to build something to slow that moment down. What I wanted was a prompt, a proper one, that would make an AI do the work I know I should do and occasionally skip:
Go and find out what actually happened the last twenty times a situation like this one occurred, count the outcomes, and hand me the distribution before I get anywhere near my own opinion.
I’d argue that in the age of AI specifically, it’s easy to produce something that has the shape of a base rate analysis (and convince yourself that you’ve “done the work”) – a comfortable list of loosely related examples with a (over-)confident percentage stapled to the bottom – without any of the substance proper base rate work would require.
This post is about why that is so hard to avoid, what the research says about the people who avoid it best, and what I eventually built. The prompt is presented at the end of this post, and you are welcome to skip straight to it if that is what you came for (the prompt is behind a paywall, though), but I figured it’s important to build and share the theoretical foundation first: why is it so important to do base rates work and consult the outside view?
Arguably, the prompt is only worth anything if you understand the failure it is designed to prevent, so let me start there.
Here’s what we will cover in this post:
Why more research can make a bad thesis stronger rather than weaker, and why the inside view never feels like bias.
The single best-evidenced forecasting technique from the largest forecasting study ever run
What Tetlock’s superforecasters do that the rest of us do not
Why a phrase like “fair chance” nearly cost Kennedy a war, and what it should change about the way you write your own memos.
Three base rate analyses on real cases (Adobe + 2x Wise), walked through in ascending order of quality, and the specific move each one makes that the previous one does not.
Why the absence of a case is a finding, and why nobody treats it as one.
Then, the full prompt, almost 150 lines of it, built to force a model through the outside view
Here’s a preview:
And here’s a short excerpt from the output on a query I ran on the stock of ResMed and the disruption risk it is allegedly facing:
About Storytelling Animals
You are a storytelling animal. So am I. This is not a metaphor or a flourish, it is the closest thing behavioural science has to a settled finding, and it has a specific and dangerous consequence for anyone whose job involves forming views about the future under conditions of incomplete information. Which is to say, for all of us.
My friend Tiho Brkan put the mechanism better than I have seen it put anywhere else, so let me “hand him the microphone” for a moment. The narrative fallacy, he writes on X, is the illusion of understanding through storytelling. Our brains are pattern-seeking machines, and so we string loose, random events into a comforting chain of cause and effect, and then, despite lacking most of the facts, we treat our limited knowledge as the complete truth. We prioritise a good story over the underlying reality.
That’s why the “price drives narrative” dynamic is so powerful and can be encountered frequently. The “hit rate” is almost 100%. A stock is down 20%, 30%, or 50%? Bearish narratives will emerge and spread; the more a stock is down, the faster and farther the narrative travels. A stock skyrockets? The bullish narratives take over.
Get 3 FREE GIFTS when you subscribe: 📈 Valuation Spreadsheet 📚 eBook: Investing Visualizations 💡 eBook: 250 Thought-Provoking Quotes - Join 4,500+ subscribers.
So why does this pattern exist? Stock markets are auction driven markets, impacted by human behavior. We believe in easy narratives because feeling is faster than thinking. A narrative imposes a far lighter cognitive load than slow, rigorous deliberation. Storytelling is the path of least resistance, and your brain is, above all else, an energy-conserving organ.
Tiho put this powerful quote in the tweet I linked above:
The storytelling mind is allergic to uncertainty, randomness, and coincidence. It is addicted to meaning. If the storytelling mind cannot find meaningful patterns in the world, it will try to impose them. In short, the storytelling mind is a factory that churns out true stories when it can, but will manufacture lies when it can't." - Jonathan Gottschall in his book titled The Storytelling Animal
Markets are the perfect habitat for this.
Tiho makes another observation here, which is that money and finance and investing may be the most successful stories ever invented, because they are the only ones with universal buy-in. Not everyone shares a religion. Not everyone accepts a human rights framework or a national identity. But everybody believes in money.
That collective belief makes markets uniquely hospitable to contagious narratives, and it means the raw material you are working with is already made of story before you add any of your own.
Disclaimer:
The analysis presented in this blog may be flawed and/or critical information may have been overlooked. The content provided should be considered an educational resource and should not be construed as individualized investment advice, nor as a recommendation to buy or sell specific securities. I may own some of the securities discussed. The stocks, funds, and assets discussed are examples only and may not be appropriate for your individual circumstances. It is the responsibility of the reader to do their own due diligence before investing in any index fund, ETF, asset, or stock mentioned or before making any sell decisions. Also double-check if the comments made are accurate. You should always consult with a financial advisor before purchasing a specific stock and making decisions regarding your portfolio.
Why more research does not fix it
Here is where most investors go wrong, and I include myself in earlier years. Faced with the suspicion that they might be telling themselves a story, they respond by doing more work. More filings. More channel checks. More transcripts.
And they end up more confident, more articulate, and no more accurate. More effort doesn’t necessarily lead to more accuracy. Or put differently, the incremental benefits of putting in more effort diminish relatively quickly.
It’s the old 80/20 rule–formally known as the Pareto Principle – which states that roughly 80% of outcomes come from 20% of inputs.
At some point, more effort might even be detrimental as you risk becoming overconfident and clouded in your judgment.
So again, if you rely solely on the inside view, your specific perception of the task at hand, you will inevitably force new evidence to fit a preexisting story. This is what Charlie Munger already observed years ago.
“I want to tell you a story from my days at Harvard Law School.” He said sometimes when they had classes at Harvard Law, the professor would bring up a case where the facts didn’t lead to an obvious conclusion about which side was in the right. It could go either way.
Then they divided the class in two halves randomly. One half would argue for the defendant, and the other half would argue against the defendant. Then the two sides went off and studied the facts and then made their arguments. After all of that was done, they surveyed the entire class. Overwhelmingly, the students who had argued for the motion believed strongly they were right, and the people who had argued against the motion also believed strongly they were right. Before they had studied the facts, they didn’t lean one way or the other. […]
Charlie said that even the temples and churches make you repeat stuff because as you shout it out, you pound it in. The best-ideas fund, Charlie said, were simply the picks the managers had spent the most time on. When they spent the most time on these ideas, they were the most excited about them. When they put all of these ideas together, things didn’t go so well.
In Poor Charlie’s Almanack, he talked about how the human mind is a lot like the human egg. Once the first idea gets in, just like the human egg, it locks up and seals off any additional ideas from coming in. You have what he calls commitment and consistency bias where we get locked into what’s taken hold in our brains. We see this even in political discourse. If you talk to folks who love Donald Trump, they can’t see anything wrong with him. If you talk to folks who are on the other side, they can’t see anything right with him. And then of course, the reality is probably some shade of gray in the middle there. Warren Buffett also has a great quote. He says what human beings are best at doing is interpreting new information so that their prior conclusions remain intact.“ - MOI Global
If the story is attractive, you develop an affinity for it. If it is not, you discard the story and the evidence with it.
This is what makes the inside view so treacherous.
I am not arguing that the inside view is worthless. Not at all! This post is more about combining both the inside and the outside view to increase to accuracy of your judgment.
Put differently, the problem is not that we use the inside view. The problem is when we reach for it, which is mostly first, and what it does to how we assess everything we look at afterwards.
Becoming A Superforecaster!
If you have read Michael Mauboussin on base rates, and most serious investors have, you may not know how far upstream the idea travels. Philip Tetlock is one of Mauboussin’s mentors, and a great many of the ideas Mauboussin writes about came directly from him.
Tiho made this point to me in a private exchange, and it reframed the whole subject. Mauboussin is the man who translated this thinking into the language of security analysis, and he did it superbly, but the underlying research is Tetlock’s, and it comes from a domain far harsher than equity markets.
Tetlock and Barbara Mellers ran a forecasting tournament for the US intelligence community, over four years, with more than five thousand participants making predictions about geopolitical and economic events. The questions were the sort that have no clean answer and no reliable model.
Will these protests spread?
Will this government fall?
Will this index close above a given level by a given date?
Out of that pool, they identified two hundred and sixty people who were consistently, measurably better than everyone else. Tetlock called them superforecasters, and the cutoff, the top two percent, was one he drew himself rather than one that emerged from the data.
What emerged from the data was more interesting than the label: the people in the top two percent tended to stay in the top two percent from one year to the next.
Roughly seven in ten held their status year over year, and the correlation between a forecaster’s performance in one year and the next ran at 0.65.
Skill, in other words.
Not a run of luck that reverted.
What they do differently
The tournament forced participants to record which techniques they used for each forecast, which means we do not have to speculate about what made the good ones good. We can look.
Of all the methods forecasters could apply, the one most strongly associated with accuracy was the use of comparison classes. Forecasts tagged with that technique averaged a Brier score of 0.17. The next-best technique averaged 0.26. Some techniques, incidentally, correlated with worse accuracy rather than better.
Comparison classes is simply another name for reference-class forecasting, which is another name for the outside view.
So the single best-evidenced forecasting technique in the largest forecasting study ever run is the same discipline Mauboussin has spent two decades trying to get portfolio managers to adopt, and most of us still treat it as an optional garnish on the analysis rather than the foundation of it.
The Sequence Matters
Here’s another observation that Tiho stressed and that made me reorganize how I think about the problem, and it is the reason I bothered to build a “base-rate prompt” at all (more on this further below):
Superforecasters strike the right balance between the inside and outside views, yes.
But more importantly, they do it in the right order.
They anchor first on the outside view, the base rate of the applicable reference class, …
… and then, and only then, do they adjust using the inside view.
Sequence, not weighting.
Once you see it, the failure mode in your own process becomes embarrassingly visible.
Most good investors do not skip the outside view entirely. What they do, however, is form their (subjective) inside view first, then go looking for a reference class, at which point the reference class may be prone to be influenced by biases too. Think back to what Charlie Munger so vividly described above:
You want the answer to be that disruption is unlikely? So you may (unconsciously) assemble a class of companies for which disruption is unlikely.
And you might not even catch yourself rigging the result, because at no point did you knowingly cheat.
You just did the steps out of order.
The inside view wants to go first, and it wants the base rate brought in afterwards as a supporting witness. But you need to reverse the process.
Say a number …
There is a second habit that runs through the superforecaster group and it follows naturally from the first. They talk in probabilities rather than in words.
We normally express uncertainty with vocabulary.
“A significant chance."
“Almost certainly.“
“A real possibility.“
Tetlock argues for numbers, and he tells a story that should be pinned above the desk of anyone who writes investment memos. In 1961, with the CIA preparing to land a force of Cuban expatriates at the Bay of Pigs, Kennedy asked the military for an unbiased assessment. The Joint Chiefs concluded the plan had a “fair chance” of success.
The officer who wrote those words later said he had in mind odds of three to one against. Kennedy was never told what fair chance meant, and understood it, not unreasonably, as encouragement.
Three to one against. Encouraging. Same phrase. Oops.
The standard objection to numerical forecasting is that the numbers are themselves guesses, so all you have done is dress up your uncertainty rather than reduce it. For someone with no practice, that is fair. It stops being fair for people who track their forecasts. If you track your forecast and benchmark it against the actual outcome, it becomes a measurable skill and to improve upon. The only way to acquire it is to start writing numbers down and checking the scoreboard over time.
Once you are working in probabilities, Bayesian updating stops being a theoretical nicety and becomes the natural next move, and this is the last piece of the superforecaster profile.
They make many small updates, with the occasional large one, which is precisely what the theory predicts. They break hard questions into manageable parts and chain the probabilities together properly. They do not, in other words, hold a view and then flip it when the pressure gets too great. They hold a number and move it a little at a time as evidence arrives.
Below, you find a 7-step framework trying to describe how superforecasters, according to Tetlock, go about making their predictions:
Fermi-ise the question: A problem that resists a direct answer will often yield if it is broken apart first, either into components or into distinct pathways to the same outcome, each of which can be attacked with more rigorous methods than the whole.
Consider my recent Dino Polska deep dive for instance. Instead of asking “how much will revenue grow over the next five years” and slap a number on it, I broke the question down into the four key growth drivers going forward (New Store Network Expansion, Like-for-Like Organic Growth, The Maturation Catch-Up Effect, New Verticals and E-Commerce) and assigned a range (!) of expected revenue growth contribution to each.
This is where it gets interesting!
Become a paying subscriber to read the rest of this post and get access to all of my other research, including valuation spreadsheets, deep dives (e.g. well-known mid- and large caps such as LVMH, Duolingo, Meta, Edenred as well as more hidden gems such as Tiger Brokers, Digital Ocean, Ashtead Technologies, InPost, Timee, and MANY more), and powerful investing frameworks.
