培养你的判断力：提升决策质量的框架

GLOBAL FINANCIAL STRATEGIES www.credit-suisse.com

Cultivating Your Judgment Skills A Framework for Improving the Quality of Decisions May 7, 2013

Authors

Michael J. Mauboussin [email protected]

Dan Callahan, CFA [email protected]

National Archives image 594412 (241 Patent 95,513).

Thoughtful investing requires having some view of the future.

The goal of this report is to provide some tools and guidance to improve decisions on a day-to-day basis.

Good judgment requires understanding causality, effectively incorporating information from past events to understand present prospects, and updating probabilities correctly based on the arrival of new information.

We can all improve across each of these facets of decision making, but the fact is that few of us move past a functional and comfortable stage.

Philip E. Tetlock, a professor of psychology and management at the University of Pennsylvania’s Wharton School, is probably best known for his book Expert Political Judgment. That book reported the findings of Tetlock’s extraordinary study of expert predictions in political, social, and economic realms. He found that experts in these fields are not particularly good at predicting outcomes, and that they present the same psychological defense mechanisms as the rest of us when confronted with evidence of their futility. Knowing the limitations of forecasters is useful, of course, but he is also concerned with how to improve judgment.

Tetlock recently taught a course called “Cultivating Your Judgment Skills: The Art and Science of Confidence Calibration in Business, Politics and Life.” In that course, he presented the students with a matrix that offers a useful roadmap to improving judgment (Exhibit 1 is a modified version). This matrix is of great utility for any investor or businessperson.

Exhibit 1: Philip Tetlock’s Judgment Matrix

Using reference Integrating new Identifying causality classes information

Stage 1 Cognitive Casually track news, Superficial Stuck in inside view Beginner, error-prone over- and underreact

Read news closely, Stage 2 Associative Overcompensate/ Point-counterpoint focus on diagnostic Good enough, fewer errors stuck in outside view information

Nuanced weighting of Accurately update Stage 3 Autonomous Shrewd integration inside and outside based on new Elite, near perfection views information

Source: Based on Philip Tetlock, “Honing Skills in Forecasting Tournaments: The Art and Science of Good Guesswork,” from the course, Cultivating Your Judgment Skills: The Art and Science of Confidence Calibration in Business, Politics and Life, lecture delivered on January 16, 2013. Used by permission.

The process of acquiring a skill follows three stages. First is the cognitive stage, where you simply try to understand the activity and are therefore prone to errors. You might recall the first time you drove a car. Each aspect of the experience required you to exert explicit mental effort, including starting the engine, putting the car in gear, steering, and braking.

Second is the associative stage, where performance improves noticeably and errors come less frequently.

Now you can drive without much effort and no longer represent a menace to society.

The final stage is the autonomous stage, where the skill becomes fluid and habitual. Only practiced drivers who can adroitly handle any driving condition, no matter how extreme, find their way to this stage. Our skill acquisition generally reaches a plateau in the associative stage—we stop when we’re good enough. But elite performers in all fields push through to the third stage.

Tetlock’s path to good judgment loosely follows these three stages and, similar to most drivers, the majority of us don’t get to the third stage of decision making. (See the rows of Exhibit 1.)

In stage 1, our understanding of what we’re trying to forecast is somewhat superficial and casual. In stage 2, we develop a deeper understanding of the topic but lack sufficient nuance—a key element in keen judgment.

Forecasters who make it to stage 3 have developed robust mental models to allow them to gain a differentiated point of view. Getting to stage 3 decision making is not natural. You have to overcome your mind’s natural laziness, which requires training, effort, and discipline.

Tetlock suggests that these stages should be cultivated across three aspects of good judgment (see the columns of Exhibit 1). The first is “identifying causality,” or what he calls the “argument map.” Next is what he calls “reference classes,” an ability to blend the specifics of the case in question with an appropriate reference class to assess proper probabilities and outcomes. Finally, there is the “integration of new information,” or what Tetlock refers to as “Bayesian updating,” which considers one’s effectiveness at updating views based on new information.

We will now walk through each of the aspects of good judgment, offering some ideas on how to improve along these dimensions.

The Theory of Theory Building

At its core, a theory is an explanation of cause and effect. We all walk around with theories in our head, whether or not we are aware of them explicitly. The goal, then, is to improve the theories that we employ.

Clayton Christensen, a professor of management at Harvard Business School best known for his theory of disruptive innovation, has written about the theory of theory building.1 His discussion of theory building fits well with the stages of improvement under the column “argument map” in Tetlock’s matrix.

Christensen describes his theory of theory building in three steps.

The first step is observation, which includes observing the phenomena at hand and carefully measuring and describing results. This allows researchers to agree on standards so they are all talking about the same issue and are using common terms to describe it.

The next step is classification, where researchers place their observations in categories that allow for clarification of differences between phenomena. Early on, these categories are based primarily on attributes.

The last step is definition, a description of the relationship between the categories and the outcomes.

These relationships are generally described by correlations.

Once a researcher has a theory, he or she then tests its predictions against actual results. That allows for the identification of anomalies, and the theory is reshaped and refined in order to accommodate the anomaly. This refining process leads to two crucial improvements. In the classification step, categories evolve beyond attributes and reflect circumstances. The categories go beyond what works to when it works. In the definition step, the theory goes beyond correlation and identifies true causation. Everyone has heard that correlation does not automatically mean causation. Good theory seeks to comprehend causal relationships.

An example of theory building is the history of manned flight. The first step in developing the theory was examining the animals that could fly. Researchers noticed that almost all of these animals had wings and feathers (observation and classification steps). Further, the correlation between wings and feathers and flight was high (descriptive step), albeit not perfect. There were creatures such as ostriches, which had wings but couldn’t fly, and bats, which had no feathers but could fly.

To test the theory, aspiring fliers built wings, attached feathers, climbed to a tall spot, jumped, flapped, and crashed. The crash was an anomaly in the theory, prompting researchers to go back through the observation-classification-definition steps. There was more to flight than wings and feathers.

Daniel Bernoulli’s studies of fluid dynamics in the 1700s led to the idea of an airfoil, a shape that generates lift by creating decreased air pressure over the top of the wing relative to the air under the wing. This is called Bernoulli’s principle. (If you want to see Bernoulli’s principle in action, cut a piece of paper in a 2 inch by 8 inch rectangle. Hold the paper with one of the short sides just below your mouth. The paper will sag as the result of gravity. Then blow straight out. You will see the paper straighten out.) Manned flight was possible when the Wright Brothers combined their understanding of this principle with materials that allowed for propulsion, steering, and stability. Bernoulli’s principle shows why birds can fly but also explains what causes flight (improved classification and definition).

Outsourcing is a good example of theory building from the world of business. Outsourcing is the practice of contracting a service that was previously done in-house to an outside company. Outsourcing appears attractive because it may allow a company to reduce its costs and invested capital. A number of companies that have been very successful have relied heavily on outsourcing. For instance, Apple generated $165 billion of revenues in calendar 2012 using about $20 billion in invested capital. The correlation between outsourcing and good financial results seems clear.

The experience that Boeing, the world’s largest airplane manufacturer, has had with their newest plane shows the limitation of the correlation between outsourcing and economic profit.2 Boeing has long used suppliers, but its traditional process was to design the plane in-house and then send detailed blueprints and specifications to the suppliers. They called this system “build-to-print.” Critical design features and vital engineering functions were handled by Boeing, but the company lowered its costs by using suppliers.

For its latest aircraft, the 787 Dreamliner, Boeing used a different approach. The company decided to have its suppliers design and build various sections of the plane, leaving only the final assembly to Boeing. Based on the company’s projections, the time to market could be trimmed by a couple of years and the time to assemble a plane of that size would drop from a month to just three days.

The program was a mess. Though the plane enjoyed strong pre-orders, the launch was repeatedly delayed as the suppliers were unable to deliver sections that functioned properly and that were ready for assembly.

Boeing hoped to create a final product by clicking together, like Legos, the 1,200 components that it had ordered. But the first plane came to Boeing in 30,000 pieces, many of which didn’t fit or work together properly. Boeing had to pull design work back in-house, at a substantial cost.

Here’s where it makes sense to pause and consider the key point: As theories improve they rely less on attributes and more on circumstances. Outsourcing as an attribute of a business correlates well with business success. But, as in the case of the 787, the correlation is not perfect. In refining the theory, researchers have figured out when outsourcing works. For example, outsourcing does not make sense for products that require complex integration of disparate subcomponents. This is because when coordination costs are high, simply getting a product to work is a difficult task. In this stage of the industry, vertical integration works best.

But when the subcomponents are modularized, a process that is not trivial, the final assembly is relatively simple and outsourcing can add value.3 Industries can flip from a vertical to a horizontal orientation. Consider IBM in the earliest days of the personal computer. The company made almost all of its own components to make sure the end product actually worked. But as the subcomponents became modules that could be clicked together, companies such as Dell arose to take advantage of the industry change.

We are now prepared to apply the “theory of theory building” to Tetlock’s matrix. In the first stage, the understanding of causal dynamics is weak. In the second stage, an individual can identify multiple sources of causality, but the emphasis tends to remain on attributes. In the final stage, judgment evolves to the point of understanding circumstances—a truer insight into causality.

We all want to know what to do in order to succeed. Many of those who supply advice on success—including academics, consultants, and practitioners—make a very common mistake that prevents them from improving judgment. The mistake is to observe success, identify common attributes associated with that success, and then proclaim that those attributes can lead others to succeed. This approach doesn’t work because it fails to properly sample failure, does not consider circumstances, and often neglects the substantial role of luck.

Beware of stories of success that rely on attributes.

Understanding causality in a complex system is an inherently tricky task. The final stage in Tetlock’s matrix, “shrewd integration,” requires a grasp of how to weigh various factors in order to come to a thoughtful conclusion. The theory of theory building prompts concerted effort to distinguish circumstances from simple attributes.

Inside versus Outside View

When we are asked to make a forecast, such as the growth rate of a company, the return for an asset class, or the performance of a basketball player, there’s a natural and intuitive approach to going about the task. We focus on the issue at hand, gather lots of information, consider some scenarios, and generally extrapolate what we see and think, with some adjustments, into the future. This is what psychologists call taking the “inside” view.

An important feature of the inside view is that we tend to dwell on what’s unique about the situation.4 Indeed, Daniel Gilbert, a psychologist at Harvard University, suggests that “we tend to think of people as more different from one another than they actually are.”5 Likewise, we tend to think of the things that we’re trying to forecast as being more unique than they really are. Not infrequently, the inside view leads to a forecast that is too optimistic, whether it’s the likely success of a new business venture or the cost and time it will take to remodel your kitchen.

The “outside” view, which requires some effort to adopt, considers a specific forecast in the context of a larger reference class. Rather than emphasizing differences as the inside view does, the outside view relies on similarity. The outside view asks, “What happened when others were in this situation before?” Embracing the outside view requires you to step away from the specifics of the situation you are dealing with and to treat the case statistically.

Take mergers and acquisitions (M&A) as an example of these contrasting approaches. The executives at the companies merging will dwell on the strengths of the combined entities, the synergies they expect to materialize, and the specific financial benefits. The uniqueness of the combined business will be front and center in their minds, and not surprisingly they will generally feel good about the deal. That’s the inside view.

The outside view would not ask about the details of the specific deal; it would ask how all deals tend to do. It turns out that about 60 percent of deals fail to create value for the acquiring company. 6 If you know absolutely nothing about a specific M&A deal, the outside view would have you assume a success rate similar to all deals.

Shortly after Daniel Kahneman won the Nobel Prize in Economics in 2002, a colleague asked him which of his 131 academic papers was his favorite.7 He answered with “On the Psychology of Prediction,” a paper he wrote with Amos Tversky that was published in Psychological Review in 1973. The paper argues that there are three types of information relevant to statistical prediction: the base rate (outside view), the specifics about the case (inside view), and the relative weights you assign to each. 8

One way to determine the relative weighting of the outside and inside view is based on where the activity lies on the luck-skill continuum. Imagine a continuum where on one end luck alone determines results—think of roulette wheels and lotteries—and where on the other end skill solely defines the outcomes—such as running or swimming races (see Exhibit 2). A blend of luck and skill reflects the results of most activities, and the relative contributions of luck and skill provide insight into the weighting of the outside versus inside view.

Exhibit 2: The Luck-Skill Continuum

Pure Pure Luck Skill

Source: Michael J. Mauboussin, The Success Equation : Untangling Skill and Luck in Business, Sports, and Investing (Boston, MA: Harvard Business Review Press, 2012), 23.

For activities where skill dominates, the inside view should receive the greatest weight. Suppose you first listen to a song played by a concert pianist followed by a tune played by a novice. Playing music is predominantly a matter of skill, so you can base the prediction of the quality of the next piece played by each musician on the inside view. The outside view has little or no bearing.

By contrast, when luck dominates the best prediction of the next outcome should stick closely to the base rate.

For example, money management has a lot of luck, especially in the short run. So if a fund has a particularly good year, a reasonable forecast for the subsequent year would be a result closer to the average of all funds.

Knowing where you are on the luck-skill continuum tells you a great deal about reversion to the mean, a concept that is frequently misunderstood. Reversion to the mean says that for an outcome that is far from average, the expected value of the subsequent outcome is closer to the average. Where skill is more important, reversion to the mean is modest. Where luck is important, results rapidly revert to the mean. So where an activity lies on the luck-skill continuum tells you a lot about the rate of reversion to the mean.

There are two analytical concepts that can help you improve your judgment. The first is an equation that allows you to estimate true skill: 9

Estimated true skill = grand average + shrinkage factor (observed average – grand average)

The shrinkage factor, represented mathematically by the letter c, has a range of zero to 1.0. Zero indicates complete reversion to the mean and 1.0 implies no reversion to the mean.10 In this equation, the shrinkage

factor tells us how much we should revert the results to the mean, and the grand average tells us the mean to which we should revert.

Here’s an example to make this concrete. Let’s say you want to estimate the true skill of a mutual fund manager based on an annual result. The grand average would be the average return for all mutual funds in a similar category (naturally, these results would be adjusted for risk). Let’s say that’s 10 percent. The observed average would be the fund’s result. We’ll assume 12 percent. In this case, the shrinkage factor would be close to zero, reflecting the high dose of luck in short-term results for mutual fund managers. Let’s call the shrinkage factor .05. The estimate of the manager’s true skill based on the result is 10.1 percent, calculated as follows:

10.1% = 10% + .05(12% - 10%)

The second concept, intimately related to the first, is how to come up with an estimate for the shrinkage factor.

It turns out that the coefficient of correlation, r, a measure of the degree of linear relationship between two variables in a pair of distributions, is a good proxy for the shrinkage factor.11 Positive correlations take a value of zero to 1.0.

Say we had a population of violinists, from beginners to concert-hall performers, and on a Monday rated the quality of their playing numerically from 1 (the worst) to 10 (the best). We then have them come back on Tuesday and rate them again. The coefficient of correlation would be very close to 1.0—the best violinists would play well both days, and the worst would be consistently bad. There is very little reversion to the mean and hence little need to appeal to the outside view. The inside view correctly receives the preponderance of the weight in forecasting results.

Now we can examine the annual performance of mutual fund excess returns. Unlike the violinists, the correlation of excess returns is relatively low. 12 That means that in the short run, returns that are well above or below average may not be a reliable indicator of skill. So it makes sense to use a shrinkage factor that is much closer to zero than to 1.0. We accord the outside view most of the weight in our forecast.

Three researchers, Dan Lovallo, Carmina Clarke, and Colin Camerer, studied how executives make strategic decisions and found that they frequently rely either on a single analogy or a handful of cases that come to mind.13 Investors likely do the same. The weakness in using an analogy or a small sample of cases from memory is that they often prevent a decision maker from sufficiently weighting the outside view. The strength is that the proper analogy or set of cases may prove to be a better match with the current decision than the broader base rate, hence providing useful information.

Exhibit 3 comes from the work of Lovallo, Clarke, and Camerer. The matrix considers reference classes (the columns) and weightings (the rows). In an ideal world, you want lots of past cases that are similar to the problem you face.

Exhibit 3: Reference Class versus Weighting Matrix

Reference Class

Recall Distribution

Reference class Event-Single analogy forecasting based (RCF)

Weighting Case-based Similarity-Similarity- based decision theory based forecasting (CBDT)

(SBF)

Source: Dan Lovallo, Carmina Clarke, and Colin Camerer, “Robust Analogizing and the Outside View: Two Empirical Tests of Case-Based Decision Making,” Strategic Management Journal, Vol. 33, No. 5, May 2012, 498.

“Single analogy,” found in the top left corner, refers to cases where an executive recalls one example and places 100 percent of his or her decision weight on that analogy. This is a very common approach, but it tends to substantially over-represent the inside view. As a result, it frequently yields assessments that are too optimistic.

“Case-based decision theory,” the bottom left corner, reflects instances when an executive considers a handful of case studies—generally through recall—that seem similar to the decision at hand. There is then an assessment of how similar the cases are to the focal decision, and the cases are weighted appropriately.

The top right corner is called “reference class forecasting.”14 Here, a decision maker considers an unbiased reference class, determines the distribution of that reference class, makes an estimate of the outcome for the focal decision, and then corrects the intuitive forecast based on the reference class. The weightings are generally event-based, which means that all of the cases in the reference class are weighted equally.

Lovallo, Clarke, and Camerer argue for what they call “similarity-based forecasting,” which starts with an unbiased reference class but assigns more weight to the cases that are more similar without discarding the less relevant cases. Done correctly, this approach combines the best of both worlds—a large reference class and means to weight relevance. 15

Similarity-based forecasting is a good way to express stage 3 of the “Using reference classes” column in Tetlock’s matrix. Stage 1 is the naïve application of the inside view. Stage 2 swings to the opposite extreme and relies too heavily on a reference class that may not be ideal. Stage 3 finds the right balance between the two and sharpens judgment.

As part of their research, Lovallo, Clarke, and Camerer ran a pair of experiments, including one with private equity investors. They asked the professionals to carefully consider a current project, including key steps to success, performance milestones, and the rate of return they expected on the deal. This revealed the inside

view. The researchers then asked the professionals to recall two past deals that were similar, to compare the quality of those deals to the project under consideration, and to write down the rate of return for those projects. This was a prompt to consider the outside view.

The average estimated return for the focal project was almost 30 percent, while the average for the comparable projects was close to 20 percent. Every subject wrote a rate of return for the focal project that was equal to, or higher, than the comparable projects. When subjects who had higher forecasts for the focal project were presented with the opportunity to revise down their forecasts in light of their estimates of the comparable projects, over 80 percent did so. The prompt to consider the outside view tempered their estimates of the rate of return for the deal under consideration. It is not hard to imagine similar results for corporate executives or investors in public markets.

Updating Information

We all walk around with beliefs in our head. The first column in Tetlock’s matrix addresses how good we are at understanding causality as we form our beliefs. Ideally, we want to improve our judgment to the point where our theories effectively reflect cause and effect. Tetlock’s second column deals with how effective we are at taking prior instances into consideration as we consider our current problem or question. The final column, which has ties to the first two, is about how effectively we update our beliefs as we learn new information.16 Bayes’s Theorem is the mathematical way to do this. The theorem tells you the probability that a theory or belief is true conditional on some event happening.

Here’s a classic example, which comes from Daniel Kahneman’s book, Thinking, Fast and Slow: 17

“A cab was involved in a hit-and-run accident at night. Two cab companies, the Green and the Blue, operate in the city. You are given the following data:

85 percent of the cabs in the City are Green and 15 percent are Blue.

A witness identified the cab as Blue. The courts tested the reliability of the witness under the circumstances that existed on the night of the accident and concluded that the witness correctly identified each of the two colors 80 percent of the time and failed 20 percent of the time.

What is the probability that the cab involved in the accident was Blue rather than Green?”

If you haven’t seen this problem before, take a moment to answer.

The most common answer is 80 percent, but the correct answer is about 41 percent.18 In this instance, the natural tendency is to place a great deal of weight on the witness’s account and, in the process, to underweight the fact that a large majority of cabs in the city are Green.

In theory, we have subjective prior beliefs that we update when new information arrives. We then base our decisions going forward on the revised beliefs. But applying Bayes’s Theorem is not intuitive for most of us, although we can improve our results based on how we consider information and how well we are trained in this kind of thinking.19

At stage 1, a number of mistakes are common. The first is that we tend to succumb to the confirmation bias, a tendency to insufficiently update beliefs in light of new information relative to a true Bayesian. Two cognitive processes are behind this bias.20 The first is that we are more likely to seek information that confirms our

belief than information that disconfirms it. The second is that we interpret ambiguous information in a way that’s favorable to our prior belief. Simply said, once we believe something, we’re inclined to make mistakes that will preserve our view.21

Tetlock discusses other common mistakes in updating beliefs based on new information. One mistake is to overreact to what he calls “pseudo-diagnostic” information. This is information that superficially appears to explain causality but in fact does not. For example, there have been cases when equity analysts have downgraded a stock following the announcement of an acquisition because of earnings dilution, only to see the stock rise because the market deemed the deal to add value. Earnings changes are a pseudo-diagnostic means to assess M&A.22

Another mistake is to underreact to “subtly-diagnostic” information. This is information that does matter but that the decision maker doesn’t recognize as causal. Continuing with the theme of M&A, subtly-diagnostic information might include a comparison of the present value of synergies with the premium pledged. Even though this information requires only modest calculations, a decision maker in stage 1 will be unable to distinguish between what matters and what doesn’t.

In stage 2, a decision maker becomes adept at figuring out what is important, even in subtler sources of information. Superior skill in identifying pseudo- and subtly-diagnostic indicators can be grounded in either intuition or theory. But intuition works only in environments that are stable where feedback is clear and recurring.23 Further, when we are presented with information in a way that makes causality clear, our ability to update according to Bayesian principles improves substantially.24 The remaining challenge is to properly revise prior views in light of the new information. In other words, the stage 2 decision maker moves in the correct direction, but an incorrect amount.

The final stage incorporates the ability to read causal clues with appropriate updating as determined by Bayes’s Theorem. So the individual gets both the causality right and revises his or her view properly in light of the new information. For people dealing with markets, getting to stage 3 is challenging in part because it is hard to receive feedback that is sufficiently timely and accurate. Experiments do suggest that market prices tend toward Bayesian values even if the revisions of the individual participants are noisy—the wisdom of crowds.25 Of course, markets do periodically veer far from fair value—the madness of crowds.26

Risk Management: Control and Reversibility

The reason we want to improve our ability to forecast is because our decisions ultimately rely on forecasts. We all understand that there’s inherent risk or uncertainty in most decisions. The goal is to decide so as to have a positive expectation.27

Peter Bernstein, a well-known economist and financial historian, suggested that there are two basic ways, beyond diversification, that we can manage risk.28 The first is to find decisions where we have some control over the outcomes. There’s a huge difference between the roll of a roulette wheel and a business investment.

In the former, you have no control over the outcome. In the latter, you can take steps to improve the chance of a profit by tweaking an offering price, changing a product design, shifting marketing spending, or replacing the managers running the business. But control often requires commitment.

A second means to manage risk is to seek situations that are reversible: if you make a mistake, you can simply undo your decision. Here you can contrast a company’s decision to build a new factory or merge with another firm, which are difficult to reverse, with the choice to buy a stock, which is easy to reverse if transaction costs are sufficiently low. Decisions with low reversibility generally require a long time horizon,

whereas those with high reversibility can have a much shorter horizon. The stock market provides the vital function of allowing an out for investors who have no control—they can sell their shares. Bernstein argues that without a properly functioning market, the separation of ownership and control is essentially impossible.

Exhibit 4 shows the trade-off between reversibility and control. Public investors, who are generally subject to relatively low transaction costs and high liquidity, have little control but can reverse their decisions readily.

Activists seek to exert control, but they must signal their seriousness and commitment by taking larger stakes and reducing their ability to reverse their decision. Private equity firms have a great deal of control, but their cost of reversibility is relatively high. Corporate decisions have the most control and the least reversibility. Not surprisingly, investment time horizons shrink as you move from left to right on the chart.

Exhibit 4: Reversibility and Control Trade-Off

High Corporate

Control Private equity Activists

Public equity Low Low High Reversibility Source: Credit Suisse.

Thoughtful investing requires having some view of the future. Good judgment requires understanding causality, effectively incorporating information from past events to understand present prospects, and updating probabilities correctly based on the arrival of new information. We can all improve across each of these facets of decision making, but the fact is that few of us move past a functional and comfortable stage. The goal of this report is to provide some tools and guidance to improve decisions on a day-to-day basis.

Endnotes:

1 Clayton M. Christensen, “The Ongoing Process of Building a Theory of Disruption,” Journal of Product Innovation Management, Vol. 23, No. 1, January 2006, 39-55. Also, Paul R. Carlile and Clayton M.

Christensen, “The Cycles of Theory Building in Management Research,” Harvard Business School Working Paper Series, No. 05-057, 2005.

2 Michael J. Mauboussin, Think Twice: Harnessing the Power of Counterintuition (Boston, MA: Harvard Business Review Press, 2009), 89-91.

3 Carliss Y. Baldwin and Kim B. Clark, Design Rules: The Power of Modularity (Cambridge, MA: MIT Press, 2000). For a broader discussion of outsourcing, see Clayton M. Christensen, Matt Verlinden, and George Westerman, “Disruption, Disintegration, and the Dissipation of Differentiability,” Industrial and Corporate Change, Vol. 11, No. 5, November 2002, 955-993.

4 Dan Lovallo and Daniel Kahneman, “Delusions of Success: How Optimism Undermines Executives’

Decisions,” Harvard Business Review, July 2003, 56-63.

5 Daniel Gilbert, Stumbling on Happiness (New York: Alfred A. Knopf, 2006), 231.

6 Mark L. Sirower and Sumit Sahni, “Avoiding the ‘Synergy Trap’: Practical Guidance on M&A Decisions for CEOs and Boards,” Journal of Applied Corporate Finance, Vol. 18, No. 3, Summer 2006, 83-95. 7 Shlomo Maital, “Daniel Kahneman, Nobel Laureate 2002: A Brief Comment,” The SABE Newsletter, Vol. 10, No. 2, Autumn 2002.

8 Daniel Kahneman and Amos Tversky, “On the Psychology of Prediction,” Psychological Review, Vol. 80, No. 4, July 1973, 237-251.

9 Bradley Efron and Carl Morris, “Stein’s Paradox in Statistics,” Scientific American, May 1977, 119-127. 10 The shrinkage factor can actually take a value from -1.0 to 1.0. A shrinkage factor of -1.0 would suggest that a good result of a certain magnitude is followed by a poor result of similar magnitude. In other words, the slope of the correlation between a past event and a present event is negative one.

11 William M.K. Trochim and James P. Donnelly, The Research Methods Knowledge Base, 3rd Edition (Mason, OH: Atomic Dog, 2008), 166.

12 Some factors, including style and fund inflows/outflows, can shape the correlation.

13 Dan Lovallo, Carmina Clarke, and Colin Camerer, “Robust Analogizing and the Outside View: Two Empirical Tests of Case-Based Decision Making,” Strategic Management Journal, Vol. 33, No. 5, May 2012, 496-512. 14 Bent Flyvbjerg is an economic geographer at Oxford University’s Saïd Business School who has analyzed large transportation infrastructure projects. He found that for the rail projects he studied the average cost overrun was 45 percent and that actual rail passenger traffic was about one-half of what the planners expected. Indeed, for a majority of the projects he analyzed the costs were higher and the benefits were lower than the authorities had planned. A detailed study of the Channel Tunnel, which links the United Kingdom and France by rail under the English Channel, concluded “that overall the British economy would have been better off had the Tunnel never been constructed, as the total resource cost outweighs the benefits generated.”

The natural question is why this is the case. Flyvbjerg suggests two explanations that fit the data. First, it may be that those who propose the project misrepresent the cost and benefit in order to get it approved.

This is most descriptive when the political stakes are high. So there is an incentive to misrepresent the project. The second explanation is the people are generally poor at planning. They assess only the facts in their own case and ignore the broad experience of others. This explanation for why we predict so poorly has great relevance in financial predictions. See Bent Flyvbjerg, “Truth and Lies about Megaprojects,” Speech at Delft University of Technology, September 26, 2007. Also, Ricard Anguera, “The Channel Tunnel—an ex post economic evaluation,” Transportation Research Part A, Vol. 40, No. 4, May 2006, 291-315.

15 Figuring out how to understand similarity is difficult in and of itself. Specifically, when we’re asked about similarity, we dwell on similarities and underestimate differences. Likewise, when we’re asked to focus on differences, we underestimate similarities. See Amos Tversky, “Features of similarity,” Psychological Review,

Vol. 84, 1977, 327-352. Reprinted in Eldar Shafir, ed. Preference, Belief, and Similarity: Selected Writings, Amos Tversky (Cambridge, MA: MIT Press, 2004).

16 In reality, factions that represent column 2 (frequentists) and column 3 (Bayesians) have debated one another for a century. Our view is that we might as well use both approaches to the degree to which they help improve our judgment. To read more about the controversy, see Sharon Bertsch McGrayne, The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy (New Haven, CT: Yale University Press, 2011). For a useful discussion of Bayesian thinking in finance, see Ricardo Rebonato, Plight of the Fortune Tellers: Why We Need to Manage Financial Risk Differently (Princeton, NJ: Princeton University Press, 2007), 40-66. For more technical applications in finance, see Svetlozar T. Rachev, John S.J. Hsu, Biliana S. Bagasheva, and Frank J. Fabozzi, Bayesian Methods in Finance (Hoboken NJ: John Wiley & Sons, 2008).

17 Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and Giroux, 2011), 166. Research regarding this problem was originally presented in Amos Tversky and Daniel Kahneman, “Evidential Impact of Base Rates,” in Daniel Kahneman, Paul Slovic, and Amos Tversky (eds.) Judgment under Uncertainty:

Heuristics and Biases (Cambridge, UK: Cambridge University Press, 1982), 153-160.

18 For a layman’s discussion of Bayes’s Theorem, see Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (New York: The Penguin Press, 2012), 243-248. To solve for the new probability, you need three quantities. First, you need a prior probability. In this case, the prior probability (x) of a Blue cab getting into an accident would be 15 percent (which assumes that Green and Blue cabs have an equivalent proclivity to get into accidents). Second, you need an estimate of the probability as a condition of the hypothesis being true (y). Here we have a witness, who has 80 percent accuracy, claiming that a Blue cab was in the accident. Finally, you need an estimate conditional on the hypothesis being false (z), which is 20 percent (the complement of 80 percent).

Bayes’s Theorem tells us the revised probability = xy = .15*.80 = .12 = 41.4% xy + z(1-x) .15*.80+.2(1 -.15) .29 19 Keith E. Stanovich, What Intelligence Tests Miss: The Psychology of Rational Thought (New Haven, CT: Yale University Press, 2009), 133-151.

20 Chetan Dave and Katherine W. Wolfe, “On Confirmation Bias and Deviations From Bayesian Updating,”

Working Paper, March 21, 2003.

21 This happens to experts as well. See Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, NJ: Princeton University Press, 2005), 122-123.

22 One case that comes vividly to mind is ConAgra’s acquisition of Beatrice from KKR, which was announced on June 8, 1990. ConAgra’s stock was downgraded that day reflecting concerns about earnings dilution. But the deal added a great deal of value (KKR participated as they took stock as part of the consideration). ConAgra’s stock rose 3.9 percent on June 8h, a day when the S&P 500 declined more than 1 percent. 23 Daniel Kahneman and Gary Klein, “Conditions for Intuitive Expertise: A Failure to Disagree,” American Psychologist, Vol. 64, No. 6, September 2009, 515-526.

24 Kahneman, 166-174.

25 Colin F. Camerer, “Do Biases in Probability Judgment Matter in Markets? Experimental Evidence,” American Economic Review, Vol. 77, No. 5, December 1987, 981-997.

26 Michael J. Mauboussin, “The Wisdom and Whims of the Collective,” CFA Institute Conference Proceedings, Vol. 24, No. 4, December 2007, 1-8.

27 Of course, not all decisions have a positive expectation. For example, insurance has a negative expectation, but is a means to diversify risk.

28 Peter L. Bernstein, “Risk, Time, and Reversibility,” Geneva Papers on Risk and Insurance, Vol. 24, No. 2, April 1999, 131-139.