厘清技能与运气：如何思考过去、现在与未来的结果

July 15, 2010

Untangling Skill and Luck How to Think About Outcomes—Past, Present, and Future

Fortuna, by Albrecht Dürer

• The outcomes for most activities combine skill and luck.

• Separating skill and luck encourages better thinking about outcomes and allows for sharply improved decision making.

• There are good methods to sort skill and luck in sports, business, and investing.

• We define the key features of skill in the investment business.

Introduction

For almost two centuries, Spain has hosted an enormously popular Christmas lottery. Based on payout, it is the biggest lottery in the world and nearly all Spaniards play. In the mid 1970s, a man sought a ticket with the last two digits ending in 48. He found a ticket, bought it, and then won the lottery. When asked why he was so intent on finding that number, he replied, “I dreamed of the number seven for seven straight nights. And 7 times 7 is 48.” 1

Outcomes from many activities—including sports, business, and investing—are the combination of skill and luck. Most people recognize that skill and luck play a role in results, yet they have a poor sense of the relative contribution of each. The ability to properly untangle skill and luck leads to much better thinking about most day-to-day outcomes, and allows for sharply improved decision making.

The process of asset allocation in the institutional investment industry is a practical example of the failure to conceptualize skill and luck. In the aggregate, institutional money tends to flow to assets that have done well and fails to consider sufficiently the role of luck. One recent study suggested that this misallocation of resources had cost these portfolios $170 billion from 1985 to 2006. The study’s authors conclude that those institutions “could have saved hundreds of billions of dollars in assets if they had simply stayed the course” instead of moving money based on a naive extrapolation of past results. 2

It’s important to define skill and luck before we get too far into the discussion. Skill is “the ability to use one’s knowledge effectively and readily in execution or performance.” You can think of skill as a process, or a series of actions to achieve a specific goal. Luck is “the events or circumstances that operate for or against an individual.” Luck, in this sense, is above and beyond skill. Consider luck as a distribution that has an average of zero. By this definition, luck tends to be transitory. 3

Here’s an example of how skill and luck might interact. Consider the proper play of a poker hand that leads you to have much better odds of winning the pot than your opponent. Your skill, the process of playing the cards well, has put you in a position to succeed. But say the dealer reveals a card that has a low probability of appearing but that makes your opponent’s hand. Your outcome was the combination of good skill and bad luck. Your opponent’s outcome, naturally, was bad skill and good luck.

A useful way to consider skill and luck is to place activities along a continuum that has all skill and no luck on one side, and no skill and all luck on the other. (See Exhibit 1.) For example, the outcomes for activities including chess and running races are close to pure skill, while games of chance, including roulette or the lottery, are close to pure luck. Most activities are in between these extremities and combine both skill and luck. To place activities, you have to think carefully about the forces that shape outcomes. Once you have a sense for where an activity resides, you have a useful basis for comparison.

Exhibit 1: The Skill-Luck Continuum

Pure Pure Skill Luck

Source: LMCM analysis.

One point is worth making right upfront: the outcomes of any activity that combine skill and luck will exhibit reversion to the mean. More technically, an extreme outcome (good or bad) will be followed by an outcome that has an expected value closer to the mean. Reversion to the mean is a tricky concept, and the relative contributions of skill and luck shed light on its significance for various activities.

There’s a simple and elegant test of whether there is skill in an activity: ask whether you can lose on purpose. 4 If you can’t lose on purpose, or if it’s really hard, luck likely dominates that activity. If it’s easy to lose on purpose, skill is more important.

In this report, we will discuss why unraveling skill and luck is so important, provide a framework for thinking about the contribution of skill and luck, offer some methods to help sort skill and luck in various domains, and define the key features of skill in the investment business.

Why It’s Important to Understand Skill ÅÆ Luck

My son’s terrific rowing coach, Yuri, a former member of the Ukrainian National Rowing Team, refuses to let anyone bid the athletes “good luck” before a race. He insists that they say, “Good effort.” Yuri understands that crew races are about skill, not luck, and doesn’t want the rowers or the spectators to think otherwise.

Yuri is ahead of the game. Appreciating the blend of skill and luck in an outcome is helpful in a number of ways:

• Creates a template for assessing outcomes. The information content in a small number of outcomes varies greatly based on where the activity lies on the continuum. When skill determines an outcome, a relatively small sample size is revealing. For example, chess players earn a rating based on their game results. That rating is a solid proxy for skill (even though the skills of players are constantly improving or deteriorating). A player who is rated 200 points higher than his or her opponent is expected to win 75 percent of the time. 5

By contrast, you need a large sample size when luck plays a large role in an outcome.

The reason is that you have to see enough outcomes to ensure that luck has evened out and that only skill is revealed. In some high-luck domains, it takes a long time to gather a sufficient sample to sort skill and luck. The best teams tend to rise to the surface over a 162-game major-league baseball season, for instance, but a short series is mostly luck.

It can also be the case that the time to assess skill and luck is short provided the number of outcomes is sufficiently large in that period. You can evaluate a trading system that generates lots of trades per day much more rapidly than a concentrated, buy-and-hold stock portfolio. While our natural tendency is to evaluate all outcomes over similar time periods (say over a quarter, a season, or a year), the key is to tailor the evaluation process to the activity. In some realms skill is easy to see, in others you must sieve for a long time before you are confident that you have identified it.

• Allows you to anticipate results. Your initial reaction may be that more luck means less predictable outcomes—which is true. But there is also an important insight that follows from knowing the relative contribution of skill and luck: the ratio of skill to luck shapes the rate of reversion to the mean. Specifically, the outcomes of activities laden with luck revert to the mean faster than activities with little luck. Reversion to the mean is present in all activities that have even a dash of luck, but the rapidity of the process hinges largely on how important a role luck plays. In fact, you can infer the relationship between skill and luck by analyzing past patterns of reversion to the mean. 6

The flip side of the speed of mean reversion is persistence. Outcomes that are highly persistent over time tend to be shaped more by skill than luck. One remarkable example is the record of Marion Tinsley in checkers. He was a world champion over the span of four decades, and only lost seven games in his 45-year career (and two of those losses were to the computer program, Chinook). You can only explain Tinsley’s record as a man of great skill playing a game of skill. 7

• Gives guidance for where you are most likely to be misled. About 40 years ago, Amos Tversky and Daniel Kahneman identified a common decision-making bias they called the “belief in the law of small numbers.” 8 The idea is that we tend to view a relatively small sample of outcomes of a population as representative of the broad population of outcomes. The magnitude of this mistake grows larger as the luck-to-skill ratio rises. For instance, if you see a dozen sprinters compete five times and the same individual wins every time, you could reasonably conclude that she is the most skilled runner. On the other hand, if you watch a big-league baseball player for ten at-bats, you would have very little basis to judge his skill. One estimate suggests that for 100 at-bats, luck determines about 80 percent of the batting average. 9

The illusion of control also comes into play here. This illusion says that when we perceive ourselves to be in control of a situation, we deem our probabilities of success to be higher than what chance dictates. Saying it differently, when we are in control we think our ratio of skill to luck is higher than it really is. Remarkably, this illusion even holds for activities that are all chance. For example, some people throw dice hard when they want a high number, and gently when they seek a small one. Like the belief in small numbers, this illusion is not a problem in high skill, low luck activities but becomes more problematic as the contribution of luck grows. Here again, our minds are poor at differentiating between activities, so what works in one setting fails miserably in another.

• Helps you share feedback properly. The best way to ensure satisfactory long-term results is to constantly improve skill, which often means enhancing a process. Gaining skill requires deliberate practice, which has a very specific meaning: it includes actions designed to improve performance, has repeatable tasks, incorporates high-quality feedback, and is not much fun. 10 Deliberate practice works well in domains that are dominated by skill—learning to play the cello, for instance.

The challenge is providing good feedback. The reason is that the outcome is the only quantity you can measure with any reliability, but it doesn’t easily reveal the contribution of luck. The natural default for most of us—whether it’s a manager evaluating the performance of a direct report or an investor sizing up how a money-manager has

done—is to rely on outcomes because that is something we can measure. What we can measure in the short term, however, may not be what matters in the long term.

Ultimately, it is a good process that leads to satisfactory outcomes, but the quality of the process gets swamped in the short term by luck.

The key, then, is to focus feedback on improving skill. This is a very difficult task in activities where luck plays a big role. For example, this means that in evaluating an analyst or a portfolio manager, it is much less important to see how she has done recently (whether her picks did well or her portfolio beat the benchmark) than it is to assess the process by which she did her job. Embracing and implementing this point of view is demanding. And make no mistake about it: the reason to emphasize process is that a good process provides the best chance for agreeable long-term outcomes.

• Provides a framework for understanding whether there’s a “best” participant. What is the purpose of a tournament, or playoff? The ostensible answer is to figure out which team or individual is the best. But it is futile to determine the preeminent participant in many instances because the sample size for the competition is too small or the nature of the matchups creates a lack of transitivity.

A small sample size is a big problem in domains with a large dose of luck. For example, in major league baseball the worst team will beat the best team in a best-of-five series about 15 percent of the time. 11 The winning percentage of weaker teams rapidly moves toward 50 percent as the disparity of the skill between teams narrows. The World Cup, an international soccer tournament held every four years, crowns a world champion. But given the large dose of luck in soccer, it is hard to argue convincingly that the team that wins the tournament is the best team. The sample size is simply too limited.

Transitivity is a key concept in assessing the outcomes of one-on-one interactions. An activity has transitive properties when competitor A beats competitor B, competitor B beats competitor C, and competitor A beats competitor C. Activities dominated by skill tend to be transitive. In contrast, in an activity that is not transitive, competitor A beats competitor B, competitor B beats competitor C, but competitor C beats competitor A. This is the set up of the game rock, paper, scissors. 12 In theory, there is no best strategy in rock, paper, scissors, and chance will dictate the winner of a game, or repeated games. (In reality, people are poor at behaving randomly. For example, in tournament play, competitors throw scissors only 29.6 percent of the time, 3.7 percentage points less than what randomness requires. 13) A number of sports show a lack of transitivity, in part reflecting the nature of match-ups.

Scott E. Page, a political scientist at the University of Michigan, illustrates why there is no objective best team in a set-up with low transitivity. Say we have four football teams with the same amount of total skill, indicated by 100 points. But each team’s skill is allocated differently across the dimensions of offense, defense, and special teams. When the teams go head to head, the team with the most points wins that dimension and the side that takes the most dimensions is the victor.

Take a look at Exhibit 2. Team A beats Team B because it has more points in offense and defense, two of the three dimensions. Likewise, Team B beats Team C because of better defense and special teams. But, like in rock, paper, scissors, Team C beats Team A. All teams beat Team D. If these teams were in a tournament, the winning team would be the one initially paired against Team D. If the pairings were set in a random fashion, then the winner would be random (except for poor Team D). There is no best team, just “the team that got to play Team D first.”

Source: Based on Scott E. Page, The Difference (Princeton, NJ: Princeton University Press, 2007).

Introducing a Framework for Thinking about the Contribution of Skill and Luck

Probably the single biggest challenge in assessing the relative contribution of skill and luck is that in most cases we can only observe outcomes. There’s no problem with outcomes at the extremes of all skill or all luck, because you know what you are getting. But almost all interesting activities have a blend of skill and luck, and it is critical to have a sense of the relative contributions of each. We need to tidy up the jumble of outcomes.

The urn model

One useful way to think about the problem is to imagine two urns, one for skill and one for luck.

Each urn contains cards that are marked with numbers that follow some sort of distribution. In a simple form, a mean and standard deviation specify the distribution, and the luck urn will always have a mean of zero. Exhibit 3 offers one example of what these distributions might look like. As we will see in a moment, these don’t have to be normal distributions.

Exhibit 3: Outcomes Combine a Skill and Luck Distribution

Skill Luck Source: LMCM.

Let’s look at a head-to-head matchup. Both participants—either individuals or teams—draw one number from a skill urn and one number from a luck urn, and then add them together. The player with the higher number wins that game. The players then return the numbers back to the urn, draw again, and decide the next outcome.

Consider again the continuum from pure skill/no luck to no skill/pure luck (Exhibit 1). We can reflect any point along that continuum by varying the mean and standard deviation of the numbers in the urns. For instance, the luck urn in an all-skill activity has a mean and standard deviation of zero. Since all participants draw only zeros from that urn, only the skill number determines competitive outcomes. At the other extremity, all-luck activities have a skill urn with a

zero mean and standard deviation, and only luck matters. (See Exhibit 4.) The challenge is to use the urn model to place activities on the skill-luck continuum.

Exhibit 4: Outcomes Combine a Skill and Luck Distribution

Pure skill and no luck Blend of skill and luck No skill and pure luck

Source: LMCM.

While using the two-urn model to assess outcomes can be of great value, many researchers fail to do so. This is a very important issue in evaluating investment results, for instance. The standard approach is to observe a distribution of outcomes and to make an estimate of the likelihood that a really good outcome—a result in the right tail of the distribution—is solely the product of luck. 14 But even near the extremity of all luck, where good results can be the result of chance, there’s a huge difference between saying that results are mostly luck and saying that they are all luck. Introducing a skill distribution with a small standard deviation allows for an important shift in mindset and raises the crucial challenge of defining skill in the investment industry.

What determines the role of luck

There are a number of factors that shift activities toward the luck side of the continuum. One, naturally, is simply sample size. A small number of observations make it very difficult to sort skill and luck. Consider observing the outcomes of five plate appearances for the best hitter and the worst hitter in major league baseball. You would have very little to go on. After 500 plate-appearances, you would be in a much better position to assess which player has greater skill.

People often assume that building sample size is a matter of time. Within an activity that is true, but what really matters is the number of trials. Some activities pack a lot of trials into a short time, and others reflect a few trials over a long time. In investing, a quantitative strategy that generates lots of signals and trades daily would be an illustration of the first, and a low-turnover, concentrated portfolio would be an example of the latter. You can evaluate a high-frequency trading strategy a lot quicker than a buy-and-hold investment approach. One analysis suggests that 12 National Football League games, 36 National Hockey League games, and 69 Major League Baseball games reveal the equivalent amount of skill. 15

Competitive parity also increases the role of luck. The idea is that as the skill levels of the participants converge, the standard deviation of skill narrows and luck becomes more prominent (in activities that allow for luck). 16 Convergence of skill can result from weaker players getting better, the dissemination of cheaper and more uniform information, or from athletes approaching biomechanical limits, slowing the rate of improvement. You might call it the paradox of skill: high and uniform skill levels suggest that luck becomes a larger determinant of outcomes. In activities that have little luck, including running and swimming races, you simply get lots of very close finishes. Some professional sports leagues use tools like salary caps to encourage parity in skill, seeking a more uniform distribution of winners over time.

In some activities the competition is not another person or team, but rather the collective bets of others. Pari-mutuel wagers on horse races are a good example. In a pari-mutuel system the house pools all of the wagers, takes a profit, and then pays out the winnings based on the outcome. You don’t make money by being smarter than the bettor next to you or by knowing which horse has the best odds of winning. You make money when the collective misprices the

odds. Studies of wagering on horse races suggest that the odds tend to be reasonable forecasts of actual outcomes. 17 The important idea is that you are competing not against other individuals, but against the wisdom of the crowd. When certain conditions are satisfied, crowds are smarter than the average individual within the crowd. This pushes the observed outcomes toward the luck side of the continuum. This is a crucial observation in markets and in structured wagering. 18

Luck is not always normal

I have depicted the contents of the skill and luck urns as normal distributions, but in reality the distributions come in very different shapes. For example, if you rank teams within a league from best to worst and assume the better team always wins, you get a distribution that looks like a relative flat line with kinks at the extremes (i.e., the best teams go undefeated and the worst teams are winless—see the Appendix).

Another example is when social processes drive the component of luck, weakening the contribution of skill (or quality). When individuals seek to judge the attractiveness of an item—say an idea, book, or movie—they often turn to the opinion of others. If a sufficient number of people like the item, it may pass a tipping point and become hugely popular. The distribution of outcomes of socially-driven products, including books and movies, tends to follow a power law, where most offerings sell poorly but a handful are wildly successful.19 In these cases, luck is not distributed according to a bell curve; rather, it does little or nothing to help most products but is a massive amplifier for a few. As relevant, the skewed contribution of luck means that the relationship between quality and commercial success is loose.

Duncan Watts, Peter Dodds, and Matthew Salganik conducted an experiment that neatly demonstrated how social processes shape luck in outcomes. 20 They set up a website called Music Lab and invited subjects to participate in a study of musical tastes. The site asked subjects to listen to and rate 48 songs by unknown bands, with an option to download the songs they liked.

Upon entering the site, the researchers assigned 20 percent of the subjects to an independent world and 10 percent each to eight worlds where people were allowed to see what other people were doing. In the independent world, subjects listened to and rated the songs and were free to download them, but had no information about what others were doing. In the other worlds the subjects also listened to and rated songs, but social influence came into play because they could see how many times other people had downloaded each song. The researchers ran a couple variations of the experiment, but in all scenarios the songs started with zero downloads.

This setup allowed for a very explicit test of social influence. The subjects in the independent group—those not swayed by the opinion of others—provided a reasonable indicator of the quality (skill) of the songs. If social influence is inconsequential, you would expect the song rankings— and downloads—to be similar in all nine worlds. On the other hand, if social influence is important, small differences in the initial download pattern in the social worlds would lead to very different rankings. Cumulative advantage (luck) would trump intrinsic quality (skill) in determining the outcome.

The study showed that song quality—skill—did play a role in the rankings. A top-five song in the independent world had about a 50 percent chance of finishing in the top five in a social influence world. And the worst songs rarely topped the charts. But the scientists also found that social influence played a huge part in the ultimate outcome. One song, “Lockdown” by the band 52metro, ranked 26th in the independent world, effectively average. Yet it was the number 1 song in one of the social influence worlds, and number 40 in another. Social influence (luck) catapulted an average song to the status of a hit in one world and relegated it to the cellar in another.

This social process makes it really difficult to predict the success or flop of a book or movie. But the same process also leads to inefficiency in markets, which an investor can use to his or her

advantage. Social imitation, then, is both an important source of randomness and the primary source of inefficiency that a skillful investor must exploit.

Placing activities on the skill-luck continuum

Now that we have an urn model to help guide our intuition and some ideas about what shapes luck, we can turn to actually placing activities along the skill-luck continuum. Where we place an activity is going to be very sensitive to the sample size we consider. In cases where luck is normally distributed, the larger the sample we consider the better we can observe skill. But large sample sizes, which can take years to accrue, have a downside if skill deteriorates. For example, skill tends to decline after an athlete reaches his or her late 20s. Further, in pure luck or near-pure luck activities, some participants will enjoy good outcomes solely as a result of chance.

One very useful method for placing activities is to combine pure skill and pure luck distributions in a proportion that matches the empirical results. Analysis of sports lends itself to this approach, and a natural period of assessment is a season. This analysis starts with three distributions: what would happen if luck determined the outcome of each game (a basic binomial model), what would happen if skill determined each game (a higher-skilled team always beats a lower-skilled team), and what actually did happen. Appendix A goes through this approach in detail based on the work of Brian Burke, the author of the terrific web site, Advanced NFL Stats. Burke concludes that luck’s contribution to the win-loss record of NFL teams is in excess of 50 percent.

Tom Tango, a respected sabermetrician, offers a four-step process that gets us to the same answer. 21 The equation he solves is: variance(skill) = variance(observed) - variance(luck). Rather than figuring out what blend of skill and luck best fits the empirical results, he determines skill by removing the role of luck from the outcomes. Here are the steps with data from the recent National Basketball Association (NBA) season.

1. Take a sufficiently large number of teams (preferably with the same number of games). We will analyze all 30 teams in the NBA for the 82-game season in 2009-2010.

2. Figure out each team’s winning percentage. For the 2009-2010 season, the Cleveland Cavaliers had the best regular-season record, winning 74 percent of its games. The most futile team was the New Jersey Nets, which managed to win only 15 percent of its games.

3. Figure out the standard deviation of the winning percentage. For the most recent season, the standard deviation of the winning percentage was 0.1630 and has averaged 0.1548 over the past five seasons. So the variance(observed) is 0.027, or 0.1630 2.

4. Figure out the standard deviation of outcomes determined by luck. This is the binomial model. The luck standard deviation = √.5 * .5/n, where n = the number of games. For the NBA, n equals 82 so the luck standard deviation is 0.0552 and the variance (luck) = 0.003, or 0.0552 2.

Knowing two of the three variables in the equation, we can solve for variance(skill):

Variance(skill) = variance(observed) – variance(luck)

Variance(skill) = 0.027 – 0.003 Variance(skill) = 0.024

Now we can look at the ratio of variance(luck) to variance(observed) to determine the contribution of luck, which equals about 11 percent. Exhibit 5 places sports along the continuum, using the skill/luck ratio based on the averages of the last five seasons for each sport. Appendix B shows the calculation for the NBA. Note that the length of the season is also an important factor: with

only 16 games, the NFL has by far the fewest games and the NBA could have a much shorter season and still have a clear sense of which teams are best.

Exhibit 5: Sports on the Skill-Luck Continuum (Average of the Last 5 Seasons)

Pure Pure Skill Luck

Source: LMCM analysis.

Professional basketball in the U.S. certainly stands out as the sport where skill plays the largest role in shaping results. One intriguing explanation for the NBA’s strong skill contribution is the height of the players. In most sports, the most skillful players within a wide range of heights can make it to the pros. But a relatively small percentage of the population is tall enough to play in the NBA. In their book, The Wages of Wins, David Berri, Martin Schmidt, and Stacey Brook note that only about 3 percent of the male population in America is 6’ 3” or taller, and a tiny percentage is above 6’ 10” (about four standard deviations from the average). Yet almost 30 percent of NBA players are at least 6’ 10”. They conclude that a “short supply of tall people” contribute to the talent disparity and hence the greater relative role of skill. The right tail of the height distribution does not overlap completely with the right tail of the skill distribution. 22

Decomposing measures to better understand skill

In many activities, we track certain statistics in order to calibrate skill. But there are plenty of cases where those statistics are sufficiently coarse that untangling the contributions of skill and luck is difficult. This section looks at how to decompose statistics to get a better handle on the role of skill.

Jim Albert, a professor of math and statistics, presents an analysis of batting average, the most widespread statistic used to measure hitters in baseball. 23 Albert wanted to determine the usefulness of batting average. He started by decomposing a plate appearance into the possible outcomes (see Exhibit 6). Batting average is the ratio of hits (singles, doubles, triples, or home runs) to at-bats. But naturally there are lots of ways to analyze hitters, including on-base percentage (roughly base on balls + hits divided by plate appearances) and strikeout rate (strikeouts divided by at-bats). Albert wanted to know which statistics were the result of skill and which ones had lots of luck.

Exhibit 6: Breakdown of Plate Appearances for a Baseball Hitter

Base on balls Hit by pitch

Plate Out appearance Strikeout

At-bat Single

In play Double

Triple

Home run

Source: Based on Jim Albert, “A Batting Average: Does It Represent Ability or Luck?” Working Paper, April 17, 2004.

He reasoned that a good way to test the skill-luck mix is to compare two years of hitting data. If a statistic accurately measures a player’s skill, you would expect the values for the statistic to be similar from one season to the next. On the other hand, if the statistic varies a great deal from year to year you can assume that luck plays a large role in that outcome.

Exhibit 7 shows scatter plots for three hitting statistics: batting average, in play, singles (a hit that lands for a single), and strikeout rate. What is clear is that both batting average and hitting for singles have a low correlation from year to year, with R2’s of less than 15 percent, suggesting that luck plays a large role in those outcomes. 24

Exhibit 7: Scatter plots of Three Baseball Hitting Statistics (2008-2009 Seasons for Players with 100+ At Bats)

2009 AVG 0.30

Source: LMCM analysis based on Jim Albert, “A Batting Average: Does It Represent Ability or Luck?” Working Paper, April 17, 2004.

By contrast, strikeout rate is highly correlated from year to year and is a good indicator of skill.

Here, the R2 is close to 70 percent. These correlations make intuitive sense. There are many factors that determine whether a ball falls for a hit when a player puts it into play, including the quality of the defense, the field, where he hits it, and the weather. On the other hand, strikeout rates match only pitcher and batter and fewer variables weigh on the outcome.

Exhibit 8 shows the correlations for eight batting statistics using data from the 2008 and 2009 seasons in MLB. We only included players with 100 or more at-bats in the sample. The analysis shows that a few statistics are very strong measures of skill, including strikeout rate, home run

rate, and base-on-ball rate (how frequently a player draws a walk). Measures like batting average, singles, and doubles are extremely noisy because of the role of luck.

Exhibit 8: Ranking of Correlations of Hitting Statistics (2008-2009 Seasons for Players with 100+ At Bats)

SO Rate: Strikeout Rate IP HR Rate: In-play home run rate BB Rate: Walk rate OBP: On-base percentage IP 2+3 Rate: In-play doubles and triples rate IP AVG: In-play batting average IP S Rate: In-play singles rate AVG: Batting average

IP 2+3 Rate IP HR Rate SO Rate IP AVG IP S Rate BB Rate OBP AVG

Skill Luck

1 0.8 0.6 0.4 0.2 0 Correlation

Source: LMCM analysis based on Jim Albert, “A Batting Average: Does It Represent Ability or Luck?” Working Paper, April 17, 2004.

In their latest book, Stumbling on Wins, David Berri and Martin Schmidt show similar statistics for football, basketball, and ice hockey. In hockey, for instance, shots on goal per minute correlates at a strong 80 percent from year to year while shooting percentage is less than 40 percent, and plus-minus (measures the goal differential when a specific player is on the ice) is less than 10 percent. 25

This analysis raises the central question of whether we can decompose measures of performance in other activities in a similar fashion. Vital to this approach is focusing on statistics that have a pair of attributes. First, the statistic should measure something that an individual, or team, actually controls and that is consistent from period to period. Second, the measure should have some direct bearing on outcomes.

The assessment of investment managers may lend itself to this method, and one measure that holds promise is active share. Active share, developed by two professors of finance at Yale University, Martijn Cremers and Antti Petajisto, reflects the fraction of the portfolio that is different from the benchmark index. The measure has a range from 0 percent (the portfolio is identical to the benchmark) to 100 percent (completely different than the benchmark). Active share is persistent, and high active share correlates well with excess returns. 26

The two-urn framework suggests a pair of methods that we can apply to various activities to better appreciate the role of luck. The first method is to study streaks of success. As Stephen Jay Gould, the famed biologist, summed up, “Long streaks are, and must be, a matter of extraordinary luck imposed on great skill.” 27 Using our urns to make the idea more vivid, streaks occur when the right tail of the skill distribution combines with the right tail of the luck distribution. Neither luck nor skill is enough, by itself, to meld a long streak.

This idea comes with a specific prediction: the longest streaks should be held by the most skillful participants. As we will see, this is the case. The idea also implies a large role for luck. Indeed, Gould’s quote was in response to a writer who suggested that Joe DiMaggio’s 56-game hitting streak in 1941 was hype because five of his hits were “narrow escapes and lucky breaks.” Gould responds, “Of course DiMaggio had a little luck during his streak. That’s what streaks are all about.” He also reinforces the point about skill: “Good players have higher characteristic probabilities, and hence longer streaks.”

Another method that the urn model suggests is a study of reversion to the mean. The rapidity of mean reversion gives you some clues about the contributions of skill and luck. Lots of skill makes outcomes stickier because good or bad luck are insufficient to sway results. When skill is absent, the luck distribution takes over and reversion to the mean tends to be rapid.

We will look at one additional approach, the degree of transitivity, which does not come directly from the urn model. The idea is that in activities with a narrow basis of competition and differential skill, transitivity holds (i.e., if A>B and B>C, then A>C). But as the basis of competition expands, transitivity weakens and outcomes become less predictable. The degree of transitivity can give us a sense of success given changing matchups and strategies.

Methods to Sort Skill and Luck Applied to Sports, Business, and Investing

We now apply these methods to gain some insight into the role of skill and luck in sports, business, and investing. When possible, we try to use the same analytical tools in each realm so as to compare them most effectively. Exhibit 9 shows some conclusions from the analysis.

Exhibit 9: Applying Skill-Luck Methods to Three Activities

Sports Business Investing

Some companies Long streaks in

Mean reversion Solid evidence for mean reversion across Strong mean reversion Mean reversion is well for results in mutual team sports. There is less mean reversion in documented over time. funds, investing styles, individual sports. and asset classes.

Different strategies

Transitivity Matchups often lack work in varying transitivity. economic situations. Different strategies Increasingly used in Circumstance versus work in varying strategy as well. attribute. Disruptive economic situations.

innovation.

Source: LMCM analysis.

While we will see evidence of skill and luck in each of these endeavors, it is worth noting up front that the relative contributions of skill will be different (just as they are within sports). In sports, players or teams compete with one another, and there are a lot of one-on-one interactions that determine outcomes. In business, firms compete against other firms and evidence of good profits invites competition. So success encourages additional competition, which tends to squeeze profits over time. Finally, investors compete with the collective of other investors. As we saw with pari-mutuel betting, being better than the person at a rival firm isn’t good enough; you must be better than the crowd. This is extremely difficult to do in practice for psychological and organizational reasons. But, as we will see, some participants clear those hurdles.

Skill has other constraints besides competition. For example, athletes follow a skill arc. At first an individual’s skill rises as he or she develops physically and hones ability. But skill then degrades as a consequence of aging. Skill can be much more persistent in cognitive tasks, as experience is additive. In cognitive activities like chess and science, for example, the peak skill occurs in the 30’s. More creative experts, including novelists, historians, and philosophers, hit the apex of skill in their 40’s or 50’s. 28

Skill can also be diluted by size. For instance, an executive who makes lots of acquisitions for the sake of growth or the investment manager who collects lots of assets under management will find it more difficult to add value as the size of the enterprise swells. Jack Bogle shows how the investable universe of stocks declines sharply as a function of fund size. Assuming that a fund can hold no more than 5 percent of the outstanding shares of any company, Bogle estimates that a fund with $1 billion of assets can choose from over 1,900 stocks while a fund with $20 billion has a universe of about 250 stocks. So it happens that success can sow the seeds of its own failure. 29

Streaks

A streak is a consecutive series of successes or failures. Streaks are one of the most elegant indicators of skill because if there is any differential capability within the population, the most skillful will hold the records for streaks. Not all skillful performers have streaks, but all long streaks of success are held by skillful performers.

With a large enough starting sample, you should expect some participants to have streaks of success solely due to luck. Teachers often illustrate this point with the example of coin tosses.

For example, if you start with 1,000 people and ask them to call coin tosses, you should expect about 3 percent of the group to get five in a row correct. I recently did this exercise with a group of about 400 students, and two students were right for seven consecutive tosses.

So the first obvious point is that in testing for differential capability using streaks, you need to compare the actual results with a null model that reflects randomness. So it’s not the existence of streaks that we’re interested in—we know that they exist. What we’re looking for are streaks that extend beyond what chance dictates, in either frequency or duration. This distinction is at the core of the debate about hot hands—the belief that recent success portends further success. The researchers who debunk the hot hand acknowledge that streaks of fruitfulness or futility exist; they just believe that those streaks are consistent with what probability dictates.

Sports are a convenient place to start the analysis of streaks because there are a lot data and we can easily assess the role of skill. And coming to a quick verdict is not difficult. Streaks of success in sports are held by the most skillful players. For example, the record for consecutive made field goals in basketball is held by Wilt Chamberlin, who drained eighteen consecutive shots in February 1967. Chamberlin made 54 percent of his field goal attempts over his career, placing him among the top twenty-five in shooting percentage in the league’s history. (He also set the single-season field goal percentage record by making 72.7 percent of his shots in the 1972-73 season.) Hockey has a similar case: Wayne Gretzky, who is by far the NHL’s leader in career goals, assists, and points, holds the record for most consecutive games with a point.

The most famous streak in sports (at least if you’re an American) is Joe DiMaggio’s 56 consecutive games with a hit, which he achieved almost 70 years ago. First, we can establish that the streak was the result of skill and luck. DiMaggio was clearly a very skilled player. For example, his batting average is in the top 50 all-time, easily in the top 2 percent of all players in history. Luck also played a prominent role, as Michael Seidel’s day-to-day chronicle of the feat attests. 30 But was it really “the most extraordinary thing that ever happened in American sports,” as Stephen Jay Gould claimed?

Sam Arbesman, a computational biologist, and Steve Strogatz, a mathematician, recently did a fresh analysis of DiMaggio’s streak. 31 They wondered what the probability was of any player in the history of baseball getting a hit in 56 straight games. Using data from actual results and simulation techniques, they found that there was somewhere between a 20 and 50 percent chance that some player would have a DiMaggio-like streak. As surprising, the simulations suggested that DiMaggio was barely in the top 50 players most likely to achieve the feat. Players including George Sisler, Ty Cobb, and even Ichiro Suzuki (who currently plays for the Seattle Mariners) were much more likely to set the record than DiMaggio was.

Thomas Powell, a professor of strategy at Oxford University, created a useful bridge between sports and business through a novel study of competitive parity. 32 Powell studied more than 20 industries in the U.S. and measured the degree of parity using a Gini coefficient. The coefficient was developed by Corrado Gini, an Italian statistician, to measure income inequality. Zero represents perfect parity and 1.00 reflects maximum disparity. Powell found that the average Gini coefficient for U.S. companies was 0.60, with a standard deviation of 0.24.

Powell then measured the Gini coefficient for non-industrial domains, including many sports (baseball, tennis, hockey, basketball, cricket, golf, football, and lacrosse) as well as other competitive fields (chess, snooker, bridge). He found that the non-industrial domains had an average Gini coefficient of 0.56, nearly identical to that of the companies, with a standard deviation of 0.24, exactly the same as the industrial sample. As he summarizes, “performance distributions in business are statistically indistinguishable from distributions in non-business domains. 33 Since we know that skill plays a meaningful role in the outcomes we see in sports, this research clearly suggests that skill—better known as competitive advantage—is also relevant in shaping business results.

Researchers typically define superior results in business as high and sustainable return on assets (ROA). 34 What is clear is that there is heterogeneity in this measure between companies, as the Gini coefficients suggest. The question relates to the source of those differences. Jerker Denrell, also a professor of strategy at Oxford University, suggests that the differential results may be the result of a random walk process. In other words, even if all firms start off at the same point, some will do a little better or worse than average by chance, allowing for differences in resource accumulation and, ultimately, corporate results. 35 In sociology, this is known as the “Matthew effect,” which basically says the rich get richer and the poor get poorer. 36 So the null model must account for the random walks.

A recent paper by Andy Henderson, Michael Raynor, and Mumtaz Ahmed does just that. 37 Raynor and Ahmed are consultants and Henderson is a professor of management at the University of Texas at Austin. They study the results of over 20,000 companies from 1965-2005, amassing over 230,000 firm-years of observations of ROA. The researchers carefully structured the analysis so that it would discern whether the occurrences of sustained superior results were beyond what chance would dictate.

The main finding of the study is that “the results consistently indicate that there are many more sustained superior performers than we would expect through the occurrence of lucky random walks.” While this is comforting because it suggests that management’s actions—skill—can help shape results, no one has been able to pinpoint what behaviors lead to superior results. So unlike sports where there are some observable measures of skill, all we can really say today is that we

cannot explain results by luck alone and that it appears that skill plays a role in shaping outcomes.

The authors also caution that it is easy to confuse superior performance with the results you would expect by chance. Just as probabilities can explain the apparent hot hands in sports, so is the case in much of business. The researchers write, “Our results show that it is easy to be fooled by randomness, and we suspect that a number of the firms that are identified as sustained superior performers based on 5-year or 10-year windows may be random walkers rather than the possessors of exceptional resources.” This is a very relevant point for management research.

Numerous studies observe superior corporate results, attach attributes to those results (great management, robust culture, etc.), and propose those attributes as a means to success. Such studies are utterly invalid if the apparent superior results are the result of luck, and this is undoubtedly the case. 38

While their analysis was not focused on streaks, the authors allow that “casual inspection of [the data] indicate[s] that streakiness was often the case.” This, as well as the other findings, is what you would expect if you combined skill and luck distributions. The challenge for researchers in competitive strategy is to get to the root causes of sustained superior results.

Streaks in the investment business have not been studied in great detail, and most critics write off streaks as the product of chance. A streak is defined as consecutive years of generating returns after costs in excess of a benchmark. For example, one pundit suggested that there was a roughly 75 percent probability that over the past 40 years some fund would generate a streak of 15 years, the duration of the longest known streak by a mutual fund. 39 One needs a complete disregard for the empirical facts to arrive at such an estimate. The only way to get there is to assume a large starting sample (in the thousands) and a coin-toss model. There were, in fact, 170 mutual funds in 1965 (the number didn’t exceed 1,000 until 1988), and only about 40 percent of mutual funds have beaten the market annually, on average, with a standard deviation of about 20 percent. 40

Andrew Mauboussin and Sam Arbesman analyzed mutual fund streaks over the past four decades or so, capturing over 50,000 mutual-fund years. Their null model applied the observed outcomes in each year to the funds in existence, capturing the role of chance. They simulated 10,000 mutual fund worlds and compared the simulated results to the actual record of streaks.

Similar to Arbesman and Strogatz, as well as Henderson, Raynor, and Ahmed, they found evidence that some funds generated streaks beyond what chance would dictate. They also observed that the funds that had established the streaks had a much higher “batting average”— the percentage of years that they successfully beat the benchmark—than did the average of all funds. 41 So the analysis of streaks indicates skill across all three activities, although the strength of the signal is by far the highest in sports.

Researchers using alternative approaches have also concluded that there is some skill in investing. 42 But the research also shows that only a small subset of the investing population is skillful, and that the percentage of funds that are skillful is declining. This is consistent with a market that steadily rises in informational efficiency over time. Note that these results take costs into consideration. One analysis suggests that about three-quarters of funds earn gross excess returns in line with the costs that they incur, making them “zero-alpha” funds. Alpha is a risk-adjusted measure of excess returns.

Reversion to the mean

Most people understand reversion to the mean in principle, but few properly reflect it in their decision making. The two-urn model is particularly useful in articulating the challenges and opportunities with reversion to the mean.

One insight the model reveals is that the rate of reversion to the mean is a function of the relative contribution of luck. For activities that are pure skill, reversion to the mean plays no role. Only shifting levels of skill will dictate outcomes. For activities that are pure luck, reversion to the mean is powerful. In the case where luck follows a normal distribution with a zero mean, each new draw from the urn has an expected value of zero. So it stands to reason that extreme events will migrate rapidly toward the middle.

You can think of skill as a drag on the reversion process for the activities that combine skill and luck. An above-average shooter in basketball, for example, may go through good or bad stretches of shooting. But she will not revert back to the average over time because of her skill. The same would be true of a below-average player. Luck may help or hinder in the short term, but reversion is limited because of the level of skill.

Humans, as natural pattern seekers, have a very difficult time dealing with reversion to the mean. The main challenge with the concept is that change within the system occurs at the same time as no change to the system. Change and no change operate side-by-side, causing a lot of confusion.

The change part is reversion to the mean. As we will see, the results of the groups that have done really well or poorly in one period tend to move toward the average in future periods. Most people find it hard to internalize reversion to the mean because it is more natural to extrapolate the performance of the recent past. So if a stock, or an asset class, has done well the natural inclination is to assume it will continue to do well and to act accordingly.

An equally common mistake with reversion to the mean is to assume that all results revert to the mean, which effectively means that the standard deviation of outcomes narrows over time. This is not true. For example, say you rank the teams in the NBA in quartiles based on their winning percentage for a season. Check in on the same quartiles in five years, and you will see that the win-loss percentages for each quartile are closer to 0.500. But, at the same time, the variance in the league’s win-loss rate will not have changed much. It will be pretty much the same as it was years ago.

What’s going on is that luck is reshuffling the teams on the distribution, even as skill is trying to keep them in the same spot. So teams that were lucky in one season may be unlucky in five years, or a team with average luck may enjoy above-average success or failure. Thus, while extreme performers migrate toward the middle, middle performers also migrate to the extremes.

The lesson is to always bear in mind the ratio of skill to luck in the activity and to recognize that recent results, especially if they are extremely good or bad, are unlikely to persist.

The world of sports is filled with reversion to the mean. You can readily see it on a team or individual level. The left panel of Exhibit 10 shows reversion to the mean of the win-loss record of Major League Baseball teams from 1999 to 2009. Even though the year-to-year outcomes appear messy, the ultimate outcome is clear: in the decade ended in 2009, the best teams see their median win percentage erode by over 14 percentage points while the teams initially in the cellar see their win percentage improve by almost 12 percentage points (see Exhibit 10, right panel).

Exhibit 10: Mean Reversion in Major League Baseball Team Winning Percentages

Winning % above .500 5.0%

Source: Baseball Prospectus and LMCM analysis.

Consistent with the idea that reversion to the mean accommodates change and no change, the win percentage of the best and worst quintiles moved toward 0.500 while the standard deviation of the distribution of winning percentage for the league held reasonably constant at about 0.07.

We can see how the interplay between luck and skill shapes reversion to the mean by analyzing hitting statistics side by side: one more reliant on luck (batting average) and one more reliant on skill (strikeout rate). Exhibit 11’s left panel shows the batting averages of roughly 100 major leaguers who had at least 150 at-bats for the five seasons ended 2009. The 0.60 basis point gap between the best and the worst quintile (0.315 versus 0.255) is cut by two-thirds by the final year (0.285 versus 0.265). This result would occur even without the assumption of any change in skill level.

Exhibit 11: Mean Reversion in Major League Baseball Player Batting Averages Batting Average Strikeout Rate

Batting Average Strikeout Rate

Source: Baseball Prospectus and LMCM analysis.

As the right panel of Exhibit 11 shows, the mean reversion is much less pronounced with strikeout rate because skill is more important. While the gap from best to worst does narrow from roughly 0.150 to 0.120, you can see that the outcomes remain relatively persistent.

The persistence of skill and the absence of luck is also the reason that the world’s top tennis players dominate the ranking for extended periods of time. For instance, four players—Pete Sampras, Roger Federer, Ivan Lendl, and Jimmy Connors—each held the number one spot for the equivalent of five or more years.

Corporate performance also shows reversion to the mean. This phenomenon has been well documented for decades. 43 For a company, skill is equivalent to competitive advantage, which confers an ability to generate returns on capital in excess of the cost of capital. Companies, like athletes, tend to follow a lifecycle. A company typically sees its skill diminish as the industry matures, as all competitors move toward optimal efficiency, and as prices are set so that they squeeze out excess profits. Competitive advantage is closely linked to barriers to entry. Bruce Greenwald, an economist at Columbia University, is fond of saying, “In the long run, everything is a toaster.” He picked the toaster to symbolize a mature, competitive business with no barriers to entry and no excess returns. 44

Here’s what the pattern of performance looks like for companies. The left panel of Exhibit 12 places the non-financial companies in the Russell 3000 that had data throughout the entire period (a sample in excess of 1,800 companies) into quintiles based on the spread between their return on invested capital (ROIC) and the weighted average cost of capital (WACC) in 1999. It then tracks the median returns for those quintiles though 2009.

Exhibit 12: Mean Reversion in ROIC – WACC Spreads for the Russell 3000 Non-Financial

ROIC - WACC 20 10.0%

ROIC-WACC (%)

Source: Capital IQ, LMCM analysis.

Specifically, the spread between the highest and lowest quintiles shrinks from 70 percentage points in 1999 to about 10 percentage points in 2009 (see Exhibit 13, right panel). Note that this powerful reversion to the mean accommodates the superior results of some companies, as we discussed in the section on streaks and persistence. Said differently, we would expect the rate of reversion to the mean to be even more rapid absent some companies with competitive

advantage. This pattern of reversion to the mean also belies the principle that the distributions do not change. The distribution of ROIC - WACC spreads, in fact, remained similar throughout the measured period. Exhibit 13 lays the 2009 distribution on that of 2004. Even a brief visual inspection confirms the similarity of the two distributions.

Exhibit 13: Change and No Change Co-Exist

800 700 600 2009

Source: Capital IQ, LMCM analysis.

Research by Robert Wiggins and Timothy Ruefli, professors of management, shows that not only is reversion to the mean in clear evidence for the corporate world, but also that returns are converging at a faster rate today than they did in the past. 45 They note that this phenomenon is not limited to technology companies but rather is evident across all industries. An empirical study of the slope of the rate of decline found that companies with high returns, high cash-flow variability, and high growth faded the fastest. Stable businesses with slow growth rates held the steadiest. 46

Reversion to the mean is a powerful force in investing, too. Jack Bogle, a luminary of the investment industry, illustrates this by ranking mutual funds in quartiles based on results in the 1990s and seeing how those quartiles performed in the 2000s. The top quartile, which had handily outpaced the average fund in the 1990s, saw a 7.8 percentage point drop in relative performance. Symmetrically, the bottom quartile in the 1990s witnessed a sharp 7.8 percentage point gain in results in the 2000s. Exhibit 14 tracks the excess returns of about 700 large cap mutual funds ranked based on 2000 results. By 2009, the excess returns for the best performing funds in 2000 was effectively zero, while the lowest quintile delivered strong excess returns.

Because investing results have a large dose of randomness, reversion to the mean is mighty. 47

Exhibit 14: Mean Reversion in Mutual Fund Results

Excess Return (%)

20 15.0

Excess Returns (%)

Source: Morningstar, LMCM analysis.

Persistence of performance has been one of the most popular topics in mutual fund research. 48 Similar to what we saw with companies, and consistent with the analysis of streaks, there is some evidence of persistence in fund performance. 49 But the strength of the signal is in part a function of the time period a researcher decides to measure, is more pronounced with poor performers than superior performers, and is weakened as academics adjust mutual fund returns for factors in stock price returns (i.e., size, value, momentum).

Importantly, reversion to the mean in the investment business extends well beyond the results for mutual funds. It applies to classifications within the market (small capitalization versus large capitalization, or value versus growth), across asset classes (bonds versus stocks) and spans geographic boundaries (U.S. versus non-U.S.). There are few corners of the investment business where reversion to the mean does not hold sway. 50

We have mentioned already that reversion to the mean ensnares a lot of decision makers. This is so important for investors, however, that it bears additional comment. The sad fact is that there is significant evidence that investors—both individual and institutional—fail to recognize and reflect reversion to the mean in their decisions. To illustrate, the S&P 500 Index generated returns of 8.2 percent in the twenty years ended 2009. The average mutual fund saw returns of about 7 percent, reflecting the performance drag of fees. But the average investor earned a return of less than 6 percent, about two-thirds of the market’s return. The reason investors did worse than the average fund is bad timing: they put money in when markets (or funds) were doing well and pulled money out when markets (or funds) were doing poorly. This is the opposite of the behavior you would expect from investors who understand reversion to the mean.

It is certainly understandable that individual investors react to emotional extremes. But what about the institutional investors who allocate capital for a living? You might guess that they are fully aware of reversion to the mean and work to counter-balance it. Yet that is not at all the behavior that researchers observe. Institutions, often guided by committees, fail to reflect reversion to the mean in their decisions, which ends up costing their beneficiaries billions of dollars.

Amit Goyal and Sunil Wahal, professors of finance, studied how well 3,400 plan sponsors (i.e., retirement plans, endowments, foundations) did in their decisions to hire and fire investment managers over a decade. 51 They found that plan sponsors hired investment managers after they had generated superior returns, only to see post-hiring excess returns revert to zero. Plan sponsors fired investment managers for a multitude of reasons (poor performance topped the list), only to see the managers they fired deliver statistically-significant excess returns.

A separate study, which looked at decisions across a large number of asset classes and over a longer time period, came to a similar conclusion. The study’s authors summarized, “Perhaps investment officers—either because they believe it themselves or their supervisors do—find comfort in extrapolating past performance when, in fact, excess performance is random or cyclical.” 52 While we would say that results are mostly random in the short term, the larger point holds. Both studies are consistent with performance chasing and the belief in hot hands, and inconsistent with an appreciation for the force of reversion to the mean.

Coping with reversion to the mean is not easy in markets, but there is a three-step process that helps guide a decision. The first step is to consider the mix of skill and luck in the activity. As we have seen, the pull of reversion to the mean will be strongest in activities dominated by luck.

Second, consider how extreme the result is versus some sense of the average. For example, periods with stock market returns that are substantially above or below the long-term average of 6-7 percent, adjusted for inflation, may be followed by periods that are closer to the average.

Finally, consider the expectations reflected in asset prices. 53Avoid assets that have performed well and embed optimistic expectations, and embrace assets that have performed poorly and embed low expectations. These steps are easy to articulate but difficult to follow due to psychological and institutional constraints.

Transitivity

Transitivity holds when there’s a clear pecking order of skill and the better competitor always wins. In reality, matchups between individuals, teams, competitors, or strategies generally yield a lack of transitivity. What allows you to win in one environment may not work in another. As a general rule, transitivity tends to decrease as the complexity of the interaction increases. It is hard to characterize the degree of transitivity thinking only of skill and luck, but the idea remains very useful for decision makers.

The importance of transitivity in sports is quite clear. We already saw how in theory the winner of a tournament might be the result of the order in which the teams played one another (Exhibit 2).

Notice that in the setup of that example, all the teams had the same total level of skill, designated as 100 points, and that it was the structure of the matchups that determined the outcome.

The significance of matchups is also evident in practice. Wayne Winston, a professor of decision sciences, analyzed interactions between NBA lineups, as well as between specific players, and found a lack of transitivity. To illustrate, he studied the 2006 playoffs and 2006-2007 regular season and found that Steve Nash outplayed Devin Harris, Harris outplayed Tony Parker, and Parker outplayed Nash. Which player is better depends on the matchup, and there is no way to say that one is best. 54 A similar lack of transitivity exists with strategies in soccer and between face-off specialists in lacrosse. 55

The degree of transitivity spills over to coaching strategy. KC Joyner, a football analyst at ESPN, separates football coaches into two categories: those who focus on personnel and those who focus on strategy. Personnel coaches try to stack their teams with the most talented players, keep the game plan simple, and try to overwhelm their opponents. Strategy coaches worry less about attracting the finest players and focus instead on outsmarting their opponents with innovative game plans. 56

Mike Leach, the former coach of the Texas Tech football team, is a great example of a strategy coach. He managed to win over 70 percent of his games in recent years despite playing a highly competitive schedule. The team’s success is particularly remarkable since few of the players were highly recruited or considered “first-rate material” by professional scouts. 57

Leach offset the talent gap by introducing more complexity into the team’s offense via a large number of formations. By creating new matchups, these formations changed the geometry of the game and forced opponents to change their defensive strategies. For example, defensive linemen were frequently forced to drop back to cover receivers. Leach explained that “defensive linemen really aren’t much good at covering receivers. They aren’t built to run around that much. And when they do, you have a bunch of people on the other team doing things they don’t have much experience doing.” You can certainly argue that Leach was skillful. The key here is that by adding complexity, Leach also made results less transitive.

Transitivity is also evident in business. One example is how a company’s product offerings fare under changing economic conditions. For instance, one automobile manufacturer may emphasize small vehicles while a competitor focuses on sport utility vehicles (SUV). When gasoline prices are high the maker of small cars will be in a better position, and when prices are low the producer of SUVs will have the advantage. So the vagaries of the fuel market dictate the outcomes, and there is no strategy that works in all environments.

Clayton Christensen’s theory of disruptive innovation is another example of transitivity in business. 58 Christensen studied why great companies with smart managements and substantial resources consistently lost to “disruptors,” companies with simpler, cheaper, and inferior products. He describes two ways that this can happen. In one case, the disruptors introduce a product that is at the low end of the market and that is neither profitable for the incumbents nor in demand from the incumbent’s current customers. Incumbents are motivated to flee the low-end segment of the market and to focus on more value-added products. This becomes a problem as the disruptors improve their offering and move up market, eventually encroaching on the core business of the incumbent, and doing so with a lower cost structure.

In the other case, disruptors introduce a product that was unavailable to consumers before, effectively competing with non-consumption. A recent example is Nintendo’s Wii video game console. Rather than focus on the high end of demanding gamers, as Microsoft’s Xbox and Sony’s Playstation 3 have, the Wii expanded the market by making the games simpler to play and more intuitive. In these instances, incumbents tend to ignore the market (as Microsoft and Sony largely did initially).

Whether the disruptor’s strategy is based on a low-end segment or on non-consumption, stronger incumbents yield to weaker challengers because of asymmetric motivations. 59 As in sports, we see cases where the more “skillful” company loses to the weaker company as the result of strategy choices. Transitivity in head-to-head matchups frequently appears as the result of asymmetric motivations. The Colonel Blotto game, a model from game theory, also sheds light on this conclusion. 60

Transitivity also exists in the investment industry. For example, the success of different styles tends to rotate. If you are a small capitalization (cap) manager, for instance, there will be times when small cap stocks will outperform large cap stocks and you will do well just by showing up to work. And since the industry frequently seeks to constrain the mandates of individual portfolio managers, success is often about style, not skill.

This effect shows up when we study the mutual fund industry. The decade of the 1990s and the first decade of the 2000s offer an interesting contrast. The 1990s were one of the worst decades for active management, with an average of only 35 percent of funds generating returns in excess of the S&P 500 annually. The 2000s were one of the best decades for active management, with an average of half of all funds beating the index in each year. Few would think of the 2000s as

better than the 1990s for active managers, because the absolute returns were so much lower in the 2000s. But on a relative basis, 2000-2009 was a golden decade for active managers, with a rate of beating the benchmark 25 percent higher than the long-term average.

The reason active managers did so much better in the recent decade has little to do with skill and a lot to do with style. Most funds that use the S&P 500 as a benchmark construct portfolios with stocks that have an average market capitalization that is much smaller than that of the broad index. 61 This suggests a simple relationship: when large cap stocks outperform small cap stocks, active managers will struggle. Conversely, when small cap outperforms large cap, active managers will shine. This was the case with the 1990s versus the 2000s. In the 1990s, large cap stocks beat small caps by an average of 6.6 percentage points a year. By contrast, small caps generated returns 4.5 percentage points greater than large caps, on average, in the 2000s. As transitivity suggests, different strategies win from one environment to the next. 62

Peter Bernstein, who was one of the investment industry’s brightest stars, wrote an article in 1998 suggesting that outsized excess returns in the investment industry were unlikely in the future. 63 Bernstein’s analysis was a riff off of an essay by Stephen Jay Gould explaining why there would never be another 0.400 hitter in baseball (Ted Williams last achieved the feat in 1941, hitting 0.406.) Gould reasoned that because all players are improving in all facets of the game, the standard deviation of results narrowed. That was in fact true for batting average, which reduced to a miniscule level the probability that even an outlier could reach 0.400.

Bernstein surmised that as markets continued their march toward efficiency, a similar pattern was occurring for money managers. The data backed it up: the standard deviation of excess returns for mutual funds had slowly and steadily declined from 1960 through 1997. In 2004, though, Bernstein reran the numbers and found that the standard deviation had exploded from roughly 10 percent in the late 1990s to almost 20 percent in 1999. He concluded that the 0.400 hitters of the investment industry had returned. 64 But the spike in standard deviations was short-lived and attributable to strong style swings. Specifically, large cap managers were narrowly focused on technology stocks in late 1999, allowing for strong returns relative to other styles. And following the technology stock bubble, small cap managers enjoyed massive relative returns. Since Bernstein’s paper in 2004, however, the standard deviations have again shrunk, consistent with his (and Gould’s) original thesis.

Conclusion

The two main ways to assess skill and luck are through an analysis of persistence of performance (with streaks being a particularly useful subset of this approach) and its alter ego, reversion to the mean. The research shows evidence for persistence of performance in sports, business, and investing, although the evidence is strongest in sports. Studies of business and investing point to skill in both domains, although the percentage of companies or investors with skill is small.

Reversion to the mean is also clear in each realm. The central insight is that the more the outcomes of an activity rely on luck (or randomness), the more powerful reversion to the mean will be. As important, it is clear that many decision makers do not behave as if they understand reversion to the mean, and predictably make decisions that are, as a consequence, harmful to their long-term outcomes. This is particularly pronounced in the investment industry.

The two-urn model is a useful mental model because it allows for differential skills and accommodates luck. Even Paul Samuelson, the Nobel-prize winning economist and efficient markets advocate, allowed for the possibility of investment skill. He wrote, “It is not ordained in heaven, or by the second law of thermodynamics, that a small group of intelligent and informed investors cannot systematically achieve higher mean portfolio gains with lower average variabilities. People differ in their heights, pulchritude, and acidity. Why not their P.Q. or performance quotient?” 65

An examination of transitivity also provides insights into where outcomes are most predictable. A lack of transitivity marks large swaths of sports, business, and investing. Since it is not always straightforward to pin low transitivity on skill or luck, the main lesson is to recognize that matchups and strategies can matter a great deal.

We now come to the final part of our discussion—defining skill in the investment business. Even if we can reasonably conclude that there are skilled investors, the challenge is to identify them before they deliver superior results. Skill in investing, like other probabilistic activities, is a process that incorporates analytical, psychological, and organizational considerations. If nothing else, the discussion of skill and luck makes clear why focusing on outcomes is less useful than focusing on process. If luck nets to zero over time—you win some, you lose some—then long-term results depend on the process.

Skill in Investing: What Comprises a Good Investment Process?

Here are some thoughts on what makes for a good investment process, or skill in the investment industry. This discussion applies directly to long-term investors, but many of the concepts apply to any type of investment approach. You can think of skill in three parts.

The first part requires you to find situations where you have an analytical edge and to allocate the appropriate amount of capital when you do have an edge. The financial community dedicates substantial resources into trying to gain an edge but less time on sizing positions so as to maximize long-term wealth.

At the core of an analytical edge is an ability to systematically distinguish between fundamentals and expectations. Fundamentals are a well thought out distribution of outcomes, and expectations are what is priced into an asset. A powerful metaphor is the racetrack. The fundamentals are how fast a given horse will run and the expectations are the odds on the tote board. As any serious handicapper knows, you make money only by finding a mispricing between the performance of the horse and the odds. There are no “good” or “bad” horses, just correctly or incorrectly priced ones. 66

An analytical edge exhibits certain characteristics. For example, assessment of the fundamentals should be consistent with the principles of economics, especially microeconomics. Investors need to grasp notions like supply and demand, economic profits, and sustainable competitive advantage. An edge should also incorporate the outside view rather than relying on the inside view. With the inside view, decision makers tend to gather information about a topic, combine it with their own inputs, and project into the future. In most cases, the inside view leads to conclusions that are too optimistic. By contrast, the outside view asks what happened when others were in a similar situation before. By leaning more on historical base rates than on individual extrapolation, the outside view provides a better grounding for analysis. 67

An analytical edge should also be repeatable in different environments. This does not mean that an investor must always find an edge; there will be times when the set of potential investments will be limited by either the investor’s correct realization of the limits to his or her competence or by a lack of attractive opportunities. It does mean that the approach to finding an edge will be steadfast over time and can be applied to various industries or asset classes.

Onlookers frequently confuse edge with style. When a certain style is doing well, a manager using that style will fare favorably whether or not he or she actively chose that exposure. Over time, some factors have generated excess risk-adjusted returns. For example, small caps have delivered higher returns than large caps since the mid 1920s. But these long-term results mask the existence of extended periods when those factors don’t work. If you had bet on small caps going into the 1980s and 1990s, as an illustration, you would have fared worse than the S&P 500. Edge means generating excess returns because of mispricing. Style suggests being in the right place at the right time. Sometimes edge and style overlap, sometimes they don’t.

Edge also implies what Ben Graham, the father of security analysis, called a margin of safety.

You have a margin of safety when you buy an asset at a price that is substantially less than its value. As Graham noted, the margin of safety “is available for absorbing the effect of miscalculations or worse than average luck.” The size of the gap between expectations and fundamentals dictates the magnitude of the margin of safety. Graham expands, “The margin of safety is always dependent on the price paid. It will be large at one price, small at some higher price, nonexistent at some still higher price.”68

Finding gaps between fundamentals and expectations is only part of the analytical task. The second challenge is to properly build portfolios to take advantage of the opportunities. There are two common mistakes in sizing positions within a portfolio. One is a failure to adjust position sizes for the attractiveness of the opportunity. In theory, the positions in more attractive risk-adjusted opportunities should be more prominent in the portfolio than less attractive opportunities. In some activities, mathematical formulas can help work out precisely how much you should bet given your perceived edge. 69 While this is difficult in practice for most money managers, the main idea remains: the best ideas deserve the most capital. The weighting in many portfolios fails to distinguish sufficiently between the quality of the ideas.

The other mistake, at the opposite end of the spectrum, is overbetting. In the past, funds that have seen their edge dwindle have boosted returns through leverage. This led to position sizes that were too large for the opportunity and ultimately disastrous in cases when the trade didn’t perform as expected. The failure of Long-Term Capital Management is one of the best-documented cases of the perils of overbetting. 70 The analytical part of a good process requires both disciplined unearthing of edge and intelligent position sizing aimed at maximizing long-term risk-adjusted returns.

The second part of skill is psychological, or behavioral. Not everyone has a temperament that is well suited to investing, and skillful investors approach markets with equanimity. One such skilled investor is Seth Klarman, founder and president of the highly-successful Baupost Group, who shared a wonderful line: “Value investing is at its core the marriage of a contrarian streak and a calculator.” 71

A large source of mispricing is when the collective becomes uniformly bullish or bearish, opening large gaps between expectations (price) and fundamentals (value). The first part of Klarman’s line emphasizes the importance of the willingness to go against the crowd. Academic research confirms what most people know: it is easier and more comfortable to be part of the crowd than it is to be alone. Skillful investors heed Ben Graham’s advice: “Have the courage of your knowledge and experience. If you have formed a conclusion from the facts and if you know your judgment is sound, act on it—even though others may hesitate or differ.” 72 However, Klarman correctly observed that it is not enough to be a contrarian because sometimes the consensus is right. The goal is to be a contrarian when it allows you to gain an edge, and the calculator helps you ensure a margin of safety.

Exposure to diverse inputs is crucial to developing sound contrarian views. As an idea takes hold in the investment community, it tends to crowd out alternative points of view. Skillful investors constantly seek input from a variety of sources, primarily through reading. Phil Tetlock, a psychologist who has done groundbreaking work on the decision making of experts, writes that “good judges tend to be . . . eclectic thinkers who are tolerant of counterarguments.” 73

This part of the process also acknowledges, and takes steps to mitigate, the biases that emanate from common heuristics. These biases include overconfidence, anchoring, the confirmation trap, and the curse of knowledge, to name just a few. 74 Overcoming these behavioral pitfalls is not easy, especially at emotional extremes. Techniques that are helpful include expressing views in probabilistic terms, constantly considering base rates, and maintaining a decision-making journal.

The last component of this part is maintaining what I call a “Mr. Market” mindset. To express a proper attitude toward markets, Ben Graham created the idea of Mr. Market, a “very obliging” fellow who offers to sell his shares to you or to buy yours. Mr. Market shows up every day, but is sometimes very optimistic and, fearful that you will snatch his shares at a low price, posts a very high price. On other occasions he is distraught, and seeks to dump his shares at a bargain-basement price.

Graham’s main lesson is that Mr. Market is there to serve you, not to educate you. You cannot let the prices entrance you. Graham writes, “Basically, price fluctuations have only one significant meaning for the true investor. They provide him with an opportunity to buy wisely when prices fall sharply and to sell wisely when they advance a great deal.” 75 This is easy to say but requires a lot of skill to do.

The third part of the process of skill addresses organizational and institutional constraints. The core issue is how to manage agency costs. Costs arise because the agent (the money manager) may have interests that are different than the principal (the investor). 76 For example, mutual fund managers who are paid fees based on assets under management may seek to prioritize asset growth over delivering excess returns. Actions to serve this priority may include heavily marketing products that have been recently successful, launching new products in hot areas, and managing portfolios to look similar to their benchmarks.

Charley Ellis made this point when he distinguished between the profession and business of investing. 77 The profession is about managing portfolios so as to maximize long-term returns, while the business is about generating earnings as an investment firm. Naturally, a vibrant business is essential to support the profession. But a focus on the business at the expense of the profession is a problem. Stated differently, you want the investment professionals focused intently on finding opportunities with edge and building sensible portfolios.

Career risk is also important. Investment managers seeking long-term excess returns will frequently have portfolios that are very different than the benchmark and that have high tracking error. If the time horizon of either the investment company or the clients is shorter than the time horizon necessary to see the fruition of the investment approach, even skilled managers risk getting fired. Professional investors have learned to play close to the index. For example, aggregate active share is down considerably over the past 30 years. 78

All three parts of investing skill are difficult. Many organizations clear one or two of the hurdles, but few can clear all three. This fits with the conclusion of our analysis of skill and luck in investing: there are differential capabilities, but only a handful of investors can clear the analytical, psychological, and organizational hurdles.

In 1984, Warren Buffett gave a speech at Columbia Business School called “The Superinvestors of Graham-and-Doddsville.” 79 He referred to the coin toss metaphor and granted that some investors would succeed by luck. But he went on to point out that a number of successful investors came from the same “small intellectual village that could be called Graham-and-Doddsville.” Common to all of the investors was that they searched “for discrepancies between the value of the business and the price of small pieces of that business.” These investors had a common patriarch, Ben Graham, but went about succeeding in different ways. Still, Buffett suggested he anticipated their success based on “their framework for investment decision making.” While some luck along the way didn’t hurt, their results were all about skill.

Endnotes 1 Stanley Meisler, “First in 1763: Spain Lottery—Not Even a War Stops It,” Los Angeles Times, December 30, 1977, A5.

2 Scott D. Stewart, John J. Neumann, Christopher R. Knittel, and Jeffrey Heilser, “Absence of Value: An Analysis of Investment Allocation Decisions by Institutional Plan Sponsors,” Financial Analysts Journal, Vol. 65, No. 6, November/December 2009, 34-51.

3 Definitions come from Webster’s Ninth New Collegiate Dictionary (Springfield, MA: Merriam-Webster, Inc., 1988). Note that many common phrases, like “you make your own luck,” “luck is what happens when preparation meets opportunity,” and “the harder I work, the luckier I get,” do not fit with our definition. In each of these cases, luck is conflated with skill. Think of luck as something in addition to skill. So, for example, books including Richard Wiseman, The Luck Factor: Changing Your Luck, Changing Your Life: The Four Essential Principles (New York: Miramax, 2003) are very entertaining but do not contribute to this discussion.

4 Annie Duke, “Testimony before the House Committee on the Judiciary,” November 14, 2007. 5 Christopher Chabris and Daniel Simons, The Invisible Gorilla: And Other Ways Our Intuitions Deceive Us (New York: Crown, 2010), 83.

6 Robert Mitchell, now with Rogerscasey Canada, tries to measure reversion to the mean by using cumulative sum (CUSUM) control charts. He emphasizes the slope of the CUSUM line, akin to measuring the rate of reversion to the mean. See Robert Mitchell, “Hiring and Firing Investment Managers,” Presentation at the IMCA Canadian Consultants Conference, May 14, 2007.

7 Jonathan Schaeffer, “Marion Tinsley: Human Perfection at Checkers?” in Richard J.

Nowakowski, ed. Games of No Chance (Cambridge, UK: Cambridge University Press, 1998), 115-118.

8 Amos Tversky and Daniel Kahneman, “Belief in the law of small numbers,” Psychological Bulletin, Vol. 76, No. 2., 1971, 105-110; also Nassim Nicholas Taleb, Fooled By Randomness: The Hidden Role of Chance in Life and in the Markets, 2nd Edition (New York: Thomson Texere, 2004), 64-68.

9 Jim Albert, “Comments on ‘Underestimating the Fog’,” By The Numbers, Vol. 15, No. 1, February 2005, 3-5.

10 Geoff Colvin, Talent is Overrated: What Really Separates World-Class Performers from Everybody Else (New York: Portfolio, 2008), 65-72.

11 Michael Lewis, Moneyball: The Art of Winning an Unfair Game (New York: W.W. Norton & Company, 2003), 274.

12 For a more in-depth discussion, see Michael J. Mauboussin, “The Colonel Blotto Game: Transitivity and How to Win When You’re an Underdog,” Mauboussin on Strategy, May 5, 2010. 13 For those of you with some free time, see: http://www.worldrps.com/how-to-beat-anyone-atrock-paper-scissors.

14 There are many examples of this. See Eugene F. Fama and Kenneth R. French, “Luck versus Skill in the Cross Section of Mutual Fund Returns,” Journal of Finance, Vol. 65, No. 5, October 2010; Richard Grinold and Ronald N. Kahn, Active Portfolio Management: A Quantitative Approach for Producing Superior Returns and Controlling Risk, Second Edition (New York: McGraw-Hill, 2000), 479-508; Robert L. Hagin, Investment Management: Portfolio Diversification, Risk, and Timing—Fact and Fiction (Hoboken, NJ: John Wiley & Sons, 2004), 169-178. For an analysis that considers two distributions, see Laurent Barras, Olivier Scaillet, and Russ Wermers, “False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas,” Journal of Finance, Vol. 65, No. 1, February 2010, 179-216. Some researchers blend the two: “The Need to Differentiate Between Noise and Skill: Too Many Fooled by Randomness?” Watson Wyatt Worldwide, March 2007.

15 See http://www.insidethebook.com/ee/index.php/site/comments/true_talent_levels_for_sports_leagues/. 16 Stephen Jay Gould, Full House: The Spread of Excellence from Plato to Darwin (New York: Harmony Books, 1996), 109.

17 Leighton Vaughan Williams, “Semi-strong and strong form information efficiency in betting markets,” in Information Efficiency in Financial and Betting Markets, Leighton Vaughan Williams,

ed. (Cambridge, UK: Cambridge University Press, 2005), 123-155. 18 James Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations (New York: Doubleday and Company, 2004); see also Michael J. Mauboussin, “Explaining the Wisdom of Crowds:

Applying the Logic of Diversity,” Mauboussin on Strategy, March 20, 2007.

19 Arthur DeVany, Hollywood Economics: How Extreme Uncertainty Shapes the Film Industry (New York: Routledge, 2004).

20 Matthew J. Salganik, Peter Sheridan Dodds, and Duncan J. Watts, “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market,” Science, February 10, 2006, 854- 856.

21 According to Wikipedia, “Tom Tango” is an alias for an unidentified expert in statistical analysis of sports. He is also co-author of The Book, an excellent reference for sabermetrics. In any case, his analysis is really interesting. You can see the discussion of variance and skill here: http://www.insidethebook.com/ee/index.php/site/article/true_talent_levels_for_sports_leagues/. In David J. Berri, Martin B. Schmidt, and Stacey L. Brook, The Wages of Wins: Taking Measure of the Many Myths in Modern Sport (Stanford, CA: Stanford University Books, 2006) the authors refer to this approach as the “Noll-Scully measure” based on the work of the economists Roger Noll and Gerald Scully. See pages 45-63.

22 Berri, Schmidt, and Brook, 61-62.

23 Jim Albert, “A Batting Average: Does It Represent Ability or Luck?” Working Paper, April 17, 2004. See: http://bayes.bgsu.edu/papers/paper_bavg.pdf.

24 Jim Albert, “Comments on ‘Underestimating the Fog’,” By the Numbers, Vol. 15, No. 1, February, 2005. See: http://www.philbirnbaum.com/btn2005-02.pdf.

25 David J. Berri and Martin B. Schmidt, Stumbling on Wins: Two Economists Expose the Pitfalls on the Road to Victory in Professional Sports (Upper Saddle River, NJ: FT Press, 2010), 33-39. 26 K. J. Martijn Cremers and Antti Petajisto, “How Active Is Your Fund Manager? A New Measure That Predicts Performance,” Review of Financial Studies, Vol. 22, No. 9, September 2009, 3329- 3365.

27 Stephen Jay Gould, “The Streak of Streaks,” The New York Review of Books, Vol. 35, No. 13, August 18, 1988.

28 Paul J. Feltovich, Kenneth M. Ford, and Robert Hoffman, eds., Expertise in Context: Human and Machine (Menlo Park, CA and Cambridge, MA: AAAI Press and The MIT Press, 1997), 27. 29 John C. Bogle, Common Sense on Mutual Funds: Fully Updated 10th Anniversary Issue (Hoboken, NJ: John Wiley & Sons, 2010), 354-371. Bogle assumes a 2 percent maximum holding. For analysis of hedge funds, see Marco Avellaneda and Paul Besson, “Hedge-funds: How big is big?” Unpublished Manuscript, 2005.

30 Michael Seidel, Streak: Joe DiMaggio and the Summer of ’41 (New York: McGraw-Hill, 1988). 31 Samuel Arbesman and Steven H. Strogatz, “A Monte Carlo Approach to Joe DiMaggio and Streaks in Baseball,” arXiv:0807.5082, August 1, 2008.

32 Thomas C. Powell, “Varieties of Competitive Parity,” Strategic Management Journal, Vol. 24, No. 1, January 2003, 61-86.

33 Thomas C. Powell and Chris J. Lloyd, “Toward a General Theory of Competitive Dominance: Comments and Extensions on Powell (2003),” Strategic Management Journal, Vol. 26, No. 4, April 2005, 385-394.

34 We believe that return on assets is far from an ideal measure, but it probably does the job for large-scale studies.

35 Jerker Denrell, “Random Walks and Sustained Competitive Advantage,” Management Science, Vol. 50, No. 7, July 2004, 922-934.

36 Robert K. Merton, “The Matthew Effect in Science,” Science, Vol. 159, No. 3810, January 5, 1968, 56-63.

37 Andrew D. Henderson, Michael E. Raynor, and Mumtaz Ahmed, “How Long Must a Firm Be Great to Rule Out Luck? Benchmarking Sustained Superior Performance Without Being Fooled By Randomness,” Academy of Management Proceedings, August 2009, 1-6. For a shorter version, see Michael E. Raynor, Mumtaz Ahmed, and Andrew D. Henderson, “Are ‘Great’

Companies Just Lucky?” Harvard Business Review, April 2009, 18-19. Also, Robert R. Wiggins and Timothy W. Ruefli, “Sustained Competitive Advantage: Temporal Dynamics and the Incidence and Persistence of Superior Economic Performance,” Organizational Science, Vol. 13, No. 1, January-February 2002, 82-105.

38 See, for example, Phil Rosenzweig, The Halo Effect . . . and the Eight Other Business Delusions That Deceive Managers (New York: Free Press, 2007). Raynor, Ahmed, and Henderson estimate that only 30 of the 228 companies held out as “excellent” companies in the 11 most popular success studies were truly successful beyond what chance would dictate. See Michael E. Raynor, Mumtaz Ahmed, and Andrew D. Henderson, “Where Have You Gone, Joe DiMaggio? Just What is Really Great Business Performance?” Ivey Business Journal, May/June 2009.

39 Leonard Mlodinow, The Drunkard’s Walk: How Randomness Rules Our Lives (New York: Pantheon Books, 2008), 180-181; also Leonard Mlodinow, “The Triumph of the Random,” The Wall Street Journal, July 16, 2009.

40 2009 Investment Company Fact Book, 110-114.

http://www.icifactbook.org/pdf/2009_factbook.pdf and John C. Bogle, The Little Book of Common Sense Investing: The Only Way to Guarantee Your Fair Share of Stock Market Returns (Hoboken, NJ: John Wiley & Sons, 2007), 30.

41 Andrew Mauboussin and Samuel Arbesman, “Differentiating Skill and Luck in Financial Markets,” Working Paper, July 2010.

42 Klaas P. Baks, Andrew Metrick, and Jessica Wachter, “Should Mutual Fund Investors Avoid All Actively Managed Funds? A Study in Bayesian Performance Evaluation,” Journal of Finance, Vol. 56, No. 1, February 2001, 45-86; Robert Koswowski, Allan Timmerman, Russ Wermers, and Hal White, “Can Mutual Fund ‘Stars’ Really Pick Stocks? New Evidence from a Bootstrap Analysis,”

Journal of Finance, Vol. 61, No. 6, December 2006, 2551-2595; Dirk Nitzsche, Keith Cuthbertson, and Niall O'Sullivan, “Mutual Fund Performance,” SSRN Working Paper, February 16, 2007; Barras, Scaillet, and Wermers; and Fama and French.

43 Horace Secrist, The Triumph of Mediocrity in Business (Evanston, IL: Bureau of Business Research, Northwestern University, 1933).

44 Bruce Greenwald and Judd Kahn, Competition Demystified: A Radically Simplified Approach to Business Strategy (New York: Portfolio, 2005), 157-159; also Rob Walker, “Not Necessarily Toast,” The New York Times, April 8, 2007.

45 Robert R. Wiggins and Timothy W. Ruefli, ”Schumpeter’s Ghost: Is Hypercompetition Making the Best of Times Shorter?” Strategic Management Journal, Vol. 26, No. 10, October 2005, 887- 911.

46 Bartley J. Madden, CFROI Valuation: A Total System Approach to Valuing the Firm (Oxford, UK: Butterworth-Heinemann, 1999), 165-167.

47 Bogle (2010), 305-328.

48 Eero J. Pätäri, “Do Hot Hands Warm the Mutual Fund Investor? The Myth of Performance Persistence Phenomenon,” International Research Journal of Finance and Economics, Vol. 34, December 2009, 117-139.

49 Werner F. M. De Bondt and Richard H. Thaler, “Anomalies: A Mean-Reverting Walk Down Wall Street,” Journal of Economic Perspectives, Vol. 3, No. 1, Winter 1989, 189-202.

50 Mark Grinblatt and Sheridan Titman, “The Persistence of Mutual Fund Performance,” Journal of Finance, Vol. 47, No. 5, December 1992, 1977-1984; Darryll Hendricks, Jayendu Patel, and Richard Zeckhauser, “Hot Hands in Mutual Funds: Short-Run Persistence of Relative Performance, 1974-1988,” Journal of Finance, Vol. 48, No. 1, March 1993, 93-129; Stephen J.

Brown and William N. Goetzmann, “Performance Persistence,” Journal of Finance, Vol. 50, No. 2, June 1995, 679-698; for a dissenting view, see Mark M. Carhart, “On the Persistence in Mutual Fund Performance,” Journal of Finance, Vol. 52, No. 1, March 1997, 57-82. Persistence tends to fade when researchers adjust fund returns for factors in stock returns. This, of course, leaves aside the possibility that the manager sought exposure to the factor.

51 Amit Goyal and Sunil Wahal, “The Selection and Termination of Investment Management Firms by Plan Sponsors,” Journal of Finance, Vol. 63, No. 4, August 2008, 1805-1847.

52 Stewart, Neumann, Knittel, and Heilser, 49.

53 Alfred Rappaport and Michael J. Mauboussin, Expectations Investing: Reading Stock Prices for Better Returns (Boston, MA: Harvard Business School Press, 2001).

54 Wayne L. Winston, Mathletics: How Gamblers, Managers, and Sports Enthusiasts Use Mathematics in Baseball, Basketball, and Football (Princeton, NJ: Princeton University Press, 2009), 229-232.

55 Jack Ewing, “What Economists Can Teach World Cup Coaches,” New York Times, July 6, 2010; Matt DaSilva, “Great X-pectations,” Lacrosse Magazine, October 2009, 11.

56 KC Joyner, Blindsided: Why the Left Tackle is Overrated and Other Contrarian Football Thoughts (Hoboken, NJ: John Wiley & Sons, 2008), 76-77.

57 Michael Lewis, “Coach Leach Goes Deep, Very Deep,” The New York Times, December 4, 2005.

58 Clayton M. Christensen, The Innovator’s Dilemma: When New Technologies Cause Great Companies to Fail (Boston: MA: Harvard Business School Press, 1997).

59 Clayton M. Christensen and Michael E. Raynor, The Innovator’s Solution: Creating and Sustaining Successful Growth (Boston, MA: Harvard Business School Press, 2003); also Clayton M. Christensen, Scott D. Anthony, and Erik A. Roth, Seeing What’s Next: Using the Theories of Innovation to Predict Industry Change (Boston, MA: Harvard Business School Press, 2004). 60 Michael J. Mauboussin, “The Colonel Blotto Game: Transitivity and How to Win When You’re an Underdog,” Mauboussin on Strategy, May 7, 2010.

61 Gerald P. Madden, Kenneth P. Nunn, Jr., and Alan Wiemann, “Mutual Fund Performance and Market Capitalization,” Financial Analysts Journal, Vol. 42, No. 4, July-August 1986, 67-70. 62 Researchers have naturally sought to control for this. The best known solution is the Fama-French three-factor model. The first factor is classic exposure to the market as expressed through the capital asset pricing model. The second factor is small cap versus large cap stocks (called SML, or “small minus large”) and the final factor is value versus growth (HML, or “high book-toprice ratio minus low book-to-price ratio”). Another factor, momentum, is also used. While these are considered “beta” factors, which dismiss the possibility of skill, it is fair to ask whether they fairly assess skill. See the seminal paper, Eugene F. Fama and Kenneth R. French, “The Cross-Section of Expected Returns,” Journal of Finance, Vol. 47, No. 2, June 1992, 427-465.

63 Peter L. Bernstein, “Where, Oh Where Are the .400 Hitters of Yesteryear?” Financial Analysts Journal, Vol. 54, No. 6, November-December 1998, 6-14.

64 Joel Chernoff, “Bernstein acknowledges the .400 hitters,” Pensions & Investments, November 15, 2004.

65 Paul A. Samuelson, “Challenge to Judgment,” The Journal of Portfolio Management, Vol. 1, No. 1, Fall 1974, 17-19. Samuelson was well-known for advocating efficient markets. See Paul A.

Samuelson, “Proof That Properly Anticipated Prices Fluctuate Randomly,” Industrial Management Review, Vol. 6, No. 2, Spring 1965, 41-49. Samuelson also owned shares of Berkshire Hathaway, run by the legendary investor, Warren Buffett. So he hedged his bet.

66 Steven Crist, “Crist on Value,” in Beyer, et al., Bet with the Best (New York: Daily Racing Form Press, 2001), 64.

67 Dan Lovallo and Daniel Kahneman, “Delusions of Success,” Harvard Business Review, July 2003, 56-63.

68 Benjamin Graham, The Intelligent Investor: A Book of Practical Counsel, Fourth Revised Edition (New York: Harper & Row, 1973), 281.

69 Michael J. Mauboussin, “Size Matters: The Kelly Criterion and the Importance of Money Management,” Mauboussin on Strategy, February 1, 2006.

70 Scott Patterson, “Old Pros Size Up the Game,” The Wall Street Journal, March 22, 2008. 71 From Seth Klarman’s speech at Columbia Business School on October 2, 2008. Reproduced in Outstanding Investor Digest, Vol. 22, No. 1 & 2, March 17, 2009, 3.

72 Graham, 287.

73 Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, NJ: Princeton University Press, 2005), 85.

74 Max H. Bazerman and Don Moore, Judgment in Managerial Decision Making, 7th Edition (Hoboken, NJ: John Wiley & Sons, 2009), 13-41.

75 Graham, 109.

76 David F. Swensen, Unconventional Success: A Fundamental Approach to Personal Investment (New York: Free Press, 2005), 220-222.

77 Charles D. Ellis, “Will Business Success Spoil the Investment Management Profession?” The Journal of Portfolio Management, Vol. 27, No. 3, Spring 2001, 11-15.

78 Cremers and Petajisto.

79 Warren E. Buffett, “The Superinvestors of Graham-and-Doddsville,” Hermes, 1984.

References

Books

Berri, David J., Martin B. Schmidt, and Stacey L. Brook, The Wages of Wins: Taking Measure of the Many Myths in Modern Sport (Stanford, CA: Stanford University Books, 2006).

Berri, David J., and Martin B. Schmidt, Stumbling on Wins: Two Economists Expose the Pitfalls on the Road to Victory in Professional Sports (Upper Saddle River, NJ: FT Press, 2010).

Bogle, John C., The Little Book of Common Sense Investing: The Only Way to Guarantee Your Fair Share of Stock Market Returns (Hoboken, NJ: John Wiley & Sons, 2007).

_____., Common Sense on Mutual Funds: Fully Updated 10th Anniversary Issue (Hoboken, NJ: John Wiley & Sons, 2010).

Chabris, Christopher, and Daniel Simons, The Invisible Gorilla: And Other Ways Our Intuitions Deceive Us (New York: Crown, 2010).

Christensen, Clayton M., The Innovator’s Dilemma: When New Technologies Cause Great Companies to Fail (Boston: MA: Harvard Business School Press, 1997).

Christensen, Clayton M., and Michael E. Raynor, The Innovator’s Solution: Creating and Sustaining Successful Growth (Boston, MA: Harvard Business School Press, 2003).

Christensen, Clayton M., Scott D. Anthony, and Erik A. Roth, Seeing What’s Next: Using the Theories of Innovation to Predict Industry Change (Boston, MA: Harvard Business School Press, 2004).

Colvin, Geoff, Talent is Overrated: What Really Separates World-Class Performers from Everybody Else (New York: Portfolio, 2008).

Crist, Steven, “Crist on Value,” in Beyer, et al., Bet with the Best (New York: Daily Racing Form Press, 2001).

DeVany, Arthur, Hollywood Economics: How Extreme Uncertainty Shapes the Film Industry (New York: Routledge, 2004).

Feltovich, Paul J., Kenneth M. Ford, and Robert Hoffman, eds., Expertise in Context: Human and Machine (Menlo Park, CA and Cambridge, MA: AAAI Press and The MIT Press, 1997).

Gould, Stephen Jay, Full House: The Spread of Excellence from Plato to Darwin (New York: Harmony Books, 1996).

Graham, Benjamin, The Intelligent Investor: A Book of Practical Counsel, Fourth Revised Edition (New York: Harper & Row, 1973).

Greenwald, Bruce, and Judd Kahn, Competition Demystified: A Radically Simplified Approach to Business Strategy (New York: Portfolio, 2005).

Grinold, Richard, and Ronald N. Kahn, Active Portfolio Management: A Quantitative Approach for Producing Superior Returns and Controlling Risk, Second Edition (New York: McGraw-Hill, 2000).

Hagin, Robert L., Investment Management: Portfolio Diversification, Risk, and Timing—Fact and Fiction (Hoboken, NJ: John Wiley & Sons, 2004).

Lewis, Michael, Moneyball: The Art of Winning an Unfair Game (New York: W.W. Norton & Company, 2003).

Madden, Bartley J., CFROI Valuation: A Total System Approach to Valuing the Firm (Oxford, UK: Butterworth-Heinemann, 1999).

Mlodinow, Leonard, The Drunkard’s Walk: How Randomness Rules Our Lives (New York: Pantheon Books, 2008).

Rappaport, Alfred, and Michael J. Mauboussin, Expectations Investing: Reading Stock Prices for Better Returns (Boston, MA: Harvard Business School Press, 2001).

Rosenzweig, Phil, The Halo Effect . . . and the Eight Other Business Delusions The Deceive Managers (New York: Free Press, 2007).

Schaeffer, Jonathan, “Marion Tinsley: Human Perfection at Checkers?” in Richard J.

Nowakowski, ed. Games of No Chance (Cambridge, UK: Cambridge University Press, 1998).

Secrist, Horace, The Triumph of Mediocrity in Business (Evanston, IL: Bureau of Business Research, Northwestern University, 1933).

Seidel, Michael, Streak: Joe DiMaggio and the Summer of ’41 (New York: McGraw-Hill, 1988).

Surowiecki, James, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations (New York: Doubleday and Company, 2004).

Swensen, David F., Unconventional Success: A Fundamental Approach to Personal Investment (New York: Free Press, 2005).

Taleb, Nassim Nicholas, Fooled By Randomness: The Hidden Role of Chance in Life and in the Markets, 2nd Edition (New York: Thomson Texere, 2004).

Tetlock, Philip E., Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, NJ: Princeton University Press, 2005).

Vaughan Williams, Leighton, “Semi-strong and strong form information efficiency in betting markets,” in Information Efficiency in Financial and Betting Markets, Leighton Vaughan Williams,

ed. (Cambridge, UK: Cambridge University Press, 2005).

Winston, Wayne L., Mathletics: How Gamblers, Managers, and Sports Enthusiasts Use Mathematics in Baseball, Basketball, and Football (Princeton, NJ: Princeton University Press, 2009).

Wiseman, Richard, The Luck Factor: Changing Your Luck, Changing Your Life: The Four Essential Principles (New York: Miramax, 2003).

Articles and papers

Albert, Jim, “A Batting Average: Does It Represent Ability or Luck?” Working Paper, April 17, 2004.

_____., “Comments on ‘Underestimating the Fog’,” By the Numbers, Vol. 15, No. 1, February, 2005.

Arbesman, Samuel, and Steven H. Strogatz, “A Monte Carlo Approach to Joe DiMaggio and Streaks in Baseball,” arXiv:0807.5082, August 1, 2008.

Avellaneda, Marco, and Paul Besson, “Hedge-funds: How big is big?” Unpublished Manuscript, 2005.

Barras, Laurent, Olivier Scaillet, and Russ Wermers, “False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas, Journal of Finance, Vol. 65, No. 1, February 2010, 179-216.

Bernstein, Peter L., “Where, Oh Where Are the .400 Hitters of Yesteryear?” Financial Analysts Journal, Vol. 54, No. 6, November- December 1998, 6-14.

Brown, Stephen J., and William N. Goetzmann, “Performance Persistence,” Journal of Finance, Vol. 50, No. 2, June 1995, 679-698.

Buffett, Warren E., “The Superinvestors of Graham-and-Doddsville,” Hermes, 1984.

Carhart, Mark M., “On the Persistence in Mutual Fund Performance,” Journal of Finance, Vol. 52, No. 1, March 1997, 57-82.

Chernoff, Joel, “Bernstein acknowledges the .400 hitters,” Pensions & Investments, November 15, 2004.

Cremers, K. J. Martijn, and Antti Petajisto, “How Active Is Your Fund Manager? A New Measure That Predicts Performance,” Review of Financial Studies, Vol. 22, No. 9, September 2009, 3329- 3365.

DaSilva, Matt, “Great X-pectations,” Lacrosse Magazine, October 2009, 11.

De Bondt, Werner F. M., and Richard H. Thaler, “Anomalies: A Mean-Reverting Walk Down Wall Street,” Journal of Economic Perspectives, Vol. 3, No. 1, Winter 1989, 189-202.

Denrell, Jerker, “Random Walks and Sustained Competitive Advantage,” Management Science, Vol. 50, No. 7, July 2004, 922-934.

Duke, Annie, “Testimony before the House Committee on the Judiciary,” November 14, 2007.

Ellis, Charles D., “Will Business Success Spoil the Investment Management Profession?” The Journal of Portfolio Management, Vol. 27, No. 3, Spring 2001, 11-15.

Ewing, Jack, “What Economists Can Teach World Cup Coaches,” New York Times, July 6, 2010.

Fama, Eugene F., and Kenneth R. French, “The Cross-Section of Expected Returns,” Journal of Finance, Vol. 47, No. 2, June 1992, 427-465.

_____., “Luck versus Skill in the Cross Section of Mutual Fund Returns,” Journal of Finance, Vol. 65, No. 5, October 2010.

Gould, Stephen Jay, “The Streak of Streaks,” The New York Review of Books, Vol. 35, No. 13, August 18, 1988.

Goyal, Amit, and Sunil Wahal, “The Selection and Termination of Investment Management Firms by Plan Sponsors,” Journal of Finance, Vol. 63, No. 4, August 2008, 1805-1847.

Grinblatt, Mark, and Sheridan Titman, “The Persistence of Mutual Fund Performance,” Journal of Finance, Vol. 47, No. 5, December 1992, 1977-1984.

Henderson, Andrew D., Michael E. Raynor, and Mumtaz Ahmed, “How Long Must a Firm Be Great to Rule Out Luck? Benchmarking Sustained Superior Performance Without Being Fooled By Randomness,” Academy of Management Proceedings, August 2009, 1-6.

Hendricks, Darryll, Jayendu Patel, and Richard Zeckhauser, “Hot Hands in Mutual Funds: Short-Run Persistence of Relative Performance, 1974-1988,” Journal of Finance, Vol. 48, No. 1, March 1993, 93-129.

Joyner, KC, Blindsided: Why the Left Tackle is Overrated and Other Contrarian Football Thoughts (Hoboken, NJ: John Wiley & Sons, 2008).

Klaas P. Baks, Andrew Metrick, and Jessica Wachter, “Should Mutual Fund Investors Avoid All Actively Managed Funds? A Study in Bayesian Performance Evaluation,” Journal of Finance, Vol. 56, No. 1, February 2001.

Koswowski, Robert, Allan Timmerman, Russ Wermers, and Hal White, “Can Mutual Fund ‘Stars’

Really Pick Stocks? New Evidence from a Bootstrap Analysis,” Journal of Finance, Vol. 61, No. 6, December 2006, 2551-2595.

Lewis, Michael, “Coach Leach Goes Deep, Very Deep,” The New York Times, December 4, 2005.

Lovallo, Dan, and Daniel Kahneman, “Delusions of Success,” Harvard Business Review, July 2003, 56-63.

Madden, Gerald P., Kenneth P. Nunn, Jr., and Alan Wiemann, “Mutual Fund Performance and Market Capitalization,” Financial Analysts Journal, Vol. 42, No. 4, July-August 1986, 67-70.

Mauboussin, Andrew, and Samuel Arbesman, “Differentiating Skill and Luck in Financial Markets,” Working Paper, July 2010.

Mauboussin, Michael J., “Size Matters: The Kelly Criterion and the Importance of Money Management,” Mauboussin on Strategy, February 1, 2006.

_____., “Explaining the Wisdom of Crowds: Applying the Logic of Diversity,” Mauboussin on Strategy, March 20, 2007.

_____., “The Colonel Blotto Game: Transitivity and How to Win When You’re an Underdog,”

Mauboussin on Strategy, May 7, 2010.

Meisler, Stanley, “First in 1763: Spain Lottery—Not Even a War Stops It,” Los Angeles Times, December 30, 1977, A5.

Merton, Robert K., “The Matthew Effect in Science,” Science, Vol. 159, No. 3810, January 5, 1968, 56-63.

Mitchell, Robert, “Hiring and Firing Investment Managers,” Presentation at the IMCA Canadian Consultants Conference, May 14, 2007.

Mlodinow, Leonard, “The Triumph of the Random,” The Wall Street Journal, July 16, 2009.

Nitzsche, Dirk, Keith Cuthbertson, and Niall O'Sullivan, “Mutual Fund Performance,” SSRN Working Paper, February 16, 2007.

Pätäri, Eero J., “Do Hot Hands Warm the Mutual Fund Investor? The Myth of Performance Persistence Phenomenon,” International Research Journal of Finance and Economics, Vol. 34, December 2009, 117-139.

Patterson, Scott, “Old Pros Size Up the Game,” The Wall Street Journal, March 22, 2008.

Powell, Thomas C., “Varieties of Competitive Parity,” Strategic Management Journal, Vol. 24, No. 1, January 2003, 61-86.

Powell, Thomas C., and Chris J. Lloyd, “Toward a General Theory of Competitive Dominance: Comments and Extensions on Powell (2003),” Strategic Management Journal, Vol. 26, No. 4, April 2005, 385-394.

Raynor, Michael E., Mumtaz Ahmed, and Andrew D. Henderson, “Are ‘Great’ Companies Just Lucky?” Harvard Business Review, April 2009, 18-19.

_____., “Where Have You Gone, Joe DiMaggio? Just What is Really Great Business Performance?” Ivey Business Journal, May/June 2009.

Salganik, Matthew J., Peter Sheridan Dodds, and Duncan J. Watts, “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market,” Science, February 10, 2006, 854- 856.

Samuelson, Paul A., “Proof That Properly Anticipated Prices Fluctuate Randomly,” Industrial Management Review, Vol. 6, No. 2, Spring 1965, 41-49.

_____., “Challenge to Judgment,” The Journal of Portfolio Management, Vol. 1, No. 1, Fall 1974, 17-19.

Stewart, Scott D., John J. Neumann, Christopher R. Knittel, and Jeffrey Heilser, “Absence of Value: An Analysis of Investment Allocation Decisions by Institutional Plan Sponsors,” Financial Analysts Journal, Vol. 65, No. 6, November/December 2009, 34-51.

Tversky, Amos, and Daniel Kahneman, “Belief in the law of small numbers,” Psychological Bulletin, Vol. 76, No. 2., 1971, 105-110.

Walker, Rob, “Not Necessarily Toast,” The New York Times, April 8, 2007.

“The Need to Differentiate Between Noise and Skill: Too Many Fooled by Randomness?” Watson Wyatt Worldwide, March 2007.

Wiggins, Robert R., and Timothy W. Ruefli, “Sustained Competitive Advantage: Temporal Dynamics and the Incidence and Persistence of Superior Economic Performance,” Organizational Science, Vol. 13, No. 1, January-February 2002, 82-105.

_____.,”Schumpeter’s Ghost: Is Hypercompetition Making the Best of Times Shorter?” Strategic Management Journal, Vol. 26, No. 10, October 2005, 887-911.

Appendix A: Blending Distributions to Understand Outcomes in the National Football League

This analysis comes from Brian Burke, author of Advanced NFL Stats. This discussion came from a series that started here: http://www.advancednflstats.com/2007/08/luck-and-nfl-outcomes.html.

Burke starts by asking what the distribution of wins and losses would look like in a pure luck world (see Exhibit 15). A binomial, or coin-toss, model expresses this distribution. This is equivalent to assuming outcomes are the combination of draws from a normally-distributed luck urn and a skill urn filled with zeros. About 20 percent of the teams go 8-8, and the probability of going either winless or undefeated is extremely low.

Exhibit 15: NFL Win-Loss Records Assuming a Coin-Toss Model

Source: Brian Burke, Advanced NFL Stats.

He then looks at the actual results for all NFL teams over the 2002-2005 seasons (see Exhibit 16). It’s clear that the distribution isn’t the same as what the binomial model generates.

Exhibit 16: Actual NFL Win-Loss Records (2002-2005)

Source: Brian Burke, Advanced NFL Stats.

The next picture provides a contrast between the random model and the empirical results (see Exhibit 17). You can see that the middle of the empirical results is lower than the random model and that there are more extreme events—teams winning or losing lots of games.

Exhibit 17: Comparison of Coin-Toss to Actual Win-Loss Records

Source: Brian Burke, Advanced NFL Stats.

Burke then turns to a pure-skill distribution (see Exhibit 18). Here he ranks all of the teams from #1 to #32, assumes that the higher-rated team always beats a lower-rated one, schedules games as they do in the NFL, and simulates the outcomes. Burke describes the distribution as “an inverted trapezoid.” The rise in undefeated and winless teams is a result of scheduling. For example, there may be some seasons when team #2 never faces team #1, so both teams go undefeated. The same holds for team #31 and team #32. This is equivalent to having a skill urn that ensures the better team always wins and a luck urn filled with zeros.

Exhibit 18: NFL Win-Loss Records Assuming Pure Skill

Source: Brian Burke, Advanced NFL Stats.

Next, he provides a contrast between the pure-skill model and the empirical results (see Exhibit 19). The pure-skill model is much too flat and assumes too many records with lots of wins or lots of losses.

Exhibit 19: Comparison of Pure-Skill to Actual Win-Loss Records

Source: Brian Burke, Advanced NFL Stats.

Finally, Burke blends the pure-skill and pure-luck models together until he gets a distribution that best fits the empirical results (see Exhibit 20). Adding the pure skill to the pure luck model helps stretch out the frequency of win-loss records, and adding the pure luck model to the pure skill model raises the middle of the distribution. Burke finds that the blend that best fits the empirical data is 52.5 percent luck. This is a little lower than the 58 percent we found because he measured the 2002-2005 seasons and we analyzed 2005-2009.

Exhibit 20: Blend of Skill and Luck that Best Fits the Empirical Results

Source: Brian Burke, Advanced NFL Stats.

Burke notes that this degree of randomness suggests NFL game prediction models to be accurate in the 75-80 percent range. He suggests that this is consistent with various computer models and oddsmakers.

Sports results naturally lend themselves to this type of analysis because both extremities are simple to specify. But the idea of comparing empirical results to simulated data is helpful in a number of other activities as well.

Appendix B: Placing the NBA on the Skill-Luck Continuum

→ variance(skill) = variance(observed) - variance(random)

→ random percentage = variance(random)/variance(observed)

variance(observed) 2.42% Last 5 regular seasons (average)

• variance(random) 0.30%

= variance(skill) 2.12%

random percentage 12.6%

Special thanks to Dan Callahan for his valuable contribution to all aspects of this project. He was crucial in data gathering, data analysis, visuals, and editing. He also provided useful comments and feedback on the text of the report.

The views expressed in this commentary reflect those of Legg Mason Capital Management (LMCM) as of the date of this commentary. These views are subject to change at any time based on market or other conditions, and LMCM disclaims any responsibility to update such views.

These views may not be relied upon as investment advice and, because investment decisions for clients of LMCM are based on numerous factors, may not be relied upon as an indication of trading intent on behalf of the firm. The information provided in this commentary should not be considered a recommendation by LMCM or any of its affiliates to purchase or sell any security. To the extent specific securities are mentioned in the commentary, they have been selected by the author on an objective basis to illustrate views expressed in the commentary. If specific securities are mentioned, they do not represent all of the securities purchased, sold or recommended for clients of LMCM and it should not be assumed that investments in such securities have been or will be profitable. There is no assurance that any security mentioned in the commentary has ever been, or will in the future be, recommended to clients of LMCM. Employees of LMCM and its affiliates may own securities referenced herein. Predictions are inherently limited and should not be relied upon as an indication of actual or future performance. Legg Mason Capital Management, Inc. consists of two legal entities, Legg Mason Capital Management and LMM LLC.