A Novel Way to Measure Investment Management Performance

Jon Corzine at MF Global. Fooled you with Randomness and then Fell on the Unlucky Bit of the Bell Curve?

I’m frequently puzzled by how simple aspects of statistical probability and analysis are merrily shunted aside when investors consider spending their money. For example, when investment performance is analyzed there is no shortage of metrics that are used to ascertain whether the investment manager has actually added anything from skill or from luck. In other words, is he generating alpha?

 The issues of how alpha is measured are beyond the scope of this review. In this post I want to focus on awareness of the processes of how we should analyze the decision making processes of investment managers before we get into relying on the metrics. One factor is sample size.

Sample Size Matters

To demonstrate this, I’m going to ask the reader to play a game and try to answer the following question which is replicated from a Kahnemann & Tversky paper on behavioral heuristics. Please try to answer the question in the quoted text before reading further. The answers given in the research are displayed in the table below. The correct answers are starred.

A Certain town is served by two hospitals. In the larger hospital about 45 babies are born each day, and in the smaller hospital about 15 babies are born each day. As you know, about 50% of all babies are boys. The exact percentage of baby boys, however, varies from day to day. Sometimes it may be higher that 50%, sometimes lower.

For a period of 1 year, each hospital recorded the days on which (more/less) than 60% of the babies born were boys. Which hospital do you think recorded more such days?

	More than 60%	Less than 60%
The Larger Hospital	12	9*
The Smaller Hospital	10*	11
About the same (i.e., within 5% of each other)	28	25

Contrary to what most people answered, the smaller hospital is more likely to record the more extreme results because of the greater variance in results due to its smaller sample size. However, most people answer with the larger hospital because it appears to be more representative. Sample size matters!

It matters a lot. For example, when drugs progress through clinical trials the sample size of the trials gets larger because it will give more accurate results. A larger sample is always statistically more significant than a subset of that same sample.

So why is this principle lost when measuring investment performance?

Measuring Investment Management Performance

Having established the importance of sample size, its time to look at how this might affect the investment world. One example is over how different funds are managed. Consider a fund that involves a manager making a large number of decisions that replicate the same activity. This fund also has a within a remit of not exposing the underlying activity to some sort of overriding direction. For example, this could be a market neutral equity hedge fund that has a decent number of positions within it. Compare this with a macro based fund that tries to pick big trends.

The former involves a large sample size of decision making and the latter involves getting a few decisions right. I contend that the performance metrics of the former will give a more accurate depiction of skill than the latter. Moreover, in the act of making this larger number of decisions the former has more data and experience with which to practice and improve.

So what made me think of this?


Skill or Luck in Investment Performance?

Consider this from a recent Bloomberg article…

'Richard Maraviglia spent January flying to Zurich, New York, Chicago and Miami to raise $250 million for his hedge fund.

Maraviglia, who now oversees about $610 million for Carlson Capital LP from London, raised the money because the almost 40 percent gain he posted last year made him a rarity'

'Maraviglia…   … started reducing bullish stock wagers in the first half of 2011, correctly betting that policy makers in China would take steps to curb inflation and the U.S. Federal Reserve would end its program of buying $600 billion of Treasuries. After a flat second quarter, the MSCI World Index lost 17 percent in the third quarter.

He then bought equities when other investors sold positions en masse in September'

So essentially, this fund made two decisions in 2011 that led to a 40% performance. Is that enough of a sample size to be sure that he has the skill to replicate this performance in 2012?

He was on the bit of the bell curve of macro betters that was profitable last year, but investors should have little confidence (just based on 2011) that he will be there next year. In the end, does it matter? He’s managed to convince investors to give him another $250m.

Investors love chasing performance, they love being fooled by randomness, they love being sold a fund that is up 40%, they love focusing on the positive bit of the bell curve and they love ignoring sample size in gauging the reasons for extreme investment performance.

Investment Banking and Behavioral Finance

In a similar vein, consider the banking industry. Once upon a time, it used to engage in the iterative process of issuing commercial, industrial and housing loans. This is a constant ongoing process, by which banks can demonstrate skill and experience. Then along came the idea that they can make big strategic bets on sub prime CDO’s and the rest is history.

Banks should focus on the activities that they have a demonstrable track record of being good at, rather than big strategic bets that they have no idea over and can bring down the economy as a consequence. Consider, John Corzine at MF Global and his big macro bets on Euro Zone Sovereign Debt.

Source:

Bloomberg Article 'Small Hedge Funds Draw Investments as Bigger Rivals Stumble' accessed 18 Feb 2012

Kahneman & Tversky 'Subjective probability: A judgement of representativeness' in 'Judgement under Uncertainty', Cambridge University Press, 2005