It Is Nearly Impossible To Beat The Stock Market Over The Long Term, But You Should Still Try.

Although you will most likely not be able to beat the stock market over the long term, you should at least try.

‍

In honor of the 50th anniversary of the publication of "A Random Walk Down Wall Street," I am going to discuss how both of these statements can be true simultaneously. The Princeton University economist Burton Malkiel, who wrote this book, argued that stock price fluctuations are random and therefore unpredictable in his book, which quickly became a classic. There are some advisers who can beat the market on a short-term basis, but there is hardly anyone who will be able to do so consistently over the long term.

‍

Malkiel's argument is supported by an overwhelming amount of evidence, as I have said many times before in support of his argument. Among my recent columns, for instance, I mentioned that the percentage of mutual funds that have been able to beat the market over successive periods is close to what you would expect from pure randomness.

‍

According to Malkiel, we should invest all of our stock-market money in a broad index fund, and never trade in the stock market at all since that is the clear implication of his argument. The results of an under-appreciated study conducted two decades ago suggest that this implication follows only when one is completely certain that there will never be a manager who can beat the market in the long run.

‍

The study suggests that as soon as we begin to relax even the tiniest bit of that belief, it becomes irrational to not give a substantial chunk of our portfolio to those managers who have proven themselves over the long term as the best at beating the market.

‍

The purpose of study, entitled "Should investors avoid all actively managed mutual funds? ", has been published in the Journal of Finance in 2002, as a study in Bayesian performance evaluation. According to the study, Klaas Baks from Emory University, Andrew Metrick from Yale University, and Jessica Wachter from the Wharton School at the University of Pennsylvania conducted the study together. According to Baks, in an interview he gave me recently, his research findings remain valid despite more than two decades of additional research into the markets.

‍

Holding onto the 'hot hand'

‍

For active investment managers, we will be focusing on whether we have prior beliefs about whether there is a possibility of market beating and how we will update those beliefs after researching the track records of these managers over a longer period of time.

‍

They calculated, in a two-decade-old academic study, how much of our equity portfolio should be allocated to active managers who have had the best long-term record over the past two decades. This was achieved by beginning with a certain percentage allocation to active management, then calculating what our prior allocation should have been to be able to justify the new percentage allocation to active management, and then working backward from there.

‍

The researchers found that to justify a zero allocation to active managers, we would have to assume at the outset that market-beating ability doesn't exist in order to justify a zero allocation. Investing a good portion of our portfolio into those managers with the best long-term records becomes rational once we let go of that skepticism even by a tiny bit, due to the fact that it becomes more rational as soon as we let go of that skepticism.

‍

What is the minimum amount of change we must make in our beliefs? Imagine that you had a prior expectation that only one manager out of 10,000 was capable of beating the market. If this is the case, then according to the professors, it becomes rational to allocate approximately 25% of your stock portfolio to follow the manager you have chosen. Assuming that one out of 1,000 professionals is truly capable of beating the market over a long period of time, then you should allocate 50% of your time to following the manager with the strongest long-term performance record.

‍

As the professors point out, there is still not enough data on the track records of active managers to distinguish between a prior belief that no market-beating ability exists, and another prior belief that such an ability, while rare, does exist nonetheless. Due to the limited amount of data available, it is impossible to determine which prior to choosing between these two priors. There is insufficient statistical evidence to justify zero investment in active managers solely based on the statistical evidence provided by the professors in their article.

‍

So there you have it, you have all the information you need. While it may seem counterintuitive to say that it is extremely difficult to beat the market, it is rational to put a significant portion of your portfolio in the hands of the best-performing active managers, despite the fact that it is extremely difficult to beat the market. It reminds me of the famous quote from F. Scott Fitzgerald: "A first-rate intelligence is able to hold two opposing ideas in the mind at once and still function."