Why Some Sophisticated Investors Don't Buy Index Funds

Despite Being a Great Choice for Many Investors, Index Funds Aren't Perfect

Sophisticated Index Fund Investing
Many sophisticated investors with knowledge of accounting and finance, and don't have to pay others a management fee since they can do the work themselves, prefer a carefully assembled collection of long-term, low turnover, financially strong, directly-held stocks over index funds. Here are some of the reasons. Martin Barraud / Caiaimage / Getty Images

In an article I wrote about index funds, I reiterated for what must now be the ten-thousandth time:

In the years I've been writing about investing, I've repeatedly hammered home that the academic evidence is abundantly clear: For the typical investor, especially those who are new to investing in anything beyond certificates of deposit, the safest, best long-term way to build wealth through equities is to follow a simple formula, without fail, through bubbles and busts, inflation and deflation:

  1. Invest in passive index funds that have rock-bottom expense ratios, offer widespread diversification, and have low turnover
  2. Practice disciplined dollar cost averaging so you take advantage of market prices over time
  3. If you can afford to forego consumption, compound more money by reinvesting your dividends
  4. Take advantage of tax strategies and shelters, including asset placement or asset positioning.​

One of the more common questions I get from people is, "Why wouldn't someone invest through index funds if it's such a great deal?".  To understand the reasons, you have to understand that there are, primarily, three different types of index fund investors:

  1. Those who index because they understand the benefits, and prefer it despite its inherent limitations and flaws either for the sake of simplicity, peace of mind, lack of suitable alternatives (such as having to choose a mutual fund within a 401(k) plan), or a disinterest in analyzing individual securities.
  2. Those who don't think much about it and index because that is what they hear you are supposed to do.
  3. Those who have an ideological obsession with it, adhering to indexing in the same way the Pharisees of old adhered to the law, having "a form of godliness but denying its power".  They know indexing works, demand that everyone else index, but fairly often don't understand the underlying mechanics or the reasons it works despite believing themselves to be an expert.

    The first group includes men like Jack Bogle, the patron saint of index fund investing and the founder of Vanguard.  I've recommended his excellent books in the past and talked about how much I admire both his mission, his message, and his career of service to making the world a better, fairer place for small investors.

     His is a life well lived.  Millions of retirees owe their financial success to his innovation and work and I think there are a lot of people in the world who ignore his advice and could experience much more lucrative results if they would take his writings to heart.

    Bogle is both intellectually honest and, in a very real sense, a scholar.  He sought to find a better way to invest and identified certain characteristics that, when combined, led to high probabilities of good long-term results.  These included things such as:

    The index fund, one of Bogle's many contributions to the world, was his prescribed mechanism to harness these forces for the average investor.

     There was nothing inherently magic about the index fund itself - it is just a regular mutual fund that bought stocks based upon a pre-determined set of rules regardless of market conditions - except that it removed the need for the retail investor's judgment as pertaining to selecting individual securities.  It also served as a psychological barrier between emotional investors, many of whom would freak out in their inexperience after seeing shares of Google or Procter & Gamble fall by 30%.  (It's shocking but a lot of people who own index funds seem completely oblivious to the actual companies they own in that index fund - if you doubt it, go ask someone with their entire net worth in an S&P 500 index fund what their top 20 stocks are.  Odds are, they can't tell you.  Yet those 20 stocks, and those 20 stocks alone represent nearly 30% of their assets!)

    The other advantage of the index fund was that, because it didn't require judgment, it wasn't necessary to pay someone to run the money.  (In a lot of cases, actively managed mutual funds do just fine compared to index funds, only falling behind once you factor in the fees that are used to compensate the portfolio managers.)

    Many successful investors throughout history have built a lot of wealth using the exact same underlying principles that Bogle identified through his research with one exception: They substituted valuation of individual securities with regular purchases, using skill sets in accounting to value stocks like they would private businessesreal estate, or bonds.  These people are not stock traders.  They are not actively chasing hot shares that rise, living like a stereotype out of a movie from the 1980's.  They are the Anne Scheibers of the world, leaving behind $22 million.  They are the Jack MacDonald's of the world, leaving behind $188 million.  These are the dairy farmers of the world with eight-figure net worths who keep their wealth a secret from even their children.  These are the William Ruanes of the world; the Charlie Mungers of the world.

    People like this spend years putting together a portfolio of companies in which they held an ownership stake, treating public stocks as they would any other asset acquired for their family.  Typically, these investors were well educated, affluent, and, in many cases, entrepreneurial themselves, not hesitating to start a business if they saw a chance to make money.  (There are always exceptions.  History has uncovered plenty of school teachers, janitors, and plumbers who fall into this category after developing a passionate love of finance that leads them to an MBA-level understanding of the income statement and balance sheet and / or the tenacity to continually purchase an ownership stake in a great company they understand.)

    For the type of sophisticated investors who harnessed the same benefits index funds offered without the index fund methodology, buying an index fund as a matter of course was, and is, irrational for many reasons.

    1. A Sophisticated Investor Might Not Want to Buy an Index Fund Because He or She May Not Want to Purchase Overvalued Assets

    As I explained once on my personal blog, imagine the year is 2001.  At that time, you could have parked your money in a 30-year Treasury bond, backed by the sovereign taxing power of the United States government, and earned 5.46% interest on your money each year.  With no risk of default, any intelligent investor was going to demand at least 5.46% + a risk premium + an inflation modification factor to justify giving up what is considered (academically) to be the safest investment in the world.  That meant a hurdle rate of no less than 11.5%, once factoring in growth, before parting with cash for an alternative asset.

    At the time, Wal-Mart Stores, Inc. was trading at $58.75 per share.  It earned $1.49 per share after taxes.  That means an investor was "buying" an earnings yield of only 2.54% after taxes, and that didn't even include the dividend taxes that would have been owed to the Federal, state, and in some cases, local governments had the earnings been distributed to owners!  Such an insane valuation would have made sense if Wal-Mart were a fast-growing start-up rolling out locations across the country but this was a firm that was literally the largest retailer on planet Earth at the time.  Owners were all but guaranteed to earn a mere 2.54% + the growth rate in profits, adjusted by changes in the valuation multiple (Bogle refers to this as the "speculative return" factor).  Far from enjoying a positive return, even keeping up with the inflation rate was going to prove difficult.

    Under no condition would an intelligent businessman or businesswoman with an accounting background have wanted to purchase shares of Wal-Mart at that time, at that price.  This had nothing to do with market timing - he or she would not have been making a prediction about whether Wal-Mart stock would be higher or lower tomorrow, next week, or even next year because such things are unknowable - it was based solely upon the mathematics of private ownership in many ways identical to how a restauranteur determines whether he wants to risk his money to open another restaurant at a given location or a real estate investor decides whether she should acquire a building at a certain price based upon the probable future rents.

    An index fund investor, meanwhile, would have seen several percentage points of his or her index fund purchases allocated to Wal-Mart shares.  For a typical worker with no accounting background, or someone who doesn't much care as they figure it will work out in the long-run, that is fine.  Why?  With enough time, dollar cost averaging, reinvested dividends, and tax shelters, it should end up okay as insane prices on the top-end are made up for by depressing lows during collapses.  For a successful manufacturing plant owner with a multi-million dollar net worth who evaluates finances all the time, or a tax attorney who is comfortable making adjustments to GAAP and determining how much money an enterprise is really earning, it's completely unnecessary.

    Should those knowledgeable investors pretend to be ignorant?  Should they be willing to buy an asset they have no desire to own at a price they think is patently stupid (and they would never pay for a private business) simply because everyone says they should?  Should they then pay an on-going mutual fund expense ratio - even though it is rock bottom the cost is still not zero - to hold said asset?  For the third group of index fund investors - the ideologically obsessed - the answer they demand, against all logic and common sense, is "yes".  Because many of them have no idea what an index fund actually is, they adhere to it with the fervor that would make the old time religion itself proud, exhibiting a form of greedy reductionism; click-whirl responses without thought.

    This isn't necessarily a bad thing.  Frankly, sophisticated investors should encourage this behavior because the type of person who is likely to give into that sort of mistake due to a lack of thought is the very type of person who needs indexing the most.  It creates this paradox where trying to inform them of their misunderstanding of both Bogle's work and the mathematics involved is likely to cause them tremendous financial damage.  (For heaven's sake, a significant portion of Bogle's argument rests upon the presence of higher than normal management expenses, none of which apply to sophisticated investors capable of making their own allocation decisions, which the zealots fail to acknowledge!)  Instead, they should be encouraged and their faith in indexing reinforced.

    This is the reason I, personally, repeat it often and from every pulpit I have: If you don't know what you are doing, buy an index fund.  If you know what you are doing but you don't want to be bothered, buy an index fund.  If you even have to question what you are doing, buy an index fund.

    2. A Sophisticated Investor Might Not Want to Buy an Index Fund Because He or She Might Not Want Exposure to Certain Areas of the Economy to Which His or Her Fortune Is Already Tied

    Two of my family's private businesses involve a lot of technology.  If there were ever a cyber attack on the United States, or some other horrible event that knocked out the telecommunications infrastructure, the cash flows would be impaired, perhaps for weeks or even months.  To diversify away from that possibility, I, personally, don't want to own very many technology firms, all of which would likely be hit just as hard under such adverse circumstances.  From our companies to our personal retirement accounts, to the custodial accounts we gift to the younger generations of our family, neither my husband nor I are in the practice of loading up on software or hardware firms.  There's too much-correlated risk.  Our personal situation is very different from the average family.  That must be taken into account when allocating our wealth.  There is no one-size-fits-all when it comes to money management.

    As a result, our portfolio contains almost no tech firms.  Even if this means we were to compound at a lower rate (it certainly hasn't worked out that way throughout the past couple of decades, but for the sake of argument, let's say it were to happen), we'd still choose to behave in such a manner because it allows us to sleep better at night.  If I found a specific technology firm that interested me, I may buy shares, but it would be an exception to the rule.

    This is basic risk management; something you must learn to do once your estate reaches a certain size if you want to avoid waking up to find your family in a ruined.  Perhaps I read too many Charles Dickens or Jane Austen novels in school but that's not the sort of outcome one should relish.  As the saying goes, "you only have to get rich once".  After you have your basic net worth built, protecting it is more important than the absolute rate of return you earn on your assets.

    3. A Sophisticated Investor Might Not Want to Buy an Index Fund Because He or She Might Want Even Better Tax Efficiency

    Let's imagine you wanted to mimic the S&P 500 or Dow Jones Industrial Averages.  Instead of buying a mutual fund run on an index mandate, anyone with a decent-sized portfolio could just as easily build the index directly themselves by purchasing the underlying shares of stock that make up the index.  This would allow advanced tax harvesting techniques, keeping more money in your pocket come April 15th when the bills to the IRS are due.

    John Bogle himself explains this in his books on indexing, writing, "Holding individual stocks for the long term may not only be wise, but be far more tax-efficient".  I once provided a model portfolio for what this would look like in the real world by using the DJIA as a template.

    4. A Sophisticated Investor Might Not Like the Methodology of the Major Index Funds

    When you buy an index fund, you are outsourcing your thinking to someone else.  In the case of the Dow Jones Industrial Average, it is the editorial board of The Wall Street Journal, which determines the components in the world's most famous stock market index.  In the case of the S&P 500, it's other investors who set the prices of common stocks by buying or selling them directly, determining the market capitalization of each firm.  Both still involve human judgment, just not yours.

    What makes this somewhat untenable to sophisticated investors is that index funds ranked by market capitalization (e.g., the S&P 500) are structured in the stupidest way possible.  It means you buy more shares of a certain stock as the price gets more expensive, and sell shares as it gets less expensive; literally buying high and selling low, or the exact opposite of how one should conduct his or her investing affairs.  Unfortunately, it's the only feasible mechanism because the assets invested in index funds are so massive, it makes fundamental or equal weightings impossible to achieve.  There is no way Vanguard could have an S&P 500 equally weighted index fund with its current asset base.  It's not possible.  A sophisticated investor could construct one directly, though.

    There Are Several Reasons a Sophisticated Investor Might Use an Index Fund

    There are many situations, reasons, or circumstances in or under which a sophisticated investor might turn to index funds.  In some cases, he or she might be at a significant disadvantage in understanding a particular sector.  For example, a well-heeled banker might be able to evaluate individual bank stocks, and choose investments wisely in his own industry, but have no idea how to analyze pharmaceutical companies.  In such a case, buying a low-cost pharmaceutical index fund might turn out to be the wisest course of action.

    In other cases, he may not want to deal with multiple currencies, paperwork, or different accounting standards arising from foreign investments.  The other day, someone on my personal blog wrote about how he had purchased more shares of the Vanguard FTSE Europe ETF (ticker symbol VGK).  For an American citizen trying to get his or her hands on shares of some of Europe's greatest companies, it's one of the best index funds out there.  With a single purchase, you buy a basket of blue chip stocks from across the pond, including incredible names such as Nestle, Novartis, Royal Dutch Shell, BP, and Total.  If I were to buy a European index fund as opposed to valuing and purchasing individual companies for long-term ownership, it would be on the top of my list.

    A sophisticated investor might want to not worry about his family after he or she has died.  Warren Buffett himself has said that upon his passing, his wife will be left with the bulk of her fortune invested 90% in S&P 500 index funds and 10% cash despite the fact that throughout his life, he has said he's never personally owned a single mutual fund.

    None of this changes the reality that index funds remain the single best way for most people to take advantage of equity ownership, particularly if they are neither interested nor capable of evaluating individual businesses they might want to acquire.  Besides, there's nothing stopping someone from indexing a large percentage of their assets and keeping a bit on the side to buying into companies they want to own.  Who says you can't keep 90% of your holdings in low-cost index funds and routinely buy shares of Coca-Cola, Diageo, General Mills, or whatever other corporation in which you strongly believe?  It is your money, after all.